public inbox for [email protected]help / color / mirror / Atom feed
Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part 39+ messages / 6 participants [nested] [flat]
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2024-09-26 10:55 Alexander Korotkov <[email protected]> 0 siblings, 2 replies; 39+ messages in thread From: Alexander Korotkov @ 2024-09-26 10:55 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Florents Tselai <[email protected]>; pgsql-hackers <[email protected]> On Thu, Sep 26, 2024 at 12:04 AM Tom Lane <[email protected]> wrote: > Florents Tselai <[email protected]> writes: > > This patch is a follow-up and generalization to [0]. > > It adds the following jsonpath methods: lower, upper, initcap, l/r/btrim, > > replace, split_part. > > How are you going to deal with the fact that this makes jsonpath > operations not guaranteed immutable? (See commit cb599b9dd > for some context.) Those are all going to have behavior that's > dependent on the underlying locale. > > We have the kluge of having separate "_tz" functions to support > non-immutable datetime operations, but that way doesn't seem like > it's going to scale well to multiple sources of mutability. While inventing "_tz" functions I was thinking about jsonpath methods and operators defined in standard then. Now I see huge interest on extending that. I wonder if we can introduce a notion of flexible mutability? Imagine that jsonb_path_query() function (and others) has another function which analyzes arguments and reports mutability. If jsonpath argument is constant and all methods inside are safe then jsonb_path_query() is immutable otherwise it is stable. I was thinking about that back working on jsonpath, but that time problem seemed too limited for this kind of solution. Now, it's possibly time to shake off the dust from this idea. What do you think? ------ Regards, Alexander Korotkov Supabase ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2024-09-26 12:59 Florents Tselai <[email protected]> parent: Alexander Korotkov <[email protected]> 1 sibling, 1 reply; 39+ messages in thread From: Florents Tselai @ 2024-09-26 12:59 UTC (permalink / raw) To: Alexander Korotkov <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers <[email protected]> On Thu, Sep 26, 2024 at 1:55 PM Alexander Korotkov <[email protected]> wrote: > On Thu, Sep 26, 2024 at 12:04 AM Tom Lane <[email protected]> wrote: > > Florents Tselai <[email protected]> writes: > > > This patch is a follow-up and generalization to [0]. > > > It adds the following jsonpath methods: lower, upper, initcap, > l/r/btrim, > > > replace, split_part. > > > > How are you going to deal with the fact that this makes jsonpath > > operations not guaranteed immutable? (See commit cb599b9dd > > for some context.) Those are all going to have behavior that's > > dependent on the underlying locale. > > > > We have the kluge of having separate "_tz" functions to support > > non-immutable datetime operations, but that way doesn't seem like > > it's going to scale well to multiple sources of mutability. > > While inventing "_tz" functions I was thinking about jsonpath methods > and operators defined in standard then. Now I see huge interest on > extending that. I wonder if we can introduce a notion of flexible > mutability? Imagine that jsonb_path_query() function (and others) has > another function which analyzes arguments and reports mutability. If > jsonpath argument is constant and all methods inside are safe then > jsonb_path_query() is immutable otherwise it is stable. I was > thinking about that back working on jsonpath, but that time problem > seemed too limited for this kind of solution. Now, it's possibly time > to shake off the dust from this idea. What do you think? > > ------ > Regards, > Alexander Korotkov > Supabase > In case you're having a deja vu, while researching this I did come across [0] where disussing this back in 2019. In this patch I've conveniently left jspIsMutable and jspIsMutableWalker untouched and under the rug, but for the few seconds I pondered over this,the best answer I came with was a simple heuristic to what Alexander says above: if all elements are safe, then the whole jsp is immutable. If we really want to tackle this and make jsonpath richer though, I don't think we can avoid being a little more flexible/explicit wrt mutability. Speaking of extensible: the jsonpath standard does mention function extensions [1] , so it looks like we're covered by the standard, and the mutability aspect is an implementation detail. No? And having said that, the whole jsonb/jsonpath parser/executor infrastructure is extremely powerful and kinda under-utilized if we use it "only" for jsonpath. Tbh, I can see it supporting more specific DSLs and even offering hooks for extensions. And I know for certain I'm not the only one thinking about this. See [2] for example where they've lifted, shifted and renamed the jsonb/jsonpath infra to build a separate language for graphs [0] https://www.postgresql.org/message-id/[email protected]... [1] https://www.rfc-editor.org/rfc/rfc9535.html#name-function-extensions [2] https://github.com/apache/age/blob/master/src/include/utils/agtype.h ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2024-09-27 09:45 David E. Wheeler <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2024-09-27 09:45 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]> On Sep 26, 2024, at 13:59, Florents Tselai <[email protected]> wrote: > Speaking of extensible: the jsonpath standard does mention function extensions [1] , > so it looks like we're covered by the standard, and the mutability aspect is an implementation detail. No? That’s not the standard used for Postgres jsonpath. Postgres follows the SQL/JSON standard in the SQL standard, which is not publicly available, but a few people on the list have copies they’ve purchased and so could provide some context. In a previous post I wondered if the SQL standard had some facility for function extensions, but I suspect not. Maybe in the next iteration? > And having said that, the whole jsonb/jsonpath parser/executor infrastructure is extremely powerful > and kinda under-utilized if we use it "only" for jsonpath. > Tbh, I can see it supporting more specific DSLs and even offering hooks for extensions. > And I know for certain I'm not the only one thinking about this. > See [2] for example where they've lifted, shifted and renamed the jsonb/jsonpath infra to build a separate language for graphs I’m all for extensibility, though jsonpath does need to continue to comply with the SQL standard. Do you have some idea of the sorts of hooks that would allow extension authors to use some of that underlying capability? Best, David ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2024-09-27 10:28 Florents Tselai <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 0 replies; 39+ messages in thread From: Florents Tselai @ 2024-09-27 10:28 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]> > On 27 Sep 2024, at 12:45 PM, David E. Wheeler <[email protected]> wrote: > > On Sep 26, 2024, at 13:59, Florents Tselai <[email protected]> wrote: > >> Speaking of extensible: the jsonpath standard does mention function extensions [1] , >> so it looks like we're covered by the standard, and the mutability aspect is an implementation detail. No? > > That’s not the standard used for Postgres jsonpath. Postgres follows the SQL/JSON standard in the SQL standard, which is not publicly available, but a few people on the list have copies they’ve purchased and so could provide some context. > > In a previous post I wondered if the SQL standard had some facility for function extensions, but I suspect not. Maybe in the next iteration? > >> And having said that, the whole jsonb/jsonpath parser/executor infrastructure is extremely powerful >> and kinda under-utilized if we use it "only" for jsonpath. >> Tbh, I can see it supporting more specific DSLs and even offering hooks for extensions. >> And I know for certain I'm not the only one thinking about this. >> See [2] for example where they've lifted, shifted and renamed the jsonb/jsonpath infra to build a separate language for graphs > > I’m all for extensibility, though jsonpath does need to continue to comply with the SQL standard. Do you have some idea of the sorts of hooks that would allow extension authors to use some of that underlying capability? Re-tracing what I had to do 1. Define a new JsonPathItemType jpiMyExtType and map it to a JsonPathKeyword 2. Add a new JsonPathKeyword and make the lexer and parser aware of that, 3. Tell the main executor executeItemOptUnwrapTarget what to do when the new type is matched. I think 1, 2 are the trickiest because they require hooks to jsonpath_scan.l and parser jsonpath_gram.y 3. is the meat of a potential hook, which would be something like extern JsonPathExecResult executeOnMyJsonpathItem(JsonPathExecContext *cxt, JsonbValue *jb, JsonValueList *found); This should be called by the main executor executeItemOptUnwrapTarget when it encounters case jpiMyExtType It looks like quite an endeavor, to be honest. ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-03-05 19:29 Florents Tselai <[email protected]> parent: Alexander Korotkov <[email protected]> 1 sibling, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-03-05 19:29 UTC (permalink / raw) To: Alexander Korotkov <[email protected]>; +Cc: Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On Thu, Sep 26, 2024 at 1:55 PM Alexander Korotkov <[email protected]> wrote: > On Thu, Sep 26, 2024 at 12:04 AM Tom Lane <[email protected]> wrote: > > Florents Tselai <[email protected]> writes: > > > This patch is a follow-up and generalization to [0]. > > > It adds the following jsonpath methods: lower, upper, initcap, > l/r/btrim, > > > replace, split_part. > > > > How are you going to deal with the fact that this makes jsonpath > > operations not guaranteed immutable? (See commit cb599b9dd > > for some context.) Those are all going to have behavior that's > > dependent on the underlying locale. > > > > We have the kluge of having separate "_tz" functions to support > > non-immutable datetime operations, but that way doesn't seem like > > it's going to scale well to multiple sources of mutability. > > While inventing "_tz" functions I was thinking about jsonpath methods > and operators defined in standard then. Now I see huge interest on > extending that. I wonder if we can introduce a notion of flexible > mutability? Imagine that jsonb_path_query() function (and others) has > another function which analyzes arguments and reports mutability. If > jsonpath argument is constant and all methods inside are safe then > jsonb_path_query() is immutable otherwise it is stable. I was > thinking about that back working on jsonpath, but that time problem > seemed too limited for this kind of solution. Now, it's possibly time > to shake off the dust from this idea. What do you think? > I was thinking about taking another stab at this. Would someone more versed in the inner workings of jsonpath like to weigh in on the immutability wrt locale? ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-09 19:50 Robert Haas <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 2 replies; 39+ messages in thread From: Robert Haas @ 2025-05-09 19:50 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On Wed, Mar 5, 2025 at 2:30 PM Florents Tselai <[email protected]> wrote: > I was thinking about taking another stab at this. > Would someone more versed in the inner workings of jsonpath like to weigh in on the immutability wrt locale? I'm not sure the issues with immutability here are particularly related to jsonpath -- I think they may just be general problems with our framework for immutability. I always struggle a bit to remember our policy on these issues -- to the best of my knowledge, we haven't documented it anywhere, and I think we probably should. I believe the way it works is that whenever a function depends on the operating system's timestamp or locale definitions, we decide it has to be stable, not immutable. We don't expect those things to be updated very often, but we know sometimes they do get updated. Now apparently what we've done for time zones is we have both json_path_exists and json_path_exists_tz, and the former only supports things that are truly immutable while the latter additionally supports things that depend on time zone, and are thus marked stable. I suppose we could just add support for these locale-dependent operations to the "tz" version and have them error out in the non-tz version. After all, the effect of depending on time zone is, as far as I know, the same as the effect of depending on locale: the function can't be immutable any more. The only real problem with that idea, at least to my knowledge, is that the function naming makes you think that it's just about time zones and not about anything else. Maybe that's a wart we can live with? Tom writes earlier in the thread that: # We have the kluge of having separate "_tz" functions to support # non-immutable datetime operations, but that way doesn't seem like # it's going to scale well to multiple sources of mutability. But I'm not sure I understand why it matters that there are multiple sources of mutability here. Maybe I'm missing a piece of the puzzle here. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-13 18:07 David E. Wheeler <[email protected]> parent: Robert Haas <[email protected]> 1 sibling, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-13 18:07 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Florents Tselai <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On May 9, 2025, at 15:50, Robert Haas <[email protected]> wrote: > # We have the kluge of having separate "_tz" functions to support > # non-immutable datetime operations, but that way doesn't seem like > # it's going to scale well to multiple sources of mutability. > > But I'm not sure I understand why it matters that there are multiple > sources of mutability here. Maybe I'm missing a piece of the puzzle > here. I read that to mean “we’re not going to add another json_path_exists_* function for every potentially immutable JSONPath function. But I take your point that it could be generalized for *any* mutable function. In which case maybe it should be renamed? Best, David Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-13 20:24 Florents Tselai <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-05-13 20:24 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> > On 13 May 2025, at 2:07 PM, David E. Wheeler <[email protected]> wrote: > > On May 9, 2025, at 15:50, Robert Haas <[email protected]> wrote: > >> # We have the kluge of having separate "_tz" functions to support >> # non-immutable datetime operations, but that way doesn't seem like >> # it's going to scale well to multiple sources of mutability. >> >> But I'm not sure I understand why it matters that there are multiple >> sources of mutability here. Maybe I'm missing a piece of the puzzle >> here. > > I read that to mean “we’re not going to add another json_path_exists_* function for every potentially immutable JSONPath function. But I take your point that it could be generalized for *any* mutable function. In which case maybe it should be renamed? > > Best, > > David > We discussed this a bit during the APFS: As Robert said—and I agree—renaming the existing _tz family would be more trouble than it’s worth, given the need for deprecations, migration paths, etc. If we were designing this today, suffixes like _stable or _volatile might have been more appropriate, but at this point, we’re better off staying consistent with the _tz family. So the path forward seems to be: - Put these new functions under the jsonb_path_*_tz family. - Raise an error if they’re used in the non-_tz versions. - Document this behavior clearly. I’ll make sure to follow the patterns in the existing _tz functions closely. Other thoughts and head’s up are, of course, welcome. Patch CF entry: https://commitfest.postgresql.org/patch/5270/ Last updated Sept 24, so it will also need a rebase to account for changes in jsonpath_scan.l. I’ll get to that shortly. ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-14 03:00 David E. Wheeler <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 2 replies; 39+ messages in thread From: David E. Wheeler @ 2025-05-14 03:00 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On May 13, 2025, at 16:24, Florents Tselai <[email protected]> wrote: > As Robert said—and I agree—renaming the existing _tz family would be more trouble than it’s worth, given the need for deprecations, migration paths, etc. If we were designing this today, suffixes like _stable or _volatile might have been more appropriate, but at this point, we’re better off staying consistent with the _tz family. I get the pragmatism, and don’t want to over-bike-shed, but what a wart to live with. [I just went back and re-read Robert’s post, and didn’t realize he used exactly the same expression!] Would it really be too effortful to create _stable or _volatile functions and leave the _tz functions as a sort of legacy? Or maybe there’s a nice backronym we could come up with for _tz. Best, David Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-14 15:33 Florents Tselai <[email protected]> parent: David E. Wheeler <[email protected]> 1 sibling, 0 replies; 39+ messages in thread From: Florents Tselai @ 2025-05-14 15:33 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> > On 13 May 2025, at 11:00 PM, David E. Wheeler <[email protected]> wrote: > > On May 13, 2025, at 16:24, Florents Tselai <[email protected]> wrote: > >> As Robert said—and I agree—renaming the existing _tz family would be more trouble than it’s worth, given the need for deprecations, migration paths, etc. If we were designing this today, suffixes like _stable or _volatile might have been more appropriate, but at this point, we’re better off staying consistent with the _tz family. > > I get the pragmatism, and don’t want to over-bike-shed, but what a wart to live with. [I just went back and re-read Robert’s post, and didn’t realize he used exactly the same expression!] Would it really be too effortful to create _stable or _volatile functions and leave the _tz functions as a sort of legacy? Thinking about it a second time, you may be right. Especially if more people are interested in adding even more methods there. Here’s a patch just merging the latest changes in the jsonpath tooling; no substantial changes to v1; mainly for CFbot to pick this up. Attachments: [application/octet-stream] v2-0001-Rebase-latest-changes.-jsonpath_scan.l-white-spac.patch (43.4K, 2-v2-0001-Rebase-latest-changes.-jsonpath_scan.l-white-spac.patch) download | inline diff: From c5c2fca8a5b480181a28233d4f3cd4baedbf2ba3 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Wed, 14 May 2025 10:47:07 -0400 Subject: [PATCH v2] Rebase latest changes. jsonpath_scan.l white spacing was a bit tricky. Need to check in detail --- src/backend/utils/adt/jsonpath.c | 176 ++++++++- src/backend/utils/adt/jsonpath_exec.c | 189 +++++++++ src/backend/utils/adt/jsonpath_gram.y | 71 +++- src/backend/utils/adt/jsonpath_scan.l | 41 ++ src/include/utils/jsonpath.h | 22 ++ src/test/regress/expected/jsonb_jsonpath.out | 393 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 72 ++++ src/test/regress/sql/jsonb_jsonpath.sql | 108 +++++ src/test/regress/sql/jsonpath.sql | 13 + 9 files changed, 1081 insertions(+), 4 deletions(-) diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..646c7b460a3 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -326,6 +326,38 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, *(int32 *) (buf->data + right) = chld - pos; } break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + { + { + /* + * First, reserve place for left/right arg's positions, then + * record both args and sets actual position in reserved + * places. + */ + int32 arg0 = reserveSpaceForItemPointer(buf); + int32 arg1 = reserveSpaceForItemPointer(buf); + + if (!item->value.method_args.arg0) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg0, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg0) = chld - pos; + + if (!item->value.method_args.arg1) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg1, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg1) = chld - pos; + } + } + break; case jpiLikeRegex: { int32 offs; @@ -362,6 +394,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +492,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +869,72 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiReplaceFunc: + appendStringInfoString(buf, ".replace("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPartFunc: + appendStringInfoString(buf, ".split_part("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrLowerFunc: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpperFunc: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcapFunc: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrimFunc: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrimFunc: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrimFunc: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +1010,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiReplaceFunc: + return "replace"; + case jpiStrLowerFunc: + return "lower"; + case jpiStrUpperFunc: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +1024,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrimFunc: + return "ltrim"; + case jpiStrRtrimFunc: + return "rtrim"; + case jpiStrBtrimFunc: + return "btrim"; + case jpiStrInitcapFunc: + return "initcap"; + case jpiStrSplitPartFunc: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1136,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; case jpiString: case jpiKey: @@ -1044,6 +1167,11 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + read_int32(v->content.method_args.arg0, base, pos); + read_int32(v->content.method_args.arg1, base, pos); + break; case jpiNot: case jpiIsUnknown: case jpiExists: @@ -1055,6 +1183,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1221,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1283,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiReplaceFunc || + v->type == jpiStrLowerFunc || + v->type == jpiStrUpperFunc || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc || + v->type == jpiStrInitcapFunc || + v->type == jpiStrSplitPartFunc); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1184,6 +1326,24 @@ jspGetLeftArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.left); } +void +jspGetArg0(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg0); +} + +void +jspGetArg1(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg1); +} + void jspGetRightArg(JsonPathItem *v, JsonPathItem *a) { @@ -1206,6 +1366,7 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.right); } + bool jspGetBool(JsonPathItem *v) { @@ -1227,7 +1388,8 @@ jspGetString(JsonPathItem *v, int32 *len) { Assert(v->type == jpiKey || v->type == jpiString || - v->type == jpiVariable); + v->type == jpiVariable || + v->type == jpiStringFunc); if (len) *len = v->content.value.datalen; @@ -1501,6 +1663,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiReplaceFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..51239121f18 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrimFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,176 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) { + Assert( jsp->type == jpiStrLowerFunc || + jsp->type == jpiStrUpperFunc || + jsp->type == jpiReplaceFunc || + jsp->type == jpiStrLtrimFunc || + jsp->type == jpiStrRtrimFunc || + jsp->type == jpiStrBtrimFunc || + jsp->type == jpiStrInitcapFunc || + jsp->type == jpiStrSplitPartFunc); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string value. The first argument to internal functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + switch (jsp->type) + { + case jpiStrLtrimFunc: + func = ltrim1; + break; + case jpiStrRtrimFunc: + func = rtrim1; + break; + case jpiStrBtrimFunc: + func = btrim1; + break; + default: ; + } + + if (jsp->content.arg) + { + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(" "))); + break; + } + + case jpiStrLowerFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpperFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcapFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiReplaceFunc: + { + char *from_str, *to_str; + int from_len, to_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPartFunc: + { + char *from_str; + Numeric n; + int from_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..c998c40cfbd 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -43,6 +43,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, JsonPathString *flags, JsonPathParseItem ** result, struct Node *escontext); +static JsonPathParseItem *makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); +static JsonPathParseItem *makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); + /* * Bison doesn't allocate anything that needs to live across parser calls, @@ -86,6 +89,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACEFUNC_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -94,8 +99,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, index_elem starts_with_initial expr_or_predicate datetime_template opt_datetime_template csv_elem datetime_precision opt_datetime_precision + str_method_arg_elem -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr csv_list opt_csv_list str_method_arg_list %type <indexs> index_list @@ -278,6 +284,32 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACEFUNC_P '(' str_method_arg_list ')' + { + if (list_length($4) == 2) + $$ = makeItemReplaceFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".replace() accepts two arguments."))); + } + | '.' STR_SPLIT_PART_P '(' str_method_arg_list ')' + { + if (list_length($4) == 2) + $$ = makeItemStrSplitPartFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".split_part() accepts two arguments."))); + } + | '.' STR_LTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrLtrimFunc, $4); } + | '.' STR_RTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrRtrimFunc, $4); } + | '.' STR_BTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrBtrimFunc, $4); } ; csv_elem: @@ -317,6 +349,15 @@ opt_datetime_template: | /* EMPTY */ { $$ = NULL; } ; +str_method_arg_elem: + STRING_P { $$ = makeItemString(&$1); } + | INT_P { $$ = makeItemNumeric(&$1); } + ; + +str_method_arg_list: + str_method_arg_elem { $$ = list_make1($1); } + | str_method_arg_list ',' str_method_arg_elem { $$ = lappend($1, $3); } + ; key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +398,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +417,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLowerFunc; } + | STR_UPPER_P { $$ = jpiStrUpperFunc; } + | STR_INITCAP_P { $$ = jpiStrInitcapFunc; } ; %% @@ -472,6 +519,28 @@ makeItemBinary(JsonPathItemType type, JsonPathParseItem *la, JsonPathParseItem * return v; } +static JsonPathParseItem * +makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiReplaceFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + +static JsonPathParseItem * +makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiStrSplitPartFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + static JsonPathParseItem * makeItemUnary(JsonPathItemType type, JsonPathParseItem *a) { diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..f20e9e5f1d3 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -433,6 +433,47 @@ static const JsonPathKeyword keywords[] = { {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, + { 2, false, IS_P, "is"}, + { 2, false, TO_P, "to"}, + { 3, false, ABS_P, "abs"}, + { 3, false, LAX_P, "lax"}, + { 4, false, DATE_P, "date"}, + { 4, false, FLAG_P, "flag"}, + { 4, false, LAST_P, "last"}, + { 4, true, NULL_P, "null"}, + { 4, false, SIZE_P, "size"}, + { 4, false, TIME_P, "time"}, + { 4, true, TRUE_P, "true"}, + { 4, false, TYPE_P, "type"}, + { 4, false, WITH_P, "with"}, + { 5, false, STR_BTRIM_P, "btrim"}, + { 5, true, FALSE_P, "false"}, + { 5, false, FLOOR_P, "floor"}, + { 5, false, STR_LOWER_P, "lower"}, + { 5, false, STR_LTRIM_P, "ltrim"}, + { 5, false, STR_RTRIM_P, "rtrim"}, + { 5, false, STR_UPPER_P, "upper"}, + { 6, false, BIGINT_P, "bigint"}, + { 6, false, DOUBLE_P, "double"}, + { 6, false, EXISTS_P, "exists"}, + { 6, false, NUMBER_P, "number"}, + { 6, false, STARTS_P, "starts"}, + { 6, false, STRICT_P, "strict"}, + { 6, false, STRINGFUNC_P, "string"}, + { 7, false, BOOLEAN_P, "boolean"}, + { 7, false, CEILING_P, "ceiling"}, + { 7, false, DECIMAL_P, "decimal"}, + { 7, false, STR_INITCAP_P, "initcap"}, + { 7, false, INTEGER_P, "integer"}, + { 7, false, STR_REPLACEFUNC_P, "replace"}, + { 7, false, TIME_TZ_P, "time_tz"}, + { 7, false, UNKNOWN_P, "unknown"}, + { 8, false, DATETIME_P, "datetime"}, + { 8, false, KEYVALUE_P, "keyvalue"}, + { 9, false, TIMESTAMP_P, "timestamp"}, + { 10,false, LIKE_REGEX_P, "like_regex"}, + { 10,false, STR_SPLIT_PART_P, "split_part"}, + { 12,false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; /* diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..4c60f9d1826 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiReplaceFunc, /* .replace() item method */ + jpiStrLowerFunc, /* .lower() item method */ + jpiStrUpperFunc, /* .upper() item method */ + jpiStrLtrimFunc, /* .ltrim() item method */ + jpiStrRtrimFunc, /* .rtrim() item method */ + jpiStrBtrimFunc, /* .btrim() item method */ + jpiStrInitcapFunc, /* .initcap() item method */ + jpiStrSplitPartFunc, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ @@ -188,6 +196,12 @@ typedef struct JsonPathItem int32 patternlen; uint32 flags; } like_regex; + + struct + { + int32 arg0; + int32 arg1; + } method_args; } content; } JsonPathItem; @@ -199,6 +213,8 @@ extern bool jspGetNext(JsonPathItem *v, JsonPathItem *a); extern void jspGetArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetLeftArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetRightArg(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg0(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg1(JsonPathItem *v, JsonPathItem *a); extern Numeric jspGetNumeric(JsonPathItem *v); extern bool jspGetBool(JsonPathItem *v); extern char *jspGetString(JsonPathItem *v, int32 *len); @@ -266,6 +282,12 @@ struct JsonPathParseItem uint32 len; char *val; /* could not be not null-terminated */ } string; + + struct + { + JsonPathParseItem *arg0; + JsonPathParseItem *arg1; + } method_args; } value; }; diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..2dbaec92129 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,399 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..1b122e877d5 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,78 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..e229e0a0d04 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,114 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..8752ffcf08c 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,19 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-21 18:06 Robert Haas <[email protected]> parent: David E. Wheeler <[email protected]> 1 sibling, 1 reply; 39+ messages in thread From: Robert Haas @ 2025-05-21 18:06 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Florents Tselai <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On Tue, May 13, 2025 at 11:00 PM David E. Wheeler <[email protected]> wrote: > On May 13, 2025, at 16:24, Florents Tselai <[email protected]> wrote: > > As Robert said—and I agree—renaming the existing _tz family would be more trouble than it’s worth, given the need for deprecations, migration paths, etc. If we were designing this today, suffixes like _stable or _volatile might have been more appropriate, but at this point, we’re better off staying consistent with the _tz family. > > I get the pragmatism, and don’t want to over-bike-shed, but what a wart to live with. [I just went back and re-read Robert’s post, and didn’t realize he used exactly the same expression!] Would it really be too effortful to create _stable or _volatile functions and leave the _tz functions as a sort of legacy? No, that wouldn't be too much work, but the issue is that people will keep using the _tz versions and when we eventually try to remove them those people will complain no matter how prominent we make the deprecation notice. If we want to go this route, maybe we should do something like: 1. Add the new versions with a _s suffix or whatever. 2. Invent a GUC jsonb_tz_warning = { on | off } that advises you to use the new functions instead, whenever you use the old ones. 3. After N years, flip the default value from off to on. 4. After M additional years, remove the old functions and the GUC. 5. Still get complaints. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-21 18:13 David E. Wheeler <[email protected]> parent: Robert Haas <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-21 18:13 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Florents Tselai <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On May 21, 2025, at 14:06, Robert Haas <[email protected]> wrote: > No, that wouldn't be too much work, but the issue is that people will > keep using the _tz versions and when we eventually try to remove them > those people will complain no matter how prominent we make the > deprecation notice. If we want to go this route, maybe we should do > something like: > > 1. Add the new versions with a _s suffix or whatever. > > 2. Invent a GUC jsonb_tz_warning = { on | off } that advises you to > use the new functions instead, whenever you use the old ones. > > 3. After N years, flip the default value from off to on. > > 4. After M additional years, remove the old functions and the GUC. > > 5. Still get complaints. Complainers gonna complain. 🫠 Any idea how widespread the use of the function is? It was added in 17, and I’ve met few who have really dug into the jonpath stuff yet, let alone needed the time zone conversion functionality. Best, David Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-21 18:31 Tom Lane <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Tom Lane @ 2025-05-21 18:31 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Robert Haas <[email protected]>; Florents Tselai <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> "David E. Wheeler" <[email protected]> writes: > On May 21, 2025, at 14:06, Robert Haas <[email protected]> wrote: >> If we want to go this route, maybe we should do >> something like: >> ... >> 5. Still get complaints. > Complainers gonna complain. Yeah. I do not see the point of that amount of effort. > Any idea how widespread the use of the function is? It was added in 17, and I’ve met few who have really dug into the jonpath stuff yet, let alone needed the time zone conversion functionality. That's a good point. We should also remember that if somebody really really doesn't want to fix their app, they can trivially create a wrapper function with the old name. Having said that, what's wrong with inventing some improved function names and never removing the old ones? regards, tom lane ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-22 14:05 Robert Haas <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Robert Haas @ 2025-05-22 14:05 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: David E. Wheeler <[email protected]>; Florents Tselai <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On Wed, May 21, 2025 at 2:31 PM Tom Lane <[email protected]> wrote: > Having said that, what's wrong with inventing some improved function > names and never removing the old ones? I don't particularly like the clutter, but if the consensus is that the clutter doesn't matter, fair enough. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-22 16:38 Florents Tselai <[email protected]> parent: Robert Haas <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-05-22 16:38 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; David E. Wheeler <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> > On 22 May 2025, at 5:05 PM, Robert Haas <[email protected]> wrote: > > On Wed, May 21, 2025 at 2:31 PM Tom Lane <[email protected]> wrote: >> Having said that, what's wrong with inventing some improved function >> names and never removing the old ones? > > I don't particularly like the clutter, but if the consensus is that > the clutter doesn't matter, fair enough. > It depends really on how much future work we expect in adding more methods in jsonpath. I think there’s a lot of potential there, but that’s a guess really. On David’s point about popularity: In my experience timestamp related stuff from jsonb documents end up in a generated column, and are indexed & queried there. I expect that to continue in PG18 onwards as we’ll have virtual gen columns too. Just to be clear, though, adding another version of these functions means we’ll have an additional (now third) set of the same 5 functions: The vanilla versions are considered stable and the suffixed *_tz or *_volatile (?) jsonb_path_exists jsonb_path_query jsonb_path_query_array jsonb_path_query_first jsonb_path_match ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-22 18:08 David E. Wheeler <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 0 replies; 39+ messages in thread From: David E. Wheeler @ 2025-05-22 18:08 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]>; Peter Eisentraut <[email protected]> On May 22, 2025, at 12:38, Florents Tselai <[email protected]> wrote: > In my experience timestamp related stuff from jsonb documents end up in a generated column, > and are indexed & queried there. Have you seen this in the wild using the _tz functions? I wouldn’t think they were indexable, given the volatility. D Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-22 20:56 Peter Eisentraut <[email protected]> parent: Robert Haas <[email protected]> 1 sibling, 2 replies; 39+ messages in thread From: Peter Eisentraut @ 2025-05-22 20:56 UTC (permalink / raw) To: Robert Haas <[email protected]>; Florents Tselai <[email protected]>; +Cc: Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On 09.05.25 21:50, Robert Haas wrote: > I always struggle a bit to remember our policy on these issues -- to > the best of my knowledge, we haven't documented it anywhere, and I > think we probably should. I believe the way it works is that whenever > a function depends on the operating system's timestamp or locale > definitions, we decide it has to be stable, not immutable. We don't > expect those things to be updated very often, but we know sometimes > they do get updated. I don't understand how this discussion got to the conclusion that functions that depend on the locale cannot be immutable. Note that the top-level functions lower, upper, and initcap themselves are immutable. ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-23 13:56 Robert Haas <[email protected]> parent: Peter Eisentraut <[email protected]> 1 sibling, 0 replies; 39+ messages in thread From: Robert Haas @ 2025-05-23 13:56 UTC (permalink / raw) To: Peter Eisentraut <[email protected]>; +Cc: Florents Tselai <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Thu, May 22, 2025 at 4:56 PM Peter Eisentraut <[email protected]> wrote: > I don't understand how this discussion got to the conclusion that > functions that depend on the locale cannot be immutable. Note that the > top-level functions lower, upper, and initcap themselves are immutable. Oh, well that was what Tom said last September and I just assumed he was right about the policy. If not, well then that's different. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-23 17:06 Florents Tselai <[email protected]> parent: Peter Eisentraut <[email protected]> 1 sibling, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-05-23 17:06 UTC (permalink / raw) To: Peter Eisentraut <[email protected]>; +Cc: Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> > On 22 May 2025, at 11:56 PM, Peter Eisentraut <[email protected]> wrote: > > On 09.05.25 21:50, Robert Haas wrote: >> I always struggle a bit to remember our policy on these issues -- to >> the best of my knowledge, we haven't documented it anywhere, and I >> think we probably should. I believe the way it works is that whenever >> a function depends on the operating system's timestamp or locale >> definitions, we decide it has to be stable, not immutable. We don't >> expect those things to be updated very often, but we know sometimes >> they do get updated. > > I don't understand how this discussion got to the conclusion that functions that depend on the locale cannot be immutable. Note that the top-level functions lower, upper, and initcap themselves are immutable. I assume you mean that they’re set at initdb time, so there’s no mutability concern? ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-23 17:52 Tom Lane <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Tom Lane @ 2025-05-23 17:52 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> Florents Tselai <[email protected]> writes: > On 22 May 2025, at 11:56 PM, Peter Eisentraut <[email protected]> wrote: >> I don't understand how this discussion got to the conclusion that functions that depend on the locale cannot be immutable. Note that the top-level functions lower, upper, and initcap themselves are immutable. > I assume you mean that they’re set at initdb time, so there’s no mutability concern? Yeah, I think Peter's right and I'm wrong. Obviously this ties into our philosophical debate about how immutable is immutable. But as long as the functions only depend on locale settings that are fixed at database creation, I think it's okay to consider them immutable. If you were, say, depending on LC_NUMERIC, it would clearly be unsafe to consider that immutable, so I'm not quite sure if this is the end of the discussion. But for what's mentioned in the thread title, I think we only care about LC_CTYPE. regards, tom lane ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-24 16:08 David E. Wheeler <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-24 16:08 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Florents Tselai <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On May 23, 2025, at 13:52, Tom Lane <[email protected]> wrote: >> I assume you mean that they’re set at initdb time, so there’s no mutability concern? > > Yeah, I think Peter's right and I'm wrong. Obviously this ties into > our philosophical debate about how immutable is immutable. But as > long as the functions only depend on locale settings that are fixed > at database creation, I think it's okay to consider them immutable. > > If you were, say, depending on LC_NUMERIC, it would clearly be unsafe > to consider that immutable, so I'm not quite sure if this is the end > of the discussion. But for what's mentioned in the thread title, > I think we only care about LC_CTYPE. Oh, so maybe all this is moot, and Florents can go ahead and add support for the functions to the non-_tz functions? Should there be some sort of inventory of what functions can be used in what contexts? D Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-24 16:51 Florents Tselai <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-05-24 16:51 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> > On 24 May 2025, at 7:08 PM, David E. Wheeler <[email protected]> wrote: > > On May 23, 2025, at 13:52, Tom Lane <[email protected]> wrote: > >>> I assume you mean that they’re set at initdb time, so there’s no mutability concern? >> >> Yeah, I think Peter's right and I'm wrong. Obviously this ties into >> our philosophical debate about how immutable is immutable. But as >> long as the functions only depend on locale settings that are fixed >> at database creation, I think it's okay to consider them immutable. >> >> If you were, say, depending on LC_NUMERIC, it would clearly be unsafe >> to consider that immutable, so I'm not quite sure if this is the end >> of the discussion. But for what's mentioned in the thread title, >> I think we only care about LC_CTYPE. > > Oh, so maybe all this is moot, and Florents can go ahead and add support for the functions to the non-_tz functions? > I think the patch is still in reasonably good shape and hasn’t changed much since September 24. So yes, I’d hope there are still some valid points to consider or improve. Otherwise, I’ll have only myself to blame for not pushing harder before the feature freeze. 😅 ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-24 21:39 David E. Wheeler <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-24 21:39 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On May 24, 2025, at 12:51, Florents Tselai <[email protected]> wrote: > I think the patch is still in reasonably good shape and hasn’t changed much since September 24.So yes, I’d hope there are still some valid points to consider or improve. Okay, here’s a review. Patch applies cleanly. All tests pass. I'm curious why you added the `arg0` and `arg1` fields to the `method_args` union. Is there some reason that the existing `left` and `right` fields wouldn't work? Admittedly these are not formally binary operators, but I don't see that it matters much. The existing string() method operates on a "JSON boolean, number, string, or datetime"; should these functions also operate on all those data types? The argument to the trim methods appears to be ignored: ``` postgres=# select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); jsonb_path_query ------------------ "zzzytest" ``` I'm wondering if the issue is the use of the opt_datetime_template in the grammar? ``` | '.' STR_LTRIM_P '(' opt_datetime_template ')' { $$ = makeItemUnary(jpiStrLtrimFunc, $4); } | '.' STR_RTRIM_P '(' opt_datetime_template ')' { $$ = makeItemUnary(jpiStrRtrimFunc, $4); } | '.' STR_BTRIM_P '(' opt_datetime_template ')' { $$ = makeItemUnary(jpiStrBtrimFunc, $4); } ``` I realize it resolves to a string, but for some reason it doesn't get picked up. But also, do you want to support variables for either of these arguments? If so, maybe rename and use starts_with_initial: ``` starts_with_initial: STRING_P { $$ = makeItemString(&$1); } | VARIABLE_P { $$ = makeItemVariable(&$1); } ; ``` split_part() does not support a negative n value: ``` postgres=# select split_part('abc,def,ghi,jkl', ',', -2) ; split_part ------------ ghi select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); ERROR: syntax error at or near "-" of jsonpath input LINE 1: select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part("... ``` Nor does a `+` work. I think you’d be better served using `csv_elem`, something like: ``` | '.' STR_SPLIT_PART_P '(' STRING_P csv_elem ‘)’ ``` I'm not sure how well these functions comply with the SQL spec. Does it have a provision for implementation-specific methods? I *think* all existing methods are defined by the spec, but someone with access to its contents would have to say for sure. And maybe we don't care, consider this a natural extension? I’ve attached a new patch with docs. Best, David Attachments: [application/octet-stream] v3-0001-Add-additional-jsonpath-string-methods.patch (51.1K, 2-v3-0001-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 411a72f1d8654696b81a2386169a46ff67d3a818 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Sat, 24 May 2025 17:33:56 -0400 Subject: [PATCH v3] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 171 ++++++++ src/backend/utils/adt/jsonpath.c | 176 ++++++++- src/backend/utils/adt/jsonpath_exec.c | 189 +++++++++ src/backend/utils/adt/jsonpath_gram.y | 71 +++- src/backend/utils/adt/jsonpath_scan.l | 41 ++ src/include/utils/jsonpath.h | 22 ++ src/test/regress/expected/jsonb_jsonpath.out | 393 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 72 ++++ src/test/regress/sql/jsonb_jsonpath.sql | 108 +++++ src/test/regress/sql/jsonpath.sql | 13 + 10 files changed, 1252 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c67688cbf5f..25d0d920dbf 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18627,6 +18627,177 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the start of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the end of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the start and end of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the start and end + of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..646c7b460a3 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -326,6 +326,38 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, *(int32 *) (buf->data + right) = chld - pos; } break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + { + { + /* + * First, reserve place for left/right arg's positions, then + * record both args and sets actual position in reserved + * places. + */ + int32 arg0 = reserveSpaceForItemPointer(buf); + int32 arg1 = reserveSpaceForItemPointer(buf); + + if (!item->value.method_args.arg0) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg0, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg0) = chld - pos; + + if (!item->value.method_args.arg1) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg1, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg1) = chld - pos; + } + } + break; case jpiLikeRegex: { int32 offs; @@ -362,6 +394,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +492,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +869,72 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiReplaceFunc: + appendStringInfoString(buf, ".replace("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPartFunc: + appendStringInfoString(buf, ".split_part("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrLowerFunc: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpperFunc: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcapFunc: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrimFunc: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrimFunc: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrimFunc: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +1010,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiReplaceFunc: + return "replace"; + case jpiStrLowerFunc: + return "lower"; + case jpiStrUpperFunc: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +1024,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrimFunc: + return "ltrim"; + case jpiStrRtrimFunc: + return "rtrim"; + case jpiStrBtrimFunc: + return "btrim"; + case jpiStrInitcapFunc: + return "initcap"; + case jpiStrSplitPartFunc: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1136,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; case jpiString: case jpiKey: @@ -1044,6 +1167,11 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + read_int32(v->content.method_args.arg0, base, pos); + read_int32(v->content.method_args.arg1, base, pos); + break; case jpiNot: case jpiIsUnknown: case jpiExists: @@ -1055,6 +1183,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1221,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1283,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiReplaceFunc || + v->type == jpiStrLowerFunc || + v->type == jpiStrUpperFunc || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc || + v->type == jpiStrInitcapFunc || + v->type == jpiStrSplitPartFunc); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1184,6 +1326,24 @@ jspGetLeftArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.left); } +void +jspGetArg0(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg0); +} + +void +jspGetArg1(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg1); +} + void jspGetRightArg(JsonPathItem *v, JsonPathItem *a) { @@ -1206,6 +1366,7 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.right); } + bool jspGetBool(JsonPathItem *v) { @@ -1227,7 +1388,8 @@ jspGetString(JsonPathItem *v, int32 *len) { Assert(v->type == jpiKey || v->type == jpiString || - v->type == jpiVariable); + v->type == jpiVariable || + v->type == jpiStringFunc); if (len) *len = v->content.value.datalen; @@ -1501,6 +1663,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiReplaceFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..51239121f18 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrimFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,176 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) { + Assert( jsp->type == jpiStrLowerFunc || + jsp->type == jpiStrUpperFunc || + jsp->type == jpiReplaceFunc || + jsp->type == jpiStrLtrimFunc || + jsp->type == jpiStrRtrimFunc || + jsp->type == jpiStrBtrimFunc || + jsp->type == jpiStrInitcapFunc || + jsp->type == jpiStrSplitPartFunc); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string value. The first argument to internal functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + switch (jsp->type) + { + case jpiStrLtrimFunc: + func = ltrim1; + break; + case jpiStrRtrimFunc: + func = rtrim1; + break; + case jpiStrBtrimFunc: + func = btrim1; + break; + default: ; + } + + if (jsp->content.arg) + { + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(" "))); + break; + } + + case jpiStrLowerFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpperFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcapFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiReplaceFunc: + { + char *from_str, *to_str; + int from_len, to_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPartFunc: + { + char *from_str; + Numeric n; + int from_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..cf7a4ff19fd 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -43,6 +43,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, JsonPathString *flags, JsonPathParseItem ** result, struct Node *escontext); +static JsonPathParseItem *makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); +static JsonPathParseItem *makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); + /* * Bison doesn't allocate anything that needs to live across parser calls, @@ -86,6 +89,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACEFUNC_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -94,8 +99,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, index_elem starts_with_initial expr_or_predicate datetime_template opt_datetime_template csv_elem datetime_precision opt_datetime_precision + str_method_arg_elem -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr csv_list opt_csv_list str_method_arg_list %type <indexs> index_list @@ -278,6 +284,32 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACEFUNC_P '(' str_method_arg_list ')' + { + if (list_length($4) == 2) + $$ = makeItemReplaceFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".replace() accepts two arguments."))); + } + | '.' STR_SPLIT_PART_P '(' STRING_P csv_elem ')' + { + if (list_length($4) == 2) + $$ = makeItemStrSplitPartFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".split_part() accepts two arguments."))); + } + | '.' STR_LTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrLtrimFunc, $4); } + | '.' STR_RTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrRtrimFunc, $4); } + | '.' STR_BTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrBtrimFunc, $4); } ; csv_elem: @@ -317,6 +349,15 @@ opt_datetime_template: | /* EMPTY */ { $$ = NULL; } ; +str_method_arg_elem: + STRING_P { $$ = makeItemString(&$1); } + | INT_P { $$ = makeItemNumeric(&$1); } + ; + +str_method_arg_list: + str_method_arg_elem { $$ = list_make1($1); } + | str_method_arg_list ',' str_method_arg_elem { $$ = lappend($1, $3); } + ; key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +398,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +417,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLowerFunc; } + | STR_UPPER_P { $$ = jpiStrUpperFunc; } + | STR_INITCAP_P { $$ = jpiStrInitcapFunc; } ; %% @@ -472,6 +519,28 @@ makeItemBinary(JsonPathItemType type, JsonPathParseItem *la, JsonPathParseItem * return v; } +static JsonPathParseItem * +makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiReplaceFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + +static JsonPathParseItem * +makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiStrSplitPartFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + static JsonPathParseItem * makeItemUnary(JsonPathItemType type, JsonPathParseItem *a) { diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..f20e9e5f1d3 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -433,6 +433,47 @@ static const JsonPathKeyword keywords[] = { {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, + { 2, false, IS_P, "is"}, + { 2, false, TO_P, "to"}, + { 3, false, ABS_P, "abs"}, + { 3, false, LAX_P, "lax"}, + { 4, false, DATE_P, "date"}, + { 4, false, FLAG_P, "flag"}, + { 4, false, LAST_P, "last"}, + { 4, true, NULL_P, "null"}, + { 4, false, SIZE_P, "size"}, + { 4, false, TIME_P, "time"}, + { 4, true, TRUE_P, "true"}, + { 4, false, TYPE_P, "type"}, + { 4, false, WITH_P, "with"}, + { 5, false, STR_BTRIM_P, "btrim"}, + { 5, true, FALSE_P, "false"}, + { 5, false, FLOOR_P, "floor"}, + { 5, false, STR_LOWER_P, "lower"}, + { 5, false, STR_LTRIM_P, "ltrim"}, + { 5, false, STR_RTRIM_P, "rtrim"}, + { 5, false, STR_UPPER_P, "upper"}, + { 6, false, BIGINT_P, "bigint"}, + { 6, false, DOUBLE_P, "double"}, + { 6, false, EXISTS_P, "exists"}, + { 6, false, NUMBER_P, "number"}, + { 6, false, STARTS_P, "starts"}, + { 6, false, STRICT_P, "strict"}, + { 6, false, STRINGFUNC_P, "string"}, + { 7, false, BOOLEAN_P, "boolean"}, + { 7, false, CEILING_P, "ceiling"}, + { 7, false, DECIMAL_P, "decimal"}, + { 7, false, STR_INITCAP_P, "initcap"}, + { 7, false, INTEGER_P, "integer"}, + { 7, false, STR_REPLACEFUNC_P, "replace"}, + { 7, false, TIME_TZ_P, "time_tz"}, + { 7, false, UNKNOWN_P, "unknown"}, + { 8, false, DATETIME_P, "datetime"}, + { 8, false, KEYVALUE_P, "keyvalue"}, + { 9, false, TIMESTAMP_P, "timestamp"}, + { 10,false, LIKE_REGEX_P, "like_regex"}, + { 10,false, STR_SPLIT_PART_P, "split_part"}, + { 12,false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; /* diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..4c60f9d1826 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiReplaceFunc, /* .replace() item method */ + jpiStrLowerFunc, /* .lower() item method */ + jpiStrUpperFunc, /* .upper() item method */ + jpiStrLtrimFunc, /* .ltrim() item method */ + jpiStrRtrimFunc, /* .rtrim() item method */ + jpiStrBtrimFunc, /* .btrim() item method */ + jpiStrInitcapFunc, /* .initcap() item method */ + jpiStrSplitPartFunc, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ @@ -188,6 +196,12 @@ typedef struct JsonPathItem int32 patternlen; uint32 flags; } like_regex; + + struct + { + int32 arg0; + int32 arg1; + } method_args; } content; } JsonPathItem; @@ -199,6 +213,8 @@ extern bool jspGetNext(JsonPathItem *v, JsonPathItem *a); extern void jspGetArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetLeftArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetRightArg(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg0(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg1(JsonPathItem *v, JsonPathItem *a); extern Numeric jspGetNumeric(JsonPathItem *v); extern bool jspGetBool(JsonPathItem *v); extern char *jspGetString(JsonPathItem *v, int32 *len); @@ -266,6 +282,12 @@ struct JsonPathParseItem uint32 len; char *val; /* could not be not null-terminated */ } string; + + struct + { + JsonPathParseItem *arg0; + JsonPathParseItem *arg1; + } method_args; } value; }; diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..2dbaec92129 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,399 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..1b122e877d5 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,78 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..e229e0a0d04 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,114 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..8752ffcf08c 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,19 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 4-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-24 21:46 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-24 21:46 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On May 24, 2025, at 17:39, David E. Wheeler <[email protected]> wrote: > I’ve attached a new patch with docs. Oh, and I forgot to mention, src/backend/utils/adt/jsonpath_scan.l looks like it has a ton of duplication in it. Shouldn’t it just add the new keywords, something like: ``` @@ -415,6 +415,11 @@ static const JsonPathKeyword keywords[] = { {4, false, WITH_P, "with"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_BTRIM_P, "btrim"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -428,10 +433,13 @@ static const JsonPathKeyword keywords[] = { {7, false, INTEGER_P, "integer"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, + {7, false, STR_INITCAP_P, "initcap"}, + {7, false, STR_REPLACEFUNC_P, "replace"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10, false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, ``` Best, David Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-24 21:55 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-24 21:55 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On May 24, 2025, at 17:46, David E. Wheeler <[email protected]> wrote: > Oh, and I forgot to mention, src/backend/utils/adt/jsonpath_scan.l looks like it has a ton of duplication in it. Shouldn’t it just add the new keywords, something like: And now I see my patch broke the grammar because I left some of my fiddling in there. Apologies. Here’s an updated patch with the updated keyword map, too. Best, David Attachments: [application/octet-stream] v4-0001-Add-additional-jsonpath-string-methods.patch (51.1K, 2-v4-0001-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 6a4705fd7cd79de9ad49e679fe15e5b2ca19fe10 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Sat, 24 May 2025 17:55:08 -0400 Subject: [PATCH v4] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 171 ++++++++ src/backend/utils/adt/jsonpath.c | 176 ++++++++- src/backend/utils/adt/jsonpath_exec.c | 189 +++++++++ src/backend/utils/adt/jsonpath_gram.y | 71 +++- src/backend/utils/adt/jsonpath_scan.l | 41 ++ src/include/utils/jsonpath.h | 22 ++ src/test/regress/expected/jsonb_jsonpath.out | 393 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 72 ++++ src/test/regress/sql/jsonb_jsonpath.sql | 108 +++++ src/test/regress/sql/jsonpath.sql | 13 + 10 files changed, 1252 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c67688cbf5f..25d0d920dbf 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18627,6 +18627,177 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the start of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the end of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the start and end of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the start and end + of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..646c7b460a3 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -326,6 +326,38 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, *(int32 *) (buf->data + right) = chld - pos; } break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + { + { + /* + * First, reserve place for left/right arg's positions, then + * record both args and sets actual position in reserved + * places. + */ + int32 arg0 = reserveSpaceForItemPointer(buf); + int32 arg1 = reserveSpaceForItemPointer(buf); + + if (!item->value.method_args.arg0) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg0, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg0) = chld - pos; + + if (!item->value.method_args.arg1) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg1, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg1) = chld - pos; + } + } + break; case jpiLikeRegex: { int32 offs; @@ -362,6 +394,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +492,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +869,72 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiReplaceFunc: + appendStringInfoString(buf, ".replace("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPartFunc: + appendStringInfoString(buf, ".split_part("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrLowerFunc: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpperFunc: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcapFunc: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrimFunc: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrimFunc: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrimFunc: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +1010,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiReplaceFunc: + return "replace"; + case jpiStrLowerFunc: + return "lower"; + case jpiStrUpperFunc: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +1024,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrimFunc: + return "ltrim"; + case jpiStrRtrimFunc: + return "rtrim"; + case jpiStrBtrimFunc: + return "btrim"; + case jpiStrInitcapFunc: + return "initcap"; + case jpiStrSplitPartFunc: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1136,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; case jpiString: case jpiKey: @@ -1044,6 +1167,11 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + read_int32(v->content.method_args.arg0, base, pos); + read_int32(v->content.method_args.arg1, base, pos); + break; case jpiNot: case jpiIsUnknown: case jpiExists: @@ -1055,6 +1183,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1221,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1283,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiReplaceFunc || + v->type == jpiStrLowerFunc || + v->type == jpiStrUpperFunc || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc || + v->type == jpiStrInitcapFunc || + v->type == jpiStrSplitPartFunc); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1184,6 +1326,24 @@ jspGetLeftArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.left); } +void +jspGetArg0(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg0); +} + +void +jspGetArg1(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg1); +} + void jspGetRightArg(JsonPathItem *v, JsonPathItem *a) { @@ -1206,6 +1366,7 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.right); } + bool jspGetBool(JsonPathItem *v) { @@ -1227,7 +1388,8 @@ jspGetString(JsonPathItem *v, int32 *len) { Assert(v->type == jpiKey || v->type == jpiString || - v->type == jpiVariable); + v->type == jpiVariable || + v->type == jpiStringFunc); if (len) *len = v->content.value.datalen; @@ -1501,6 +1663,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiReplaceFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..51239121f18 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrimFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,176 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) { + Assert( jsp->type == jpiStrLowerFunc || + jsp->type == jpiStrUpperFunc || + jsp->type == jpiReplaceFunc || + jsp->type == jpiStrLtrimFunc || + jsp->type == jpiStrRtrimFunc || + jsp->type == jpiStrBtrimFunc || + jsp->type == jpiStrInitcapFunc || + jsp->type == jpiStrSplitPartFunc); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string value. The first argument to internal functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + switch (jsp->type) + { + case jpiStrLtrimFunc: + func = ltrim1; + break; + case jpiStrRtrimFunc: + func = rtrim1; + break; + case jpiStrBtrimFunc: + func = btrim1; + break; + default: ; + } + + if (jsp->content.arg) + { + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(" "))); + break; + } + + case jpiStrLowerFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpperFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcapFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiReplaceFunc: + { + char *from_str, *to_str; + int from_len, to_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPartFunc: + { + char *from_str; + Numeric n; + int from_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..c998c40cfbd 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -43,6 +43,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, JsonPathString *flags, JsonPathParseItem ** result, struct Node *escontext); +static JsonPathParseItem *makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); +static JsonPathParseItem *makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); + /* * Bison doesn't allocate anything that needs to live across parser calls, @@ -86,6 +89,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACEFUNC_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -94,8 +99,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, index_elem starts_with_initial expr_or_predicate datetime_template opt_datetime_template csv_elem datetime_precision opt_datetime_precision + str_method_arg_elem -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr csv_list opt_csv_list str_method_arg_list %type <indexs> index_list @@ -278,6 +284,32 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACEFUNC_P '(' str_method_arg_list ')' + { + if (list_length($4) == 2) + $$ = makeItemReplaceFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".replace() accepts two arguments."))); + } + | '.' STR_SPLIT_PART_P '(' str_method_arg_list ')' + { + if (list_length($4) == 2) + $$ = makeItemStrSplitPartFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".split_part() accepts two arguments."))); + } + | '.' STR_LTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrLtrimFunc, $4); } + | '.' STR_RTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrRtrimFunc, $4); } + | '.' STR_BTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrBtrimFunc, $4); } ; csv_elem: @@ -317,6 +349,15 @@ opt_datetime_template: | /* EMPTY */ { $$ = NULL; } ; +str_method_arg_elem: + STRING_P { $$ = makeItemString(&$1); } + | INT_P { $$ = makeItemNumeric(&$1); } + ; + +str_method_arg_list: + str_method_arg_elem { $$ = list_make1($1); } + | str_method_arg_list ',' str_method_arg_elem { $$ = lappend($1, $3); } + ; key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +398,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +417,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLowerFunc; } + | STR_UPPER_P { $$ = jpiStrUpperFunc; } + | STR_INITCAP_P { $$ = jpiStrInitcapFunc; } ; %% @@ -472,6 +519,28 @@ makeItemBinary(JsonPathItemType type, JsonPathParseItem *la, JsonPathParseItem * return v; } +static JsonPathParseItem * +makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiReplaceFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + +static JsonPathParseItem * +makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiStrSplitPartFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + static JsonPathParseItem * makeItemUnary(JsonPathItemType type, JsonPathParseItem *a) { diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..f20e9e5f1d3 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -433,6 +433,47 @@ static const JsonPathKeyword keywords[] = { {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, + { 2, false, IS_P, "is"}, + { 2, false, TO_P, "to"}, + { 3, false, ABS_P, "abs"}, + { 3, false, LAX_P, "lax"}, + { 4, false, DATE_P, "date"}, + { 4, false, FLAG_P, "flag"}, + { 4, false, LAST_P, "last"}, + { 4, true, NULL_P, "null"}, + { 4, false, SIZE_P, "size"}, + { 4, false, TIME_P, "time"}, + { 4, true, TRUE_P, "true"}, + { 4, false, TYPE_P, "type"}, + { 4, false, WITH_P, "with"}, + { 5, false, STR_BTRIM_P, "btrim"}, + { 5, true, FALSE_P, "false"}, + { 5, false, FLOOR_P, "floor"}, + { 5, false, STR_LOWER_P, "lower"}, + { 5, false, STR_LTRIM_P, "ltrim"}, + { 5, false, STR_RTRIM_P, "rtrim"}, + { 5, false, STR_UPPER_P, "upper"}, + { 6, false, BIGINT_P, "bigint"}, + { 6, false, DOUBLE_P, "double"}, + { 6, false, EXISTS_P, "exists"}, + { 6, false, NUMBER_P, "number"}, + { 6, false, STARTS_P, "starts"}, + { 6, false, STRICT_P, "strict"}, + { 6, false, STRINGFUNC_P, "string"}, + { 7, false, BOOLEAN_P, "boolean"}, + { 7, false, CEILING_P, "ceiling"}, + { 7, false, DECIMAL_P, "decimal"}, + { 7, false, STR_INITCAP_P, "initcap"}, + { 7, false, INTEGER_P, "integer"}, + { 7, false, STR_REPLACEFUNC_P, "replace"}, + { 7, false, TIME_TZ_P, "time_tz"}, + { 7, false, UNKNOWN_P, "unknown"}, + { 8, false, DATETIME_P, "datetime"}, + { 8, false, KEYVALUE_P, "keyvalue"}, + { 9, false, TIMESTAMP_P, "timestamp"}, + { 10,false, LIKE_REGEX_P, "like_regex"}, + { 10,false, STR_SPLIT_PART_P, "split_part"}, + { 12,false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; /* diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..4c60f9d1826 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiReplaceFunc, /* .replace() item method */ + jpiStrLowerFunc, /* .lower() item method */ + jpiStrUpperFunc, /* .upper() item method */ + jpiStrLtrimFunc, /* .ltrim() item method */ + jpiStrRtrimFunc, /* .rtrim() item method */ + jpiStrBtrimFunc, /* .btrim() item method */ + jpiStrInitcapFunc, /* .initcap() item method */ + jpiStrSplitPartFunc, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ @@ -188,6 +196,12 @@ typedef struct JsonPathItem int32 patternlen; uint32 flags; } like_regex; + + struct + { + int32 arg0; + int32 arg1; + } method_args; } content; } JsonPathItem; @@ -199,6 +213,8 @@ extern bool jspGetNext(JsonPathItem *v, JsonPathItem *a); extern void jspGetArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetLeftArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetRightArg(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg0(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg1(JsonPathItem *v, JsonPathItem *a); extern Numeric jspGetNumeric(JsonPathItem *v); extern bool jspGetBool(JsonPathItem *v); extern char *jspGetString(JsonPathItem *v, int32 *len); @@ -266,6 +282,12 @@ struct JsonPathParseItem uint32 len; char *val; /* could not be not null-terminated */ } string; + + struct + { + JsonPathParseItem *arg0; + JsonPathParseItem *arg1; + } method_args; } value; }; diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..2dbaec92129 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,399 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..1b122e877d5 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,78 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..e229e0a0d04 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,114 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..8752ffcf08c 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,19 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 3-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-24 22:01 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-24 22:01 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On May 24, 2025, at 17:55, David E. Wheeler <[email protected]> wrote: > And now I see my patch broke the grammar because I left some of my fiddling in there. Apologies. Here’s an updated patch with the updated keyword map, too. No, really :sigh: D Attachments: [application/octet-stream] v5-0001-Add-additional-jsonpath-string-methods.patch (50.4K, 2-v5-0001-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 6c61d2190fe0bef195032f4adcf50f7ddcc82bf5 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Sat, 24 May 2025 17:59:54 -0400 Subject: [PATCH v5] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 171 ++++++++ src/backend/utils/adt/jsonpath.c | 176 ++++++++- src/backend/utils/adt/jsonpath_exec.c | 189 +++++++++ src/backend/utils/adt/jsonpath_gram.y | 71 +++- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 22 ++ src/test/regress/expected/jsonb_jsonpath.out | 393 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 72 ++++ src/test/regress/sql/jsonb_jsonpath.sql | 108 +++++ src/test/regress/sql/jsonpath.sql | 13 + 10 files changed, 1219 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c67688cbf5f..25d0d920dbf 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18627,6 +18627,177 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the start of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the end of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with spaces removed from the start and end of + <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only characters in + <replaceable>characters</replaceable> removed from the start and end + of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..646c7b460a3 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -326,6 +326,38 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, *(int32 *) (buf->data + right) = chld - pos; } break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + { + { + /* + * First, reserve place for left/right arg's positions, then + * record both args and sets actual position in reserved + * places. + */ + int32 arg0 = reserveSpaceForItemPointer(buf); + int32 arg1 = reserveSpaceForItemPointer(buf); + + if (!item->value.method_args.arg0) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg0, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg0) = chld - pos; + + if (!item->value.method_args.arg1) + chld = pos; + else if (!flattenJsonPathParseItem(buf, &chld, escontext, + item->value.method_args.arg1, + nestingLevel + argNestingLevel, + insideArraySubscript)) + return false; + *(int32 *) (buf->data + arg1) = chld - pos; + } + } + break; case jpiLikeRegex: { int32 offs; @@ -362,6 +394,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +492,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +869,72 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiReplaceFunc: + appendStringInfoString(buf, ".replace("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPartFunc: + appendStringInfoString(buf, ".split_part("); + if (v->content.method_args.arg0) + { + jspGetArg0(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.method_args.arg1) + { + appendStringInfoChar(buf, ','); + jspGetArg1(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrLowerFunc: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpperFunc: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcapFunc: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrimFunc: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrimFunc: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrimFunc: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +1010,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiReplaceFunc: + return "replace"; + case jpiStrLowerFunc: + return "lower"; + case jpiStrUpperFunc: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +1024,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrimFunc: + return "ltrim"; + case jpiStrRtrimFunc: + return "rtrim"; + case jpiStrBtrimFunc: + return "btrim"; + case jpiStrInitcapFunc: + return "initcap"; + case jpiStrSplitPartFunc: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1136,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrInitcapFunc: break; case jpiString: case jpiKey: @@ -1044,6 +1167,11 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; + case jpiReplaceFunc: + case jpiStrSplitPartFunc: + read_int32(v->content.method_args.arg0, base, pos); + read_int32(v->content.method_args.arg1, base, pos); + break; case jpiNot: case jpiIsUnknown: case jpiExists: @@ -1055,6 +1183,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1221,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1283,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiReplaceFunc || + v->type == jpiStrLowerFunc || + v->type == jpiStrUpperFunc || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrimFunc || + v->type == jpiStrRtrimFunc || + v->type == jpiStrBtrimFunc || + v->type == jpiStrInitcapFunc || + v->type == jpiStrSplitPartFunc); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1184,6 +1326,24 @@ jspGetLeftArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.left); } +void +jspGetArg0(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg0); +} + +void +jspGetArg1(JsonPathItem *v, JsonPathItem *a) +{ + Assert(v->type == jpiReplaceFunc || + v->type == jpiStrSplitPartFunc); + + jspInitByBuffer(a, v->base, v->content.method_args.arg1); +} + void jspGetRightArg(JsonPathItem *v, JsonPathItem *a) { @@ -1206,6 +1366,7 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) jspInitByBuffer(a, v->base, v->content.args.right); } + bool jspGetBool(JsonPathItem *v) { @@ -1227,7 +1388,8 @@ jspGetString(JsonPathItem *v, int32 *len) { Assert(v->type == jpiKey || v->type == jpiString || - v->type == jpiVariable); + v->type == jpiVariable || + v->type == jpiStringFunc); if (len) *len = v->content.value.datalen; @@ -1501,6 +1663,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiReplaceFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..51239121f18 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrimFunc: + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,176 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) { + Assert( jsp->type == jpiStrLowerFunc || + jsp->type == jpiStrUpperFunc || + jsp->type == jpiReplaceFunc || + jsp->type == jpiStrLtrimFunc || + jsp->type == jpiStrRtrimFunc || + jsp->type == jpiStrBtrimFunc || + jsp->type == jpiStrInitcapFunc || + jsp->type == jpiStrSplitPartFunc); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string value. The first argument to internal functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + switch (jsp->type) + { + case jpiStrLtrimFunc: + func = ltrim1; + break; + case jpiStrRtrimFunc: + func = rtrim1; + break; + case jpiStrBtrimFunc: + func = btrim1; + break; + default: ; + } + + if (jsp->content.arg) + { + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(" "))); + break; + } + + case jpiStrLowerFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpperFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcapFunc: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiReplaceFunc: + { + char *from_str, *to_str; + int from_len, to_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPartFunc: + { + char *from_str; + Numeric n; + int from_len; + + jspGetArg0(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetArg1(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLowerFunc: + case jpiStrUpperFunc: + case jpiReplaceFunc: + case jpiStrLtrimFunc: + case jpiStrRtrimFunc: + case jpiStrBtrimFunc: + case jpiStrInitcapFunc: + case jpiStrSplitPartFunc: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..c998c40cfbd 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -43,6 +43,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, JsonPathString *flags, JsonPathParseItem ** result, struct Node *escontext); +static JsonPathParseItem *makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); +static JsonPathParseItem *makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1); + /* * Bison doesn't allocate anything that needs to live across parser calls, @@ -86,6 +89,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACEFUNC_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -94,8 +99,9 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, index_elem starts_with_initial expr_or_predicate datetime_template opt_datetime_template csv_elem datetime_precision opt_datetime_precision + str_method_arg_elem -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr csv_list opt_csv_list str_method_arg_list %type <indexs> index_list @@ -278,6 +284,32 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACEFUNC_P '(' str_method_arg_list ')' + { + if (list_length($4) == 2) + $$ = makeItemReplaceFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".replace() accepts two arguments."))); + } + | '.' STR_SPLIT_PART_P '(' str_method_arg_list ')' + { + if (list_length($4) == 2) + $$ = makeItemStrSplitPartFunc(linitial($4), lsecond($4)); + else + ereturn(escontext, false, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("invalid input syntax for type %s", "jsonpath"), + errdetail(".split_part() accepts two arguments."))); + } + | '.' STR_LTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrLtrimFunc, $4); } + | '.' STR_RTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrRtrimFunc, $4); } + | '.' STR_BTRIM_P '(' opt_datetime_template ')' + { $$ = makeItemUnary(jpiStrBtrimFunc, $4); } ; csv_elem: @@ -317,6 +349,15 @@ opt_datetime_template: | /* EMPTY */ { $$ = NULL; } ; +str_method_arg_elem: + STRING_P { $$ = makeItemString(&$1); } + | INT_P { $$ = makeItemNumeric(&$1); } + ; + +str_method_arg_list: + str_method_arg_elem { $$ = list_make1($1); } + | str_method_arg_list ',' str_method_arg_elem { $$ = lappend($1, $3); } + ; key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +398,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +417,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLowerFunc; } + | STR_UPPER_P { $$ = jpiStrUpperFunc; } + | STR_INITCAP_P { $$ = jpiStrInitcapFunc; } ; %% @@ -472,6 +519,28 @@ makeItemBinary(JsonPathItemType type, JsonPathParseItem *la, JsonPathParseItem * return v; } +static JsonPathParseItem * +makeItemReplaceFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiReplaceFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + +static JsonPathParseItem * +makeItemStrSplitPartFunc(JsonPathParseItem *arg0, JsonPathParseItem *arg1) +{ + JsonPathParseItem *v = makeItemType(jpiStrSplitPartFunc); + + v->value.method_args.arg0 = arg0; + v->value.method_args.arg1 = arg1; + + return v; +} + static JsonPathParseItem * makeItemUnary(JsonPathItemType type, JsonPathParseItem *a) { diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..f12d358e81d 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -415,6 +415,11 @@ static const JsonPathKeyword keywords[] = { {4, false, WITH_P, "with"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_BTRIM_P, "btrim"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -428,10 +433,13 @@ static const JsonPathKeyword keywords[] = { {7, false, INTEGER_P, "integer"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, + {7, false, STR_INITCAP_P, "initcap"}, + {7, false, STR_REPLACEFUNC_P, "replace"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..4c60f9d1826 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiReplaceFunc, /* .replace() item method */ + jpiStrLowerFunc, /* .lower() item method */ + jpiStrUpperFunc, /* .upper() item method */ + jpiStrLtrimFunc, /* .ltrim() item method */ + jpiStrRtrimFunc, /* .rtrim() item method */ + jpiStrBtrimFunc, /* .btrim() item method */ + jpiStrInitcapFunc, /* .initcap() item method */ + jpiStrSplitPartFunc, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ @@ -188,6 +196,12 @@ typedef struct JsonPathItem int32 patternlen; uint32 flags; } like_regex; + + struct + { + int32 arg0; + int32 arg1; + } method_args; } content; } JsonPathItem; @@ -199,6 +213,8 @@ extern bool jspGetNext(JsonPathItem *v, JsonPathItem *a); extern void jspGetArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetLeftArg(JsonPathItem *v, JsonPathItem *a); extern void jspGetRightArg(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg0(JsonPathItem *v, JsonPathItem *a); +extern void jspGetArg1(JsonPathItem *v, JsonPathItem *a); extern Numeric jspGetNumeric(JsonPathItem *v); extern bool jspGetBool(JsonPathItem *v); extern char *jspGetString(JsonPathItem *v, int32 *len); @@ -266,6 +282,12 @@ struct JsonPathParseItem uint32 len; char *val; /* could not be not null-terminated */ } string; + + struct + { + JsonPathParseItem *arg0; + JsonPathParseItem *arg1; + } method_args; } value; }; diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..2dbaec92129 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,399 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..1b122e877d5 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,78 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..e229e0a0d04 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,114 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..8752ffcf08c 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,19 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 3-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-25 04:16 Florents Tselai <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-05-25 04:16 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> > On 25 May 2025, at 1:01 AM, David E. Wheeler <[email protected]> wrote: > > On May 24, 2025, at 17:55, David E. Wheeler <[email protected]> wrote: > >> And now I see my patch broke the grammar because I left some of my fiddling in there. Apologies. Here’s an updated patch with the updated keyword map, too. > > No, really :sigh: > > D > > <v5-0001-Add-additional-jsonpath-string-methods.patch> The most important problem in jsonpath_scan.l now is the fact that I broke the alphabetical ordering of keywords in v2 , and you followed that too. > I'm curious why you added the `arg0` and `arg1` fields to the `method_args` union. Is there some reason that the existing `left` and `right` fields wouldn't work? The left-right ended-up being more of a brain teaser to work with in jsonpath_exec. Until before these methods, the opt_datetime_template was the only argument passed in existing jsonpath functions, So initially I used that as a template to add to the scann-parser infra, but then realized it may make morese sense to have a way to access indexed-args. IIRC, with an eye in the future I found it much more convenient - less of the to work with indexed-args. I should have gone back and use them for *TRIM_P too But you may be onto something with the split_part thing. > The existing string() method operates on a "JSON boolean, number, string, or datetime"; should these functions also operate on all those data types? You mean implicitely conversion to string first? I don’t think so: I’d expect to work like ‘$…string().replace()…' > I'm not sure how well these functions comply with the SQL spec. The fact that Peter hasn’t raized this as an issue, makes me think it's not one ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-05-26 22:00 David E. Wheeler <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-05-26 22:00 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On May 25, 2025, at 00:16, Florents Tselai <[email protected]> wrote: > The most important problem in jsonpath_scan.l now is the fact that I broke the alphabetical ordering of keywords in v2 , > and you followed that too. Oh. They have been organized by length; I didn’t notice they were also alphabetical. > But you may be onto something with the split_part thing. Yes, I think it would be best if the grammar was a bit stricter --- and therefore more self-explanatory --- by making the args closer to what the functions actually expect. >> The existing string() method operates on a "JSON boolean, number, string, or datetime"; should these functions also operate on all those data types? > > You mean implicitely conversion to string first? > I don’t think so: I’d expect to work like ‘$…string().replace()…' Yes. Each of the existing methods has well-defined rules for what types of values they operate on, and many accept multiple types. Looking again, though, it appears that all the date/time methods operate only on strings, so I think you’re correct to follow that precedent and people can use `.string()` if they need it. We can also loosen it up later if use cases demand it. >> I'm not sure how well these functions comply with the SQL spec. > > The fact that Peter hasn’t raized this as an issue, makes me think it's not one Fair enough. Best, David Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-06-03 19:02 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-06-03 19:02 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> Hackers, On May 26, 2025, at 18:00, David E. Wheeler <[email protected]> wrote: > Yes, I think it would be best if the grammar was a bit stricter --- and therefore more self-explanatory --- by making the args closer to what the functions actually expect. I chatted with Florents and went ahead and simplified the grammar and fixed the other issues I identified in my original review. Note that there are two commits, now: `v6-0001-Rename-jsonpath-method-arg-tokens.patch` Renames some of the symbols in the jsonpath grammar so that they’re less generic (`csv*`) and more specific to their contents. This is with the expectation that they will be used by other methods in the next patch and in the future. I thought it best to separate this refactoring from the feature patch. `v6-0002-Add-additional-jsonpath-string-methods.patch` is that feature patch. The grammar now parses the exact number and types of each method argument, eliminating the need for explicit error checking. It also uses the existing patterns for handling methods with two parameters, removing a bunch of duplicate code. Overall I think this is ready for committer review, although now that I’m not just reviewing but hacking on this thing, maybe someone else should review it first. Patches attached, GitHub PR here: https://github.com/theory/postgres/pull/12 Best, David Attachments: [application/octet-stream] v6-0001-Rename-jsonpath-method-arg-tokens.patch (3.9K, 2-v6-0001-Rename-jsonpath-method-arg-tokens.patch) download | inline diff: From 86704d1c079b205ccdcb79da974a690d4dca2ad3 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" <[email protected]> Date: Mon, 2 Jun 2025 17:14:10 -0400 Subject: [PATCH v6 1/2] Rename jsonpath method arg tokens Rename the `csv_` tokens to `int_`, because they represent signed or unsigned integers, as follows: * `csv_elem` => `int_elem` * `csv_list` => `int_list` * `opt_csv_list` => `opt_int_list` Rename the `datetime_precision` tokens to `uint_arg`, as they represent unsigned integers and will be useful for other methods in the future, as follows: * `datetime_precision` => `uint_elem` * `opt_datetime_precision` => `opt_uint_arg` Rename the `datetime_template` tokens to `str_arg`, as they represent strings and will be useful for other methods in the future, as follows: * `datetime_template` => `str_elem` * `opt_datetime_template` => `opt_str_arg` --- src/backend/utils/adt/jsonpath_gram.y | 42 +++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..0b16cec18c4 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -92,10 +92,10 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %type <value> scalar_value path_primary expr array_accessor any_path accessor_op key predicate delimited_predicate index_elem starts_with_initial expr_or_predicate - datetime_template opt_datetime_template csv_elem - datetime_precision opt_datetime_precision + str_elem opt_str_arg int_elem + uint_elem opt_uint_arg -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr int_list opt_int_list %type <indexs> index_list @@ -254,7 +254,7 @@ accessor_op: | '.' any_path { $$ = $2; } | '.' method '(' ')' { $$ = makeItemType($2); } | '?' '(' predicate ')' { $$ = makeItemUnary(jpiFilter, $3); } - | '.' DECIMAL_P '(' opt_csv_list ')' + | '.' DECIMAL_P '(' opt_int_list ')' { if (list_length($4) == 0) $$ = makeItemBinary(jpiDecimal, NULL, NULL); @@ -268,19 +268,19 @@ accessor_op: errmsg("invalid input syntax for type %s", "jsonpath"), errdetail(".decimal() can only have an optional precision[,scale]."))); } - | '.' DATETIME_P '(' opt_datetime_template ')' + | '.' DATETIME_P '(' opt_str_arg ')' { $$ = makeItemUnary(jpiDatetime, $4); } - | '.' TIME_P '(' opt_datetime_precision ')' + | '.' TIME_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTime, $4); } - | '.' TIME_TZ_P '(' opt_datetime_precision ')' + | '.' TIME_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimeTz, $4); } - | '.' TIMESTAMP_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestamp, $4); } - | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } ; -csv_elem: +int_elem: INT_P { $$ = makeItemNumeric(&$1); } | '+' INT_P %prec UMINUS @@ -289,31 +289,31 @@ csv_elem: { $$ = makeItemUnary(jpiMinus, makeItemNumeric(&$2)); } ; -csv_list: - csv_elem { $$ = list_make1($1); } - | csv_list ',' csv_elem { $$ = lappend($1, $3); } +int_list: + int_elem { $$ = list_make1($1); } + | int_list ',' int_elem { $$ = lappend($1, $3); } ; -opt_csv_list: - csv_list { $$ = $1; } +opt_int_list: + int_list { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_precision: +uint_elem: INT_P { $$ = makeItemNumeric(&$1); } ; -opt_datetime_precision: - datetime_precision { $$ = $1; } +opt_uint_arg: + uint_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_template: +str_elem: STRING_P { $$ = makeItemString(&$1); } ; -opt_datetime_template: - datetime_template { $$ = $1; } +opt_str_arg: + str_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -- 2.49.0 [application/octet-stream] v6-0002-Add-additional-jsonpath-string-methods.patch (48.4K, 3-v6-0002-Add-additional-jsonpath-string-methods.patch) download | inline diff: From fc273e5445a38ff77ecdb77757dc2ff4fba42091 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Tue, 3 Jun 2025 14:58:36 -0400 Subject: [PATCH v6 2/2] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 140 +++++++ src/backend/utils/adt/jsonpath.c | 126 +++++- src/backend/utils/adt/jsonpath_exec.c | 200 ++++++++++ src/backend/utils/adt/jsonpath_gram.y | 29 +- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 8 + src/test/regress/expected/jsonb_jsonpath.out | 399 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 137 +++++++ src/test/regress/sql/jsonb_jsonpath.sql | 109 +++++ src/test/regress/sql/jsonpath.sql | 31 ++ 10 files changed, 1183 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c67688cbf5f..674c22c78d8 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18627,6 +18627,146 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start and end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..524f0f8daec 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -298,6 +298,8 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: { /* * First, reserve place for left/right arg's positions, then @@ -362,6 +364,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +462,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +839,72 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiStrReplace: + appendStringInfoString(buf, ".replace("); + if (v->content.args.left) + { + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.args.right) + { + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPart: + appendStringInfoString(buf, ".split_part("); + if (v->content.args.left) + { + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + if (v->content.args.right) + { + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrLower: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpper: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcap: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrim: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrim: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrim: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +980,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiStrReplace: + return "replace"; + case jpiStrLower: + return "lower"; + case jpiStrUpper: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +994,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrim: + return "ltrim"; + case jpiStrRtrim: + return "rtrim"; + case jpiStrBtrim: + return "btrim"; + case jpiStrInitcap: + return "initcap"; + case jpiStrSplitPart: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1106,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; case jpiString: case jpiKey: @@ -1041,6 +1134,8 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; @@ -1055,6 +1150,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1188,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1250,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiStrReplace || + v->type == jpiStrLower || + v->type == jpiStrUpper || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim || + v->type == jpiStrInitcap || + v->type == jpiStrSplitPart); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1201,11 +1310,14 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.right); } + bool jspGetBool(JsonPathItem *v) { @@ -1501,6 +1613,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrReplace: + case jpiStrLower: + case jpiStrUpper: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..2095f2a9c21 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrim: + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,187 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) { + Assert( jsp->type == jpiStrLower || + jsp->type == jpiStrUpper || + jsp->type == jpiStrReplace || + jsp->type == jpiStrLtrim || + jsp->type == jpiStrRtrim || + jsp->type == jpiStrBtrim || + jsp->type == jpiStrInitcap || + jsp->type == jpiStrSplitPart); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string value. The first argument to internal functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + if (jsp->content.arg) + { + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim; + break; + case jpiStrRtrim: + func = rtrim; + break; + case jpiStrBtrim: + func = btrim; + break; + default: ; + } + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim1; + break; + case jpiStrRtrim: + func = rtrim1; + break; + case jpiStrBtrim: + func = btrim1; + break; + default: ; + } + resStr = TextDatumGetCString(DirectFunctionCall1Coll(func, + DEFAULT_COLLATION_OID, str)); + break; + } + + case jpiStrLower: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpper: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcap: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrReplace: + { + char *from_str, *to_str; + int from_len, to_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPart: + { + char *from_str; + Numeric n; + int from_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 0b16cec18c4..a48d80f560f 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -44,6 +44,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, JsonPathParseItem ** result, struct Node *escontext); + /* * Bison doesn't allocate anything that needs to live across parser calls, * so we can easily have it use palloc instead of malloc. This prevents @@ -86,6 +87,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACE_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -95,7 +98,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, str_elem opt_str_arg int_elem uint_elem opt_uint_arg -%type <elems> accessor_expr int_list opt_int_list +%type <elems> accessor_expr int_list opt_int_list str_int_args str_str_args %type <indexs> index_list @@ -278,6 +281,16 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACE_P '(' str_str_args ')' + { $$ = makeItemBinary(jpiStrReplace, linitial($4), lsecond($4)); } + | '.' STR_SPLIT_PART_P '(' str_int_args ')' + { $$ = makeItemBinary(jpiStrSplitPart, linitial($4), lsecond($4)); } + | '.' STR_LTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrLtrim, $4); } + | '.' STR_RTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrRtrim, $4); } + | '.' STR_BTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrBtrim, $4); } ; int_elem: @@ -317,6 +330,14 @@ opt_str_arg: | /* EMPTY */ { $$ = NULL; } ; +str_int_args: + str_elem ',' int_elem { $$ = list_make2($1, $3) } + ; + +str_str_args: + str_elem ',' str_elem { $$ = list_make2($1, $3) } + ; + key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +378,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +397,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLower; } + | STR_UPPER_P { $$ = jpiStrUpper; } + | STR_INITCAP_P { $$ = jpiStrInitcap; } ; %% diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..69c61f98500 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -413,8 +413,13 @@ static const JsonPathKeyword keywords[] = { {4, true, TRUE_P, "true"}, {4, false, TYPE_P, "type"}, {4, false, WITH_P, "with"}, + {5, false, STR_BTRIM_P, "btrim"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -425,13 +430,16 @@ static const JsonPathKeyword keywords[] = { {7, false, BOOLEAN_P, "boolean"}, {7, false, CEILING_P, "ceiling"}, {7, false, DECIMAL_P, "decimal"}, + {7, false, STR_INITCAP_P, "initcap"}, {7, false, INTEGER_P, "integer"}, + {7, false, STR_REPLACE_P, "replace"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..2d0c53f7b06 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiStrReplace, /* .replace() item method */ + jpiStrLower, /* .lower() item method */ + jpiStrUpper, /* .upper() item method */ + jpiStrLtrim, /* .ltrim() item method */ + jpiStrRtrim, /* .rtrim() item method */ + jpiStrBtrim, /* .btrim() item method */ + jpiStrInitcap, /* .initcap() item method */ + jpiStrSplitPart, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..cea539717f9 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,405 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); + jsonb_path_query +------------------ + "trim" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + jsonb_path_query +------------------ + "ghi" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..a13022feabd 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,143 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + +-- Parse errors +select '$.replace("hello")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace("hello")'::jsonpath; + ^ +select '$.replace()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace()'::jsonpath; + ^ +select '$.replace("hello","bye","extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.replace("hello","bye","extra")'::jsonpath; + ^ +select '$.split_part("~@~")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part("~@~")'::jsonpath; + ^ +select '$.split_part()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part()'::jsonpath; + ^ +select '$.split_part("~@~", "hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.split_part("~@~", "hi")'::jsonpath; + ^ +select '$.split_part("~@~", 2, "extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.split_part("~@~", 2, "extra")'::jsonpath; + ^ +select '$.lower("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.lower("hi")'::jsonpath; + ^ +select '$.upper("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.upper("hi")'::jsonpath; + ^ +select '$.initcap("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.initcap("hi")'::jsonpath; + ^ +select '$.ltrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.ltrim(42)'::jsonpath; + ^ +select '$.ltrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.ltrim("x", "y")'::jsonpath; + ^ +select '$.rtrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.rtrim(42)'::jsonpath; + ^ +select '$.rtrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.rtrim("x", "y")'::jsonpath; + ^ +select '$.trim(42)'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim(42)'::jsonpath; + ^ +select '$.trim("x", "y")'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim("x", "y")'::jsonpath; + ^ select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..8169719343e 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,115 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..7dd2e57e7a6 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,37 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + +-- Parse errors +select '$.replace("hello")'::jsonpath; +select '$.replace()'::jsonpath; +select '$.replace("hello","bye","extra")'::jsonpath; +select '$.split_part("~@~")'::jsonpath; +select '$.split_part()'::jsonpath; +select '$.split_part("~@~", "hi")'::jsonpath; +select '$.split_part("~@~", 2, "extra")'::jsonpath; +select '$.lower("hi")'::jsonpath; +select '$.upper("hi")'::jsonpath; +select '$.initcap("hi")'::jsonpath; +select '$.ltrim(42)'::jsonpath; +select '$.ltrim("x", "y")'::jsonpath; +select '$.rtrim(42)'::jsonpath; +select '$.rtrim("x", "y")'::jsonpath; +select '$.trim(42)'::jsonpath; +select '$.trim("x", "y")'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 5-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-06-03 19:10 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-06-03 19:10 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jun 3, 2025, at 15:02, David E. Wheeler <[email protected]> wrote: > Patches attached, GitHub PR here: > > https://github.com/theory/postgres/pull/12 Found a little more unnecessary code to remove. Updated patches attached. Best, David Attachments: [application/octet-stream] v7-0001-Rename-jsonpath-method-arg-tokens.patch (3.9K, 2-v7-0001-Rename-jsonpath-method-arg-tokens.patch) download | inline diff: From 86704d1c079b205ccdcb79da974a690d4dca2ad3 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" <[email protected]> Date: Mon, 2 Jun 2025 17:14:10 -0400 Subject: [PATCH v7 1/2] Rename jsonpath method arg tokens Rename the `csv_` tokens to `int_`, because they represent signed or unsigned integers, as follows: * `csv_elem` => `int_elem` * `csv_list` => `int_list` * `opt_csv_list` => `opt_int_list` Rename the `datetime_precision` tokens to `uint_arg`, as they represent unsigned integers and will be useful for other methods in the future, as follows: * `datetime_precision` => `uint_elem` * `opt_datetime_precision` => `opt_uint_arg` Rename the `datetime_template` tokens to `str_arg`, as they represent strings and will be useful for other methods in the future, as follows: * `datetime_template` => `str_elem` * `opt_datetime_template` => `opt_str_arg` --- src/backend/utils/adt/jsonpath_gram.y | 42 +++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..0b16cec18c4 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -92,10 +92,10 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %type <value> scalar_value path_primary expr array_accessor any_path accessor_op key predicate delimited_predicate index_elem starts_with_initial expr_or_predicate - datetime_template opt_datetime_template csv_elem - datetime_precision opt_datetime_precision + str_elem opt_str_arg int_elem + uint_elem opt_uint_arg -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr int_list opt_int_list %type <indexs> index_list @@ -254,7 +254,7 @@ accessor_op: | '.' any_path { $$ = $2; } | '.' method '(' ')' { $$ = makeItemType($2); } | '?' '(' predicate ')' { $$ = makeItemUnary(jpiFilter, $3); } - | '.' DECIMAL_P '(' opt_csv_list ')' + | '.' DECIMAL_P '(' opt_int_list ')' { if (list_length($4) == 0) $$ = makeItemBinary(jpiDecimal, NULL, NULL); @@ -268,19 +268,19 @@ accessor_op: errmsg("invalid input syntax for type %s", "jsonpath"), errdetail(".decimal() can only have an optional precision[,scale]."))); } - | '.' DATETIME_P '(' opt_datetime_template ')' + | '.' DATETIME_P '(' opt_str_arg ')' { $$ = makeItemUnary(jpiDatetime, $4); } - | '.' TIME_P '(' opt_datetime_precision ')' + | '.' TIME_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTime, $4); } - | '.' TIME_TZ_P '(' opt_datetime_precision ')' + | '.' TIME_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimeTz, $4); } - | '.' TIMESTAMP_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestamp, $4); } - | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } ; -csv_elem: +int_elem: INT_P { $$ = makeItemNumeric(&$1); } | '+' INT_P %prec UMINUS @@ -289,31 +289,31 @@ csv_elem: { $$ = makeItemUnary(jpiMinus, makeItemNumeric(&$2)); } ; -csv_list: - csv_elem { $$ = list_make1($1); } - | csv_list ',' csv_elem { $$ = lappend($1, $3); } +int_list: + int_elem { $$ = list_make1($1); } + | int_list ',' int_elem { $$ = lappend($1, $3); } ; -opt_csv_list: - csv_list { $$ = $1; } +opt_int_list: + int_list { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_precision: +uint_elem: INT_P { $$ = makeItemNumeric(&$1); } ; -opt_datetime_precision: - datetime_precision { $$ = $1; } +opt_uint_arg: + uint_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_template: +str_elem: STRING_P { $$ = makeItemString(&$1); } ; -opt_datetime_template: - datetime_template { $$ = $1; } +opt_str_arg: + str_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -- 2.49.0 [application/octet-stream] v7-0002-Add-additional-jsonpath-string-methods.patch (47.9K, 3-v7-0002-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 1e80d72d20c466fc198043e059556d649adbaf5b Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Tue, 3 Jun 2025 15:09:38 -0400 Subject: [PATCH v7 2/2] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 140 +++++++ src/backend/utils/adt/jsonpath.c | 113 +++++- src/backend/utils/adt/jsonpath_exec.c | 200 ++++++++++ src/backend/utils/adt/jsonpath_gram.y | 28 +- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 8 + src/test/regress/expected/jsonb_jsonpath.out | 399 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 137 +++++++ src/test/regress/sql/jsonb_jsonpath.sql | 109 +++++ src/test/regress/sql/jsonpath.sql | 31 ++ 10 files changed, 1169 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c67688cbf5f..674c22c78d8 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18627,6 +18627,146 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start and end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..0872b115697 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -298,6 +298,8 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: { /* * First, reserve place for left/right arg's positions, then @@ -362,6 +364,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +462,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +839,60 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiStrReplace: + appendStringInfoString(buf, ".replace("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPart: + appendStringInfoString(buf, ".split_part("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrLower: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpper: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcap: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrim: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrim: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrim: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +968,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiStrReplace: + return "replace"; + case jpiStrLower: + return "lower"; + case jpiStrUpper: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +982,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrim: + return "ltrim"; + case jpiStrRtrim: + return "rtrim"; + case jpiStrBtrim: + return "btrim"; + case jpiStrInitcap: + return "initcap"; + case jpiStrSplitPart: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1094,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; case jpiString: case jpiKey: @@ -1041,6 +1122,8 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; @@ -1055,6 +1138,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1176,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1238,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiStrReplace || + v->type == jpiStrLower || + v->type == jpiStrUpper || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim || + v->type == jpiStrInitcap || + v->type == jpiStrSplitPart); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1201,7 +1298,9 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.right); } @@ -1501,6 +1600,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrReplace: + case jpiStrLower: + case jpiStrUpper: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..2095f2a9c21 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrim: + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,187 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) { + Assert( jsp->type == jpiStrLower || + jsp->type == jpiStrUpper || + jsp->type == jpiStrReplace || + jsp->type == jpiStrLtrim || + jsp->type == jpiStrRtrim || + jsp->type == jpiStrBtrim || + jsp->type == jpiStrInitcap || + jsp->type == jpiStrSplitPart); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string value. The first argument to internal functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + if (jsp->content.arg) + { + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim; + break; + case jpiStrRtrim: + func = rtrim; + break; + case jpiStrBtrim: + func = btrim; + break; + default: ; + } + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim1; + break; + case jpiStrRtrim: + func = rtrim1; + break; + case jpiStrBtrim: + func = btrim1; + break; + default: ; + } + resStr = TextDatumGetCString(DirectFunctionCall1Coll(func, + DEFAULT_COLLATION_OID, str)); + break; + } + + case jpiStrLower: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpper: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcap: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrReplace: + { + char *from_str, *to_str; + int from_len, to_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPart: + { + char *from_str; + Numeric n; + int from_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 0b16cec18c4..d4bc3d6034b 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -86,6 +86,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACE_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -95,7 +97,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, str_elem opt_str_arg int_elem uint_elem opt_uint_arg -%type <elems> accessor_expr int_list opt_int_list +%type <elems> accessor_expr int_list opt_int_list str_int_args str_str_args %type <indexs> index_list @@ -278,6 +280,16 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACE_P '(' str_str_args ')' + { $$ = makeItemBinary(jpiStrReplace, linitial($4), lsecond($4)); } + | '.' STR_SPLIT_PART_P '(' str_int_args ')' + { $$ = makeItemBinary(jpiStrSplitPart, linitial($4), lsecond($4)); } + | '.' STR_LTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrLtrim, $4); } + | '.' STR_RTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrRtrim, $4); } + | '.' STR_BTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrBtrim, $4); } ; int_elem: @@ -317,6 +329,14 @@ opt_str_arg: | /* EMPTY */ { $$ = NULL; } ; +str_int_args: + str_elem ',' int_elem { $$ = list_make2($1, $3) } + ; + +str_str_args: + str_elem ',' str_elem { $$ = list_make2($1, $3) } + ; + key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +377,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +396,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLower; } + | STR_UPPER_P { $$ = jpiStrUpper; } + | STR_INITCAP_P { $$ = jpiStrInitcap; } ; %% diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..69c61f98500 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -413,8 +413,13 @@ static const JsonPathKeyword keywords[] = { {4, true, TRUE_P, "true"}, {4, false, TYPE_P, "type"}, {4, false, WITH_P, "with"}, + {5, false, STR_BTRIM_P, "btrim"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -425,13 +430,16 @@ static const JsonPathKeyword keywords[] = { {7, false, BOOLEAN_P, "boolean"}, {7, false, CEILING_P, "ceiling"}, {7, false, DECIMAL_P, "decimal"}, + {7, false, STR_INITCAP_P, "initcap"}, {7, false, INTEGER_P, "integer"}, + {7, false, STR_REPLACE_P, "replace"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..2d0c53f7b06 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiStrReplace, /* .replace() item method */ + jpiStrLower, /* .lower() item method */ + jpiStrUpper, /* .upper() item method */ + jpiStrLtrim, /* .ltrim() item method */ + jpiStrRtrim, /* .rtrim() item method */ + jpiStrBtrim, /* .btrim() item method */ + jpiStrInitcap, /* .initcap() item method */ + jpiStrSplitPart, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..cea539717f9 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,405 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); + jsonb_path_query +------------------ + "trim" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + jsonb_path_query +------------------ + "ghi" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..a13022feabd 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,143 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + +-- Parse errors +select '$.replace("hello")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace("hello")'::jsonpath; + ^ +select '$.replace()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace()'::jsonpath; + ^ +select '$.replace("hello","bye","extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.replace("hello","bye","extra")'::jsonpath; + ^ +select '$.split_part("~@~")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part("~@~")'::jsonpath; + ^ +select '$.split_part()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part()'::jsonpath; + ^ +select '$.split_part("~@~", "hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.split_part("~@~", "hi")'::jsonpath; + ^ +select '$.split_part("~@~", 2, "extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.split_part("~@~", 2, "extra")'::jsonpath; + ^ +select '$.lower("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.lower("hi")'::jsonpath; + ^ +select '$.upper("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.upper("hi")'::jsonpath; + ^ +select '$.initcap("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.initcap("hi")'::jsonpath; + ^ +select '$.ltrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.ltrim(42)'::jsonpath; + ^ +select '$.ltrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.ltrim("x", "y")'::jsonpath; + ^ +select '$.rtrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.rtrim(42)'::jsonpath; + ^ +select '$.rtrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.rtrim("x", "y")'::jsonpath; + ^ +select '$.trim(42)'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim(42)'::jsonpath; + ^ +select '$.trim("x", "y")'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim("x", "y")'::jsonpath; + ^ select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..8169719343e 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,115 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..7dd2e57e7a6 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,37 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + +-- Parse errors +select '$.replace("hello")'::jsonpath; +select '$.replace()'::jsonpath; +select '$.replace("hello","bye","extra")'::jsonpath; +select '$.split_part("~@~")'::jsonpath; +select '$.split_part()'::jsonpath; +select '$.split_part("~@~", "hi")'::jsonpath; +select '$.split_part("~@~", 2, "extra")'::jsonpath; +select '$.lower("hi")'::jsonpath; +select '$.upper("hi")'::jsonpath; +select '$.initcap("hi")'::jsonpath; +select '$.ltrim(42)'::jsonpath; +select '$.ltrim("x", "y")'::jsonpath; +select '$.rtrim(42)'::jsonpath; +select '$.rtrim("x", "y")'::jsonpath; +select '$.trim(42)'::jsonpath; +select '$.trim("x", "y")'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 4-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-06-04 15:27 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-06-04 15:27 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jun 3, 2025, at 15:10, David E. Wheeler <[email protected]> wrote: >> https://github.com/theory/postgres/pull/12 > > Found a little more unnecessary code to remove. Updated patches attached. And these should fix the CI failure. I also ran pgindent. Best, David Attachments: [application/octet-stream] v8-0001-Rename-jsonpath-method-arg-tokens.patch (3.9K, 2-v8-0001-Rename-jsonpath-method-arg-tokens.patch) download | inline diff: From 86704d1c079b205ccdcb79da974a690d4dca2ad3 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" <[email protected]> Date: Mon, 2 Jun 2025 17:14:10 -0400 Subject: [PATCH v8 1/2] Rename jsonpath method arg tokens Rename the `csv_` tokens to `int_`, because they represent signed or unsigned integers, as follows: * `csv_elem` => `int_elem` * `csv_list` => `int_list` * `opt_csv_list` => `opt_int_list` Rename the `datetime_precision` tokens to `uint_arg`, as they represent unsigned integers and will be useful for other methods in the future, as follows: * `datetime_precision` => `uint_elem` * `opt_datetime_precision` => `opt_uint_arg` Rename the `datetime_template` tokens to `str_arg`, as they represent strings and will be useful for other methods in the future, as follows: * `datetime_template` => `str_elem` * `opt_datetime_template` => `opt_str_arg` --- src/backend/utils/adt/jsonpath_gram.y | 42 +++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..0b16cec18c4 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -92,10 +92,10 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %type <value> scalar_value path_primary expr array_accessor any_path accessor_op key predicate delimited_predicate index_elem starts_with_initial expr_or_predicate - datetime_template opt_datetime_template csv_elem - datetime_precision opt_datetime_precision + str_elem opt_str_arg int_elem + uint_elem opt_uint_arg -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr int_list opt_int_list %type <indexs> index_list @@ -254,7 +254,7 @@ accessor_op: | '.' any_path { $$ = $2; } | '.' method '(' ')' { $$ = makeItemType($2); } | '?' '(' predicate ')' { $$ = makeItemUnary(jpiFilter, $3); } - | '.' DECIMAL_P '(' opt_csv_list ')' + | '.' DECIMAL_P '(' opt_int_list ')' { if (list_length($4) == 0) $$ = makeItemBinary(jpiDecimal, NULL, NULL); @@ -268,19 +268,19 @@ accessor_op: errmsg("invalid input syntax for type %s", "jsonpath"), errdetail(".decimal() can only have an optional precision[,scale]."))); } - | '.' DATETIME_P '(' opt_datetime_template ')' + | '.' DATETIME_P '(' opt_str_arg ')' { $$ = makeItemUnary(jpiDatetime, $4); } - | '.' TIME_P '(' opt_datetime_precision ')' + | '.' TIME_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTime, $4); } - | '.' TIME_TZ_P '(' opt_datetime_precision ')' + | '.' TIME_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimeTz, $4); } - | '.' TIMESTAMP_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestamp, $4); } - | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } ; -csv_elem: +int_elem: INT_P { $$ = makeItemNumeric(&$1); } | '+' INT_P %prec UMINUS @@ -289,31 +289,31 @@ csv_elem: { $$ = makeItemUnary(jpiMinus, makeItemNumeric(&$2)); } ; -csv_list: - csv_elem { $$ = list_make1($1); } - | csv_list ',' csv_elem { $$ = lappend($1, $3); } +int_list: + int_elem { $$ = list_make1($1); } + | int_list ',' int_elem { $$ = lappend($1, $3); } ; -opt_csv_list: - csv_list { $$ = $1; } +opt_int_list: + int_list { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_precision: +uint_elem: INT_P { $$ = makeItemNumeric(&$1); } ; -opt_datetime_precision: - datetime_precision { $$ = $1; } +opt_uint_arg: + uint_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_template: +str_elem: STRING_P { $$ = makeItemString(&$1); } ; -opt_datetime_template: - datetime_template { $$ = $1; } +opt_str_arg: + str_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -- 2.49.0 [application/octet-stream] v8-0002-Add-additional-jsonpath-string-methods.patch (48.2K, 3-v8-0002-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 8f2146ad7c27a9b2aa094878e647c6bb6327df9b Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Wed, 4 Jun 2025 11:21:23 -0400 Subject: [PATCH v8 2/2] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 140 +++++++ src/backend/utils/adt/jsonpath.c | 113 +++++- src/backend/utils/adt/jsonpath_exec.c | 206 ++++++++++ src/backend/utils/adt/jsonpath_gram.y | 28 +- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 8 + src/test/regress/expected/jsonb_jsonpath.out | 399 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 137 +++++++ src/test/regress/sql/jsonb_jsonpath.sql | 109 +++++ src/test/regress/sql/jsonpath.sql | 31 ++ 10 files changed, 1175 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c67688cbf5f..674c22c78d8 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18627,6 +18627,146 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start and end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..0872b115697 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -298,6 +298,8 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: { /* * First, reserve place for left/right arg's positions, then @@ -362,6 +364,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +462,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +839,60 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiStrReplace: + appendStringInfoString(buf, ".replace("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPart: + appendStringInfoString(buf, ".split_part("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrLower: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpper: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcap: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrim: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrim: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrim: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +968,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiStrReplace: + return "replace"; + case jpiStrLower: + return "lower"; + case jpiStrUpper: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +982,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrim: + return "ltrim"; + case jpiStrRtrim: + return "rtrim"; + case jpiStrBtrim: + return "btrim"; + case jpiStrInitcap: + return "initcap"; + case jpiStrSplitPart: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1094,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; case jpiString: case jpiKey: @@ -1041,6 +1122,8 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; @@ -1055,6 +1138,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1176,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1238,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiStrReplace || + v->type == jpiStrLower || + v->type == jpiStrUpper || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim || + v->type == jpiStrInitcap || + v->type == jpiStrSplitPart); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1201,7 +1298,9 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.right); } @@ -1501,6 +1600,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrReplace: + case jpiStrLower: + case jpiStrUpper: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..28a04b24bfa 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrim: + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,193 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult +executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) +{ + Assert(jsp->type == jpiStrLower || + jsp->type == jpiStrUpper || + jsp->type == jpiStrReplace || + jsp->type == jpiStrLtrim || + jsp->type == jpiStrRtrim || + jsp->type == jpiStrBtrim || + jsp->type == jpiStrInitcap || + jsp->type == jpiStrSplitPart); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string + * value. The first argument to internal + * functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + if (jsp->content.arg) + { + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim; + break; + case jpiStrRtrim: + func = rtrim; + break; + case jpiStrBtrim: + func = btrim; + break; + default:; + } + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim1; + break; + case jpiStrRtrim: + func = rtrim1; + break; + case jpiStrBtrim: + func = btrim1; + break; + default:; + } + resStr = TextDatumGetCString(DirectFunctionCall1Coll(func, + DEFAULT_COLLATION_OID, str)); + break; + } + + case jpiStrLower: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpper: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcap: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrReplace: + { + char *from_str, + *to_str; + int from_len, + to_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPart: + { + char *from_str; + Numeric n; + int from_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 0b16cec18c4..53bf779e152 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -86,6 +86,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACE_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -95,7 +97,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, str_elem opt_str_arg int_elem uint_elem opt_uint_arg -%type <elems> accessor_expr int_list opt_int_list +%type <elems> accessor_expr int_list opt_int_list str_int_args str_str_args %type <indexs> index_list @@ -278,6 +280,16 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACE_P '(' str_str_args ')' + { $$ = makeItemBinary(jpiStrReplace, linitial($4), lsecond($4)); } + | '.' STR_SPLIT_PART_P '(' str_int_args ')' + { $$ = makeItemBinary(jpiStrSplitPart, linitial($4), lsecond($4)); } + | '.' STR_LTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrLtrim, $4); } + | '.' STR_RTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrRtrim, $4); } + | '.' STR_BTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrBtrim, $4); } ; int_elem: @@ -317,6 +329,14 @@ opt_str_arg: | /* EMPTY */ { $$ = NULL; } ; +str_int_args: + str_elem ',' int_elem { $$ = list_make2($1, $3); } + ; + +str_str_args: + str_elem ',' str_elem { $$ = list_make2($1, $3); } + ; + key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +377,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +396,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLower; } + | STR_UPPER_P { $$ = jpiStrUpper; } + | STR_INITCAP_P { $$ = jpiStrInitcap; } ; %% diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..69c61f98500 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -413,8 +413,13 @@ static const JsonPathKeyword keywords[] = { {4, true, TRUE_P, "true"}, {4, false, TYPE_P, "type"}, {4, false, WITH_P, "with"}, + {5, false, STR_BTRIM_P, "btrim"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -425,13 +430,16 @@ static const JsonPathKeyword keywords[] = { {7, false, BOOLEAN_P, "boolean"}, {7, false, CEILING_P, "ceiling"}, {7, false, DECIMAL_P, "decimal"}, + {7, false, STR_INITCAP_P, "initcap"}, {7, false, INTEGER_P, "integer"}, + {7, false, STR_REPLACE_P, "replace"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..2d0c53f7b06 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiStrReplace, /* .replace() item method */ + jpiStrLower, /* .lower() item method */ + jpiStrUpper, /* .upper() item method */ + jpiStrLtrim, /* .ltrim() item method */ + jpiStrRtrim, /* .rtrim() item method */ + jpiStrBtrim, /* .btrim() item method */ + jpiStrInitcap, /* .initcap() item method */ + jpiStrSplitPart, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..cea539717f9 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,405 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); + jsonb_path_query +------------------ + "trim" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + jsonb_path_query +------------------ + "ghi" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..a13022feabd 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,143 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + +-- Parse errors +select '$.replace("hello")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace("hello")'::jsonpath; + ^ +select '$.replace()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace()'::jsonpath; + ^ +select '$.replace("hello","bye","extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.replace("hello","bye","extra")'::jsonpath; + ^ +select '$.split_part("~@~")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part("~@~")'::jsonpath; + ^ +select '$.split_part()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part()'::jsonpath; + ^ +select '$.split_part("~@~", "hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.split_part("~@~", "hi")'::jsonpath; + ^ +select '$.split_part("~@~", 2, "extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.split_part("~@~", 2, "extra")'::jsonpath; + ^ +select '$.lower("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.lower("hi")'::jsonpath; + ^ +select '$.upper("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.upper("hi")'::jsonpath; + ^ +select '$.initcap("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.initcap("hi")'::jsonpath; + ^ +select '$.ltrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.ltrim(42)'::jsonpath; + ^ +select '$.ltrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.ltrim("x", "y")'::jsonpath; + ^ +select '$.rtrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.rtrim(42)'::jsonpath; + ^ +select '$.rtrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.rtrim("x", "y")'::jsonpath; + ^ +select '$.trim(42)'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim(42)'::jsonpath; + ^ +select '$.trim("x", "y")'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim("x", "y")'::jsonpath; + ^ select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..8169719343e 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,115 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..7dd2e57e7a6 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,37 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + +-- Parse errors +select '$.replace("hello")'::jsonpath; +select '$.replace()'::jsonpath; +select '$.replace("hello","bye","extra")'::jsonpath; +select '$.split_part("~@~")'::jsonpath; +select '$.split_part()'::jsonpath; +select '$.split_part("~@~", "hi")'::jsonpath; +select '$.split_part("~@~", 2, "extra")'::jsonpath; +select '$.lower("hi")'::jsonpath; +select '$.upper("hi")'::jsonpath; +select '$.initcap("hi")'::jsonpath; +select '$.ltrim(42)'::jsonpath; +select '$.ltrim("x", "y")'::jsonpath; +select '$.rtrim(42)'::jsonpath; +select '$.rtrim("x", "y")'::jsonpath; +select '$.trim(42)'::jsonpath; +select '$.trim("x", "y")'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 4-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-06-14 15:08 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-06-14 15:08 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jun 4, 2025, at 11:27, David E. Wheeler <[email protected]> wrote: > And these should fix the CI failure. I also ran pgindent. Here’s a quick rebase. I think it’s ready for committer review, but since I’ve poked at it quite a bit myself, I updated the Commitfest item [1] to “Needs Review”. Best, David [1]: https://commitfest.postgresql.org/patch/5270/ Attachments: [application/octet-stream] v9-0001-Rename-jsonpath-method-arg-tokens.patch (3.9K, 2-v9-0001-Rename-jsonpath-method-arg-tokens.patch) download | inline diff: From 11c3b981fbf80eaa6f21130b2e9e59df4855df2c Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" <[email protected]> Date: Mon, 2 Jun 2025 17:14:10 -0400 Subject: [PATCH v9 1/2] Rename jsonpath method arg tokens Rename the `csv_` tokens to `int_`, because they represent signed or unsigned integers, as follows: * `csv_elem` => `int_elem` * `csv_list` => `int_list` * `opt_csv_list` => `opt_int_list` Rename the `datetime_precision` tokens to `uint_arg`, as they represent unsigned integers and will be useful for other methods in the future, as follows: * `datetime_precision` => `uint_elem` * `opt_datetime_precision` => `opt_uint_arg` Rename the `datetime_template` tokens to `str_arg`, as they represent strings and will be useful for other methods in the future, as follows: * `datetime_template` => `str_elem` * `opt_datetime_template` => `opt_str_arg` --- src/backend/utils/adt/jsonpath_gram.y | 42 +++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..0b16cec18c4 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -92,10 +92,10 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %type <value> scalar_value path_primary expr array_accessor any_path accessor_op key predicate delimited_predicate index_elem starts_with_initial expr_or_predicate - datetime_template opt_datetime_template csv_elem - datetime_precision opt_datetime_precision + str_elem opt_str_arg int_elem + uint_elem opt_uint_arg -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr int_list opt_int_list %type <indexs> index_list @@ -254,7 +254,7 @@ accessor_op: | '.' any_path { $$ = $2; } | '.' method '(' ')' { $$ = makeItemType($2); } | '?' '(' predicate ')' { $$ = makeItemUnary(jpiFilter, $3); } - | '.' DECIMAL_P '(' opt_csv_list ')' + | '.' DECIMAL_P '(' opt_int_list ')' { if (list_length($4) == 0) $$ = makeItemBinary(jpiDecimal, NULL, NULL); @@ -268,19 +268,19 @@ accessor_op: errmsg("invalid input syntax for type %s", "jsonpath"), errdetail(".decimal() can only have an optional precision[,scale]."))); } - | '.' DATETIME_P '(' opt_datetime_template ')' + | '.' DATETIME_P '(' opt_str_arg ')' { $$ = makeItemUnary(jpiDatetime, $4); } - | '.' TIME_P '(' opt_datetime_precision ')' + | '.' TIME_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTime, $4); } - | '.' TIME_TZ_P '(' opt_datetime_precision ')' + | '.' TIME_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimeTz, $4); } - | '.' TIMESTAMP_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestamp, $4); } - | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } ; -csv_elem: +int_elem: INT_P { $$ = makeItemNumeric(&$1); } | '+' INT_P %prec UMINUS @@ -289,31 +289,31 @@ csv_elem: { $$ = makeItemUnary(jpiMinus, makeItemNumeric(&$2)); } ; -csv_list: - csv_elem { $$ = list_make1($1); } - | csv_list ',' csv_elem { $$ = lappend($1, $3); } +int_list: + int_elem { $$ = list_make1($1); } + | int_list ',' int_elem { $$ = lappend($1, $3); } ; -opt_csv_list: - csv_list { $$ = $1; } +opt_int_list: + int_list { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_precision: +uint_elem: INT_P { $$ = makeItemNumeric(&$1); } ; -opt_datetime_precision: - datetime_precision { $$ = $1; } +opt_uint_arg: + uint_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_template: +str_elem: STRING_P { $$ = makeItemString(&$1); } ; -opt_datetime_template: - datetime_template { $$ = $1; } +opt_str_arg: + str_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -- 2.49.0 [application/octet-stream] v9-0002-Add-additional-jsonpath-string-methods.patch (48.2K, 3-v9-0002-Add-additional-jsonpath-string-methods.patch) download | inline diff: From ea19345548e215aedff6ef03274a2455b56a05ad Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Wed, 4 Jun 2025 11:21:23 -0400 Subject: [PATCH v9 2/2] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 140 +++++++ src/backend/utils/adt/jsonpath.c | 113 +++++- src/backend/utils/adt/jsonpath_exec.c | 206 ++++++++++ src/backend/utils/adt/jsonpath_gram.y | 28 +- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 8 + src/test/regress/expected/jsonb_jsonpath.out | 399 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 137 +++++++ src/test/regress/sql/jsonb_jsonpath.sql | 109 +++++ src/test/regress/sql/jsonpath.sql | 31 ++ 10 files changed, 1175 insertions(+), 4 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c67688cbf5f..674c22c78d8 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18627,6 +18627,146 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start and end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..0872b115697 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -298,6 +298,8 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: { /* * First, reserve place for left/right arg's positions, then @@ -362,6 +364,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +462,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +839,60 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiStrReplace: + appendStringInfoString(buf, ".replace("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPart: + appendStringInfoString(buf, ".split_part("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrLower: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpper: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcap: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrim: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrim: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrim: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +968,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiStrReplace: + return "replace"; + case jpiStrLower: + return "lower"; + case jpiStrUpper: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +982,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrim: + return "ltrim"; + case jpiStrRtrim: + return "rtrim"; + case jpiStrBtrim: + return "btrim"; + case jpiStrInitcap: + return "initcap"; + case jpiStrSplitPart: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1094,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; case jpiString: case jpiKey: @@ -1041,6 +1122,8 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; @@ -1055,6 +1138,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1176,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1238,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiStrReplace || + v->type == jpiStrLower || + v->type == jpiStrUpper || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim || + v->type == jpiStrInitcap || + v->type == jpiStrSplitPart); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1201,7 +1298,9 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.right); } @@ -1501,6 +1600,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrReplace: + case jpiStrLower: + case jpiStrUpper: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..28a04b24bfa 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrim: + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,193 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult +executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) +{ + Assert(jsp->type == jpiStrLower || + jsp->type == jpiStrUpper || + jsp->type == jpiStrReplace || + jsp->type == jpiStrLtrim || + jsp->type == jpiStrRtrim || + jsp->type == jpiStrBtrim || + jsp->type == jpiStrInitcap || + jsp->type == jpiStrSplitPart); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string + * value. The first argument to internal + * functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + if (jsp->content.arg) + { + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim; + break; + case jpiStrRtrim: + func = rtrim; + break; + case jpiStrBtrim: + func = btrim; + break; + default:; + } + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim1; + break; + case jpiStrRtrim: + func = rtrim1; + break; + case jpiStrBtrim: + func = btrim1; + break; + default:; + } + resStr = TextDatumGetCString(DirectFunctionCall1Coll(func, + DEFAULT_COLLATION_OID, str)); + break; + } + + case jpiStrLower: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpper: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcap: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrReplace: + { + char *from_str, + *to_str; + int from_len, + to_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPart: + { + char *from_str; + Numeric n; + int from_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 0b16cec18c4..53bf779e152 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -86,6 +86,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACE_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -95,7 +97,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, str_elem opt_str_arg int_elem uint_elem opt_uint_arg -%type <elems> accessor_expr int_list opt_int_list +%type <elems> accessor_expr int_list opt_int_list str_int_args str_str_args %type <indexs> index_list @@ -278,6 +280,16 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACE_P '(' str_str_args ')' + { $$ = makeItemBinary(jpiStrReplace, linitial($4), lsecond($4)); } + | '.' STR_SPLIT_PART_P '(' str_int_args ')' + { $$ = makeItemBinary(jpiStrSplitPart, linitial($4), lsecond($4)); } + | '.' STR_LTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrLtrim, $4); } + | '.' STR_RTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrRtrim, $4); } + | '.' STR_BTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrBtrim, $4); } ; int_elem: @@ -317,6 +329,14 @@ opt_str_arg: | /* EMPTY */ { $$ = NULL; } ; +str_int_args: + str_elem ',' int_elem { $$ = list_make2($1, $3); } + ; + +str_str_args: + str_elem ',' str_elem { $$ = list_make2($1, $3); } + ; + key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +377,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +396,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLower; } + | STR_UPPER_P { $$ = jpiStrUpper; } + | STR_INITCAP_P { $$ = jpiStrInitcap; } ; %% diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..69c61f98500 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -413,8 +413,13 @@ static const JsonPathKeyword keywords[] = { {4, true, TRUE_P, "true"}, {4, false, TYPE_P, "type"}, {4, false, WITH_P, "with"}, + {5, false, STR_BTRIM_P, "btrim"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -425,13 +430,16 @@ static const JsonPathKeyword keywords[] = { {7, false, BOOLEAN_P, "boolean"}, {7, false, CEILING_P, "ceiling"}, {7, false, DECIMAL_P, "decimal"}, + {7, false, STR_INITCAP_P, "initcap"}, {7, false, INTEGER_P, "integer"}, + {7, false, STR_REPLACE_P, "replace"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..2d0c53f7b06 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiStrReplace, /* .replace() item method */ + jpiStrLower, /* .lower() item method */ + jpiStrUpper, /* .upper() item method */ + jpiStrLtrim, /* .ltrim() item method */ + jpiStrRtrim, /* .rtrim() item method */ + jpiStrBtrim, /* .btrim() item method */ + jpiStrInitcap, /* .initcap() item method */ + jpiStrSplitPart, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..cea539717f9 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,405 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); + jsonb_path_query +------------------ + "trim" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + jsonb_path_query +------------------ + "ghi" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..a13022feabd 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,143 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + +-- Parse errors +select '$.replace("hello")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace("hello")'::jsonpath; + ^ +select '$.replace()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace()'::jsonpath; + ^ +select '$.replace("hello","bye","extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.replace("hello","bye","extra")'::jsonpath; + ^ +select '$.split_part("~@~")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part("~@~")'::jsonpath; + ^ +select '$.split_part()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part()'::jsonpath; + ^ +select '$.split_part("~@~", "hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.split_part("~@~", "hi")'::jsonpath; + ^ +select '$.split_part("~@~", 2, "extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.split_part("~@~", 2, "extra")'::jsonpath; + ^ +select '$.lower("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.lower("hi")'::jsonpath; + ^ +select '$.upper("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.upper("hi")'::jsonpath; + ^ +select '$.initcap("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.initcap("hi")'::jsonpath; + ^ +select '$.ltrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.ltrim(42)'::jsonpath; + ^ +select '$.ltrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.ltrim("x", "y")'::jsonpath; + ^ +select '$.rtrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.rtrim(42)'::jsonpath; + ^ +select '$.rtrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.rtrim("x", "y")'::jsonpath; + ^ +select '$.trim(42)'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim(42)'::jsonpath; + ^ +select '$.trim("x", "y")'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim("x", "y")'::jsonpath; + ^ select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..8169719343e 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,115 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..7dd2e57e7a6 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,37 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + +-- Parse errors +select '$.replace("hello")'::jsonpath; +select '$.replace()'::jsonpath; +select '$.replace("hello","bye","extra")'::jsonpath; +select '$.split_part("~@~")'::jsonpath; +select '$.split_part()'::jsonpath; +select '$.split_part("~@~", "hi")'::jsonpath; +select '$.split_part("~@~", 2, "extra")'::jsonpath; +select '$.lower("hi")'::jsonpath; +select '$.upper("hi")'::jsonpath; +select '$.initcap("hi")'::jsonpath; +select '$.ltrim(42)'::jsonpath; +select '$.ltrim("x", "y")'::jsonpath; +select '$.rtrim(42)'::jsonpath; +select '$.rtrim("x", "y")'::jsonpath; +select '$.trim(42)'::jsonpath; +select '$.trim("x", "y")'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 5-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-07-10 17:41 Florents Tselai <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-07-10 17:41 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> > On 14 Jun 2025, at 6:08 PM, David E. Wheeler <[email protected]> wrote: > > On Jun 4, 2025, at 11:27, David E. Wheeler <[email protected]> wrote: > >> And these should fix the CI failure. I also ran pgindent. > > Here’s a quick rebase. I think it’s ready for committer review, but since I’ve poked at it quite a bit myself, I updated the Commitfest item [1] to “Needs Review”. > > Best, > > David > > [1]: https://commitfest.postgresql.org/patch/5270/ > > <v9-0001-Rename-jsonpath-method-arg-tokens.patch><v9-0002-Add-additional-jsonpath-string-methods.patch> > The basic problem I see with these latest revisions/refactorings is that they fail for pg_upgrade afaict. Probably this means that some of the rearrangements on the parser/scanner are not that flexible. ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-07-10 18:13 David E. Wheeler <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-07-10 18:13 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jul 10, 2025, at 13:41, Florents Tselai <[email protected]> wrote: > The basic problem I see with these latest revisions/refactorings is that they fail for pg_upgrade afaict. > Probably this means that some of the rearrangements on the parser/scanner are not that flexible. Oh, is that what’s happening? What needs to happen to properly support pg_upgrade? Best, David Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-07-10 22:40 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-07-10 22:40 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jul 10, 2025, at 14:13, David E. Wheeler <[email protected]> wrote: > Oh, is that what’s happening? What needs to happen to properly support pg_upgrade? Turns out there was an assertion failure that David Johnson spotted in the core dump of the test output and then in the regress log. Turns out I wasn’t using `--enable-assert` in my testing. With that I was able to replicate it and find the core dump in the “Crash Reports” tab of the macOS Console.app with this line: {"imageOffset":8079100,"sourceLine":1265,"sourceFile":"jsonpath.c","symbol":"jspGetLeftArg","imageIndex":0,"symbolLocation":348}, When I switched to using jspGetLeftArg and jspGetRightArg in the last patch, I forgot to add the assertions you originally had in your patch, Florents. Resolved in the attached, which now passes `make check-world` for me. Also available as a pull request[1]. Best, David [1] https://github.com/theory/postgres/pull/12/files Attachments: [application/octet-stream] v10-0001-Rename-jsonpath-method-arg-tokens.patch (3.9K, 2-v10-0001-Rename-jsonpath-method-arg-tokens.patch) download | inline diff: From b63773f20c615a79b84522cfcb1b68d4bdfdc7a9 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" <[email protected]> Date: Mon, 2 Jun 2025 17:14:10 -0400 Subject: [PATCH v10 1/2] Rename jsonpath method arg tokens Rename the `csv_` tokens to `int_`, because they represent signed or unsigned integers, as follows: * `csv_elem` => `int_elem` * `csv_list` => `int_list` * `opt_csv_list` => `opt_int_list` Rename the `datetime_precision` tokens to `uint_arg`, as they represent unsigned integers and will be useful for other methods in the future, as follows: * `datetime_precision` => `uint_elem` * `opt_datetime_precision` => `opt_uint_arg` Rename the `datetime_template` tokens to `str_arg`, as they represent strings and will be useful for other methods in the future, as follows: * `datetime_template` => `str_elem` * `opt_datetime_template` => `opt_str_arg` --- src/backend/utils/adt/jsonpath_gram.y | 42 +++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..0b16cec18c4 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -92,10 +92,10 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %type <value> scalar_value path_primary expr array_accessor any_path accessor_op key predicate delimited_predicate index_elem starts_with_initial expr_or_predicate - datetime_template opt_datetime_template csv_elem - datetime_precision opt_datetime_precision + str_elem opt_str_arg int_elem + uint_elem opt_uint_arg -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr int_list opt_int_list %type <indexs> index_list @@ -254,7 +254,7 @@ accessor_op: | '.' any_path { $$ = $2; } | '.' method '(' ')' { $$ = makeItemType($2); } | '?' '(' predicate ')' { $$ = makeItemUnary(jpiFilter, $3); } - | '.' DECIMAL_P '(' opt_csv_list ')' + | '.' DECIMAL_P '(' opt_int_list ')' { if (list_length($4) == 0) $$ = makeItemBinary(jpiDecimal, NULL, NULL); @@ -268,19 +268,19 @@ accessor_op: errmsg("invalid input syntax for type %s", "jsonpath"), errdetail(".decimal() can only have an optional precision[,scale]."))); } - | '.' DATETIME_P '(' opt_datetime_template ')' + | '.' DATETIME_P '(' opt_str_arg ')' { $$ = makeItemUnary(jpiDatetime, $4); } - | '.' TIME_P '(' opt_datetime_precision ')' + | '.' TIME_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTime, $4); } - | '.' TIME_TZ_P '(' opt_datetime_precision ')' + | '.' TIME_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimeTz, $4); } - | '.' TIMESTAMP_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestamp, $4); } - | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } ; -csv_elem: +int_elem: INT_P { $$ = makeItemNumeric(&$1); } | '+' INT_P %prec UMINUS @@ -289,31 +289,31 @@ csv_elem: { $$ = makeItemUnary(jpiMinus, makeItemNumeric(&$2)); } ; -csv_list: - csv_elem { $$ = list_make1($1); } - | csv_list ',' csv_elem { $$ = lappend($1, $3); } +int_list: + int_elem { $$ = list_make1($1); } + | int_list ',' int_elem { $$ = lappend($1, $3); } ; -opt_csv_list: - csv_list { $$ = $1; } +opt_int_list: + int_list { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_precision: +uint_elem: INT_P { $$ = makeItemNumeric(&$1); } ; -opt_datetime_precision: - datetime_precision { $$ = $1; } +opt_uint_arg: + uint_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_template: +str_elem: STRING_P { $$ = makeItemString(&$1); } ; -opt_datetime_template: - datetime_template { $$ = $1; } +opt_str_arg: + str_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -- 2.49.0 [application/octet-stream] v10-0002-Add-additional-jsonpath-string-methods.patch (48.5K, 3-v10-0002-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 98f8bcdcda4296936e9f090ce6f2e15c6b216167 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Thu, 10 Jul 2025 18:38:09 -0400 Subject: [PATCH v10 2/2] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 140 +++++++ src/backend/utils/adt/jsonpath.c | 117 +++++- src/backend/utils/adt/jsonpath_exec.c | 206 ++++++++++ src/backend/utils/adt/jsonpath_gram.y | 28 +- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 8 + src/test/regress/expected/jsonb_jsonpath.out | 399 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 137 +++++++ src/test/regress/sql/jsonb_jsonpath.sql | 109 +++++ src/test/regress/sql/jsonpath.sql | 31 ++ 10 files changed, 1178 insertions(+), 5 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c28aa71f570..23710921869 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18637,6 +18637,146 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start and end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..f3f2119a4a8 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -298,6 +298,8 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: { /* * First, reserve place for left/right arg's positions, then @@ -362,6 +364,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +462,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +839,60 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiStrReplace: + appendStringInfoString(buf, ".replace("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPart: + appendStringInfoString(buf, ".split_part("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrLower: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpper: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcap: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrim: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrim: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrim: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +968,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiStrReplace: + return "replace"; + case jpiStrLower: + return "lower"; + case jpiStrUpper: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +982,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrim: + return "ltrim"; + case jpiStrRtrim: + return "rtrim"; + case jpiStrBtrim: + return "btrim"; + case jpiStrInitcap: + return "initcap"; + case jpiStrSplitPart: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1094,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; case jpiString: case jpiKey: @@ -1041,6 +1122,8 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; @@ -1055,6 +1138,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1176,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1238,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiStrReplace || + v->type == jpiStrLower || + v->type == jpiStrUpper || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim || + v->type == jpiStrInitcap || + v->type == jpiStrSplitPart); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1179,7 +1276,9 @@ jspGetLeftArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.left); } @@ -1201,7 +1300,9 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.right); } @@ -1501,6 +1602,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrReplace: + case jpiStrLower: + case jpiStrUpper: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..28a04b24bfa 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrim: + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,193 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult +executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) +{ + Assert(jsp->type == jpiStrLower || + jsp->type == jpiStrUpper || + jsp->type == jpiStrReplace || + jsp->type == jpiStrLtrim || + jsp->type == jpiStrRtrim || + jsp->type == jpiStrBtrim || + jsp->type == jpiStrInitcap || + jsp->type == jpiStrSplitPart); + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string + * value. The first argument to internal + * functions */ + char *tmp = NULL; + char *resStr = NULL; + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + if (jsp->content.arg) + { + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim; + break; + case jpiStrRtrim: + func = rtrim; + break; + case jpiStrBtrim: + func = btrim; + break; + default:; + } + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim1; + break; + case jpiStrRtrim: + func = rtrim1; + break; + case jpiStrBtrim: + func = btrim1; + break; + default:; + } + resStr = TextDatumGetCString(DirectFunctionCall1Coll(func, + DEFAULT_COLLATION_OID, str)); + break; + } + + case jpiStrLower: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpper: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcap: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrReplace: + { + char *from_str, + *to_str; + int from_len, + to_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPart: + { + char *from_str; + Numeric n; + int from_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 0b16cec18c4..53bf779e152 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -86,6 +86,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACE_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -95,7 +97,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, str_elem opt_str_arg int_elem uint_elem opt_uint_arg -%type <elems> accessor_expr int_list opt_int_list +%type <elems> accessor_expr int_list opt_int_list str_int_args str_str_args %type <indexs> index_list @@ -278,6 +280,16 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACE_P '(' str_str_args ')' + { $$ = makeItemBinary(jpiStrReplace, linitial($4), lsecond($4)); } + | '.' STR_SPLIT_PART_P '(' str_int_args ')' + { $$ = makeItemBinary(jpiStrSplitPart, linitial($4), lsecond($4)); } + | '.' STR_LTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrLtrim, $4); } + | '.' STR_RTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrRtrim, $4); } + | '.' STR_BTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrBtrim, $4); } ; int_elem: @@ -317,6 +329,14 @@ opt_str_arg: | /* EMPTY */ { $$ = NULL; } ; +str_int_args: + str_elem ',' int_elem { $$ = list_make2($1, $3); } + ; + +str_str_args: + str_elem ',' str_elem { $$ = list_make2($1, $3); } + ; + key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +377,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +396,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLower; } + | STR_UPPER_P { $$ = jpiStrUpper; } + | STR_INITCAP_P { $$ = jpiStrInitcap; } ; %% diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..69c61f98500 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -413,8 +413,13 @@ static const JsonPathKeyword keywords[] = { {4, true, TRUE_P, "true"}, {4, false, TYPE_P, "type"}, {4, false, WITH_P, "with"}, + {5, false, STR_BTRIM_P, "btrim"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -425,13 +430,16 @@ static const JsonPathKeyword keywords[] = { {7, false, BOOLEAN_P, "boolean"}, {7, false, CEILING_P, "ceiling"}, {7, false, DECIMAL_P, "decimal"}, + {7, false, STR_INITCAP_P, "initcap"}, {7, false, INTEGER_P, "integer"}, + {7, false, STR_REPLACE_P, "replace"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..2d0c53f7b06 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiStrReplace, /* .replace() item method */ + jpiStrLower, /* .lower() item method */ + jpiStrUpper, /* .upper() item method */ + jpiStrLtrim, /* .ltrim() item method */ + jpiStrRtrim, /* .rtrim() item method */ + jpiStrBtrim, /* .btrim() item method */ + jpiStrInitcap, /* .initcap() item method */ + jpiStrSplitPart, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..cea539717f9 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,405 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); + jsonb_path_query +------------------ + "trim" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + jsonb_path_query +------------------ + "ghi" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..a13022feabd 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,143 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + +-- Parse errors +select '$.replace("hello")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace("hello")'::jsonpath; + ^ +select '$.replace()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace()'::jsonpath; + ^ +select '$.replace("hello","bye","extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.replace("hello","bye","extra")'::jsonpath; + ^ +select '$.split_part("~@~")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part("~@~")'::jsonpath; + ^ +select '$.split_part()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part()'::jsonpath; + ^ +select '$.split_part("~@~", "hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.split_part("~@~", "hi")'::jsonpath; + ^ +select '$.split_part("~@~", 2, "extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.split_part("~@~", 2, "extra")'::jsonpath; + ^ +select '$.lower("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.lower("hi")'::jsonpath; + ^ +select '$.upper("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.upper("hi")'::jsonpath; + ^ +select '$.initcap("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.initcap("hi")'::jsonpath; + ^ +select '$.ltrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.ltrim(42)'::jsonpath; + ^ +select '$.ltrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.ltrim("x", "y")'::jsonpath; + ^ +select '$.rtrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.rtrim(42)'::jsonpath; + ^ +select '$.rtrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.rtrim("x", "y")'::jsonpath; + ^ +select '$.trim(42)'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim(42)'::jsonpath; + ^ +select '$.trim("x", "y")'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim("x", "y")'::jsonpath; + ^ select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..8169719343e 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,115 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..7dd2e57e7a6 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,37 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + +-- Parse errors +select '$.replace("hello")'::jsonpath; +select '$.replace()'::jsonpath; +select '$.replace("hello","bye","extra")'::jsonpath; +select '$.split_part("~@~")'::jsonpath; +select '$.split_part()'::jsonpath; +select '$.split_part("~@~", "hi")'::jsonpath; +select '$.split_part("~@~", 2, "extra")'::jsonpath; +select '$.lower("hi")'::jsonpath; +select '$.upper("hi")'::jsonpath; +select '$.initcap("hi")'::jsonpath; +select '$.ltrim(42)'::jsonpath; +select '$.ltrim("x", "y")'::jsonpath; +select '$.rtrim(42)'::jsonpath; +select '$.rtrim("x", "y")'::jsonpath; +select '$.trim(42)'::jsonpath; +select '$.trim("x", "y")'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 5-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-07-10 23:23 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-07-10 23:23 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jul 10, 2025, at 18:40, David E. Wheeler <[email protected]> wrote: > Resolved in the attached, which now passes `make check-world` for me. > > Also available as a pull request[1]. Now with the `ISO C90 forbids mixed declarations and code` warning cleared up. Weird that there’s a failure on Bookworm with Meson [1] (pg_regress diffs [2]) but not Bookworm with Configure [3]. Collation issue, perhaps? Best, David [1]: https://cirrus-ci.com/task/5363472541679616 [2]: https://api.cirrus-ci.com/v1/artifact/task/5363472541679616/testrun/build-32/testrun/regress/regress... [3]: https://cirrus-ci.com/task/5926422495100928 Attachments: [application/octet-stream] v11-0001-Rename-jsonpath-method-arg-tokens.patch (3.9K, 2-v11-0001-Rename-jsonpath-method-arg-tokens.patch) download | inline diff: From b63773f20c615a79b84522cfcb1b68d4bdfdc7a9 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" <[email protected]> Date: Mon, 2 Jun 2025 17:14:10 -0400 Subject: [PATCH v11 1/2] Rename jsonpath method arg tokens Rename the `csv_` tokens to `int_`, because they represent signed or unsigned integers, as follows: * `csv_elem` => `int_elem` * `csv_list` => `int_list` * `opt_csv_list` => `opt_int_list` Rename the `datetime_precision` tokens to `uint_arg`, as they represent unsigned integers and will be useful for other methods in the future, as follows: * `datetime_precision` => `uint_elem` * `opt_datetime_precision` => `opt_uint_arg` Rename the `datetime_template` tokens to `str_arg`, as they represent strings and will be useful for other methods in the future, as follows: * `datetime_template` => `str_elem` * `opt_datetime_template` => `opt_str_arg` --- src/backend/utils/adt/jsonpath_gram.y | 42 +++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..0b16cec18c4 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -92,10 +92,10 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %type <value> scalar_value path_primary expr array_accessor any_path accessor_op key predicate delimited_predicate index_elem starts_with_initial expr_or_predicate - datetime_template opt_datetime_template csv_elem - datetime_precision opt_datetime_precision + str_elem opt_str_arg int_elem + uint_elem opt_uint_arg -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr int_list opt_int_list %type <indexs> index_list @@ -254,7 +254,7 @@ accessor_op: | '.' any_path { $$ = $2; } | '.' method '(' ')' { $$ = makeItemType($2); } | '?' '(' predicate ')' { $$ = makeItemUnary(jpiFilter, $3); } - | '.' DECIMAL_P '(' opt_csv_list ')' + | '.' DECIMAL_P '(' opt_int_list ')' { if (list_length($4) == 0) $$ = makeItemBinary(jpiDecimal, NULL, NULL); @@ -268,19 +268,19 @@ accessor_op: errmsg("invalid input syntax for type %s", "jsonpath"), errdetail(".decimal() can only have an optional precision[,scale]."))); } - | '.' DATETIME_P '(' opt_datetime_template ')' + | '.' DATETIME_P '(' opt_str_arg ')' { $$ = makeItemUnary(jpiDatetime, $4); } - | '.' TIME_P '(' opt_datetime_precision ')' + | '.' TIME_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTime, $4); } - | '.' TIME_TZ_P '(' opt_datetime_precision ')' + | '.' TIME_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimeTz, $4); } - | '.' TIMESTAMP_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestamp, $4); } - | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } ; -csv_elem: +int_elem: INT_P { $$ = makeItemNumeric(&$1); } | '+' INT_P %prec UMINUS @@ -289,31 +289,31 @@ csv_elem: { $$ = makeItemUnary(jpiMinus, makeItemNumeric(&$2)); } ; -csv_list: - csv_elem { $$ = list_make1($1); } - | csv_list ',' csv_elem { $$ = lappend($1, $3); } +int_list: + int_elem { $$ = list_make1($1); } + | int_list ',' int_elem { $$ = lappend($1, $3); } ; -opt_csv_list: - csv_list { $$ = $1; } +opt_int_list: + int_list { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_precision: +uint_elem: INT_P { $$ = makeItemNumeric(&$1); } ; -opt_datetime_precision: - datetime_precision { $$ = $1; } +opt_uint_arg: + uint_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_template: +str_elem: STRING_P { $$ = makeItemString(&$1); } ; -opt_datetime_template: - datetime_template { $$ = $1; } +opt_str_arg: + str_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -- 2.49.0 [application/octet-stream] v11-0002-Add-additional-jsonpath-string-methods.patch (48.5K, 3-v11-0002-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 0db127dd34b0955f9533977103b8a9f7cc78bbf0 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Thu, 10 Jul 2025 19:01:34 -0400 Subject: [PATCH v11 2/2] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 140 +++++++ src/backend/utils/adt/jsonpath.c | 117 +++++- src/backend/utils/adt/jsonpath_exec.c | 207 ++++++++++ src/backend/utils/adt/jsonpath_gram.y | 28 +- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 8 + src/test/regress/expected/jsonb_jsonpath.out | 399 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 137 +++++++ src/test/regress/sql/jsonb_jsonpath.sql | 109 +++++ src/test/regress/sql/jsonpath.sql | 31 ++ 10 files changed, 1179 insertions(+), 5 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c28aa71f570..23710921869 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18637,6 +18637,146 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start and end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..f3f2119a4a8 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -298,6 +298,8 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: { /* * First, reserve place for left/right arg's positions, then @@ -362,6 +364,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +462,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +839,60 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiStrReplace: + appendStringInfoString(buf, ".replace("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPart: + appendStringInfoString(buf, ".split_part("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrLower: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpper: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcap: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrim: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrim: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrim: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +968,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiStrReplace: + return "replace"; + case jpiStrLower: + return "lower"; + case jpiStrUpper: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +982,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrim: + return "ltrim"; + case jpiStrRtrim: + return "rtrim"; + case jpiStrBtrim: + return "btrim"; + case jpiStrInitcap: + return "initcap"; + case jpiStrSplitPart: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1094,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; case jpiString: case jpiKey: @@ -1041,6 +1122,8 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; @@ -1055,6 +1138,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1176,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1238,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiStrReplace || + v->type == jpiStrLower || + v->type == jpiStrUpper || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim || + v->type == jpiStrInitcap || + v->type == jpiStrSplitPart); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1179,7 +1276,9 @@ jspGetLeftArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.left); } @@ -1201,7 +1300,9 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.right); } @@ -1501,6 +1602,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrReplace: + case jpiStrLower: + case jpiStrUpper: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..a8161f2f7fa 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrim: + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,194 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult +executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) +{ + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string + * value. The first argument to internal + * functions */ + char *tmp = NULL; + char *resStr = NULL; + + Assert(jsp->type == jpiStrLower || + jsp->type == jpiStrUpper || + jsp->type == jpiStrReplace || + jsp->type == jpiStrLtrim || + jsp->type == jpiStrRtrim || + jsp->type == jpiStrBtrim || + jsp->type == jpiStrInitcap || + jsp->type == jpiStrSplitPart); + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + if (jsp->content.arg) + { + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim; + break; + case jpiStrRtrim: + func = rtrim; + break; + case jpiStrBtrim: + func = btrim; + break; + default:; + } + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim1; + break; + case jpiStrRtrim: + func = rtrim1; + break; + case jpiStrBtrim: + func = btrim1; + break; + default:; + } + resStr = TextDatumGetCString(DirectFunctionCall1Coll(func, + DEFAULT_COLLATION_OID, str)); + break; + } + + case jpiStrLower: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpper: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcap: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrReplace: + { + char *from_str, + *to_str; + int from_len, + to_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPart: + { + char *from_str; + Numeric n; + int from_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 0b16cec18c4..53bf779e152 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -86,6 +86,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACE_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -95,7 +97,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, str_elem opt_str_arg int_elem uint_elem opt_uint_arg -%type <elems> accessor_expr int_list opt_int_list +%type <elems> accessor_expr int_list opt_int_list str_int_args str_str_args %type <indexs> index_list @@ -278,6 +280,16 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACE_P '(' str_str_args ')' + { $$ = makeItemBinary(jpiStrReplace, linitial($4), lsecond($4)); } + | '.' STR_SPLIT_PART_P '(' str_int_args ')' + { $$ = makeItemBinary(jpiStrSplitPart, linitial($4), lsecond($4)); } + | '.' STR_LTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrLtrim, $4); } + | '.' STR_RTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrRtrim, $4); } + | '.' STR_BTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrBtrim, $4); } ; int_elem: @@ -317,6 +329,14 @@ opt_str_arg: | /* EMPTY */ { $$ = NULL; } ; +str_int_args: + str_elem ',' int_elem { $$ = list_make2($1, $3); } + ; + +str_str_args: + str_elem ',' str_elem { $$ = list_make2($1, $3); } + ; + key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +377,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +396,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLower; } + | STR_UPPER_P { $$ = jpiStrUpper; } + | STR_INITCAP_P { $$ = jpiStrInitcap; } ; %% diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..69c61f98500 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -413,8 +413,13 @@ static const JsonPathKeyword keywords[] = { {4, true, TRUE_P, "true"}, {4, false, TYPE_P, "type"}, {4, false, WITH_P, "with"}, + {5, false, STR_BTRIM_P, "btrim"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -425,13 +430,16 @@ static const JsonPathKeyword keywords[] = { {7, false, BOOLEAN_P, "boolean"}, {7, false, CEILING_P, "ceiling"}, {7, false, DECIMAL_P, "decimal"}, + {7, false, STR_INITCAP_P, "initcap"}, {7, false, INTEGER_P, "integer"}, + {7, false, STR_REPLACE_P, "replace"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..2d0c53f7b06 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiStrReplace, /* .replace() item method */ + jpiStrLower, /* .lower() item method */ + jpiStrUpper, /* .upper() item method */ + jpiStrLtrim, /* .ltrim() item method */ + jpiStrRtrim, /* .rtrim() item method */ + jpiStrBtrim, /* .btrim() item method */ + jpiStrInitcap, /* .initcap() item method */ + jpiStrSplitPart, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..cea539717f9 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,405 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); + jsonb_path_query +------------------ + "trim" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + jsonb_path_query +------------------ + "ghi" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..a13022feabd 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,143 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + +-- Parse errors +select '$.replace("hello")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace("hello")'::jsonpath; + ^ +select '$.replace()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace()'::jsonpath; + ^ +select '$.replace("hello","bye","extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.replace("hello","bye","extra")'::jsonpath; + ^ +select '$.split_part("~@~")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part("~@~")'::jsonpath; + ^ +select '$.split_part()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part()'::jsonpath; + ^ +select '$.split_part("~@~", "hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.split_part("~@~", "hi")'::jsonpath; + ^ +select '$.split_part("~@~", 2, "extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.split_part("~@~", 2, "extra")'::jsonpath; + ^ +select '$.lower("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.lower("hi")'::jsonpath; + ^ +select '$.upper("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.upper("hi")'::jsonpath; + ^ +select '$.initcap("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.initcap("hi")'::jsonpath; + ^ +select '$.ltrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.ltrim(42)'::jsonpath; + ^ +select '$.ltrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.ltrim("x", "y")'::jsonpath; + ^ +select '$.rtrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.rtrim(42)'::jsonpath; + ^ +select '$.rtrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.rtrim("x", "y")'::jsonpath; + ^ +select '$.trim(42)'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim(42)'::jsonpath; + ^ +select '$.trim("x", "y")'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim("x", "y")'::jsonpath; + ^ select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..8169719343e 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,115 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..7dd2e57e7a6 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,37 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + +-- Parse errors +select '$.replace("hello")'::jsonpath; +select '$.replace()'::jsonpath; +select '$.replace("hello","bye","extra")'::jsonpath; +select '$.split_part("~@~")'::jsonpath; +select '$.split_part()'::jsonpath; +select '$.split_part("~@~", "hi")'::jsonpath; +select '$.split_part("~@~", 2, "extra")'::jsonpath; +select '$.lower("hi")'::jsonpath; +select '$.upper("hi")'::jsonpath; +select '$.initcap("hi")'::jsonpath; +select '$.ltrim(42)'::jsonpath; +select '$.ltrim("x", "y")'::jsonpath; +select '$.rtrim(42)'::jsonpath; +select '$.rtrim("x", "y")'::jsonpath; +select '$.trim(42)'::jsonpath; +select '$.trim("x", "y")'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 5-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-07-11 18:48 David E. Wheeler <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: David E. Wheeler @ 2025-07-11 18:48 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jul 10, 2025, at 19:23, David E. Wheeler <[email protected]> wrote: > Now with the `ISO C90 forbids mixed declarations and code` warning cleared up. > > Weird that there’s a failure on Bookworm with Meson [1] (pg_regress diffs [2]) but not Bookworm with Configure [3]. Collation issue, perhaps? David Johnson noticed that this build is 32-bit. I looked at the split_path function and after trying a couple of things, realized that it was passing an int8 when the SQL function in Marlena.c passes an int4. This change got the test passing in my clone (indentation reduced): ```patch --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -2959,7 +2959,7 @@ executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, C_COLLATION_OID, CStringGetTextDatum(tmp), CStringGetTextDatum(from_str), - DirectFunctionCall1(numeric_int8, NumericGetDatum(n)))); + DirectFunctionCall1(numeric_int4, NumericGetDatum(n)))); break; } default: ``` v12 attached. Best, David Attachments: [application/octet-stream] v12-0001-Rename-jsonpath-method-arg-tokens.patch (3.9K, 2-v12-0001-Rename-jsonpath-method-arg-tokens.patch) download | inline diff: From b63773f20c615a79b84522cfcb1b68d4bdfdc7a9 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" <[email protected]> Date: Mon, 2 Jun 2025 17:14:10 -0400 Subject: [PATCH v12 1/2] Rename jsonpath method arg tokens Rename the `csv_` tokens to `int_`, because they represent signed or unsigned integers, as follows: * `csv_elem` => `int_elem` * `csv_list` => `int_list` * `opt_csv_list` => `opt_int_list` Rename the `datetime_precision` tokens to `uint_arg`, as they represent unsigned integers and will be useful for other methods in the future, as follows: * `datetime_precision` => `uint_elem` * `opt_datetime_precision` => `opt_uint_arg` Rename the `datetime_template` tokens to `str_arg`, as they represent strings and will be useful for other methods in the future, as follows: * `datetime_template` => `str_elem` * `opt_datetime_template` => `opt_str_arg` --- src/backend/utils/adt/jsonpath_gram.y | 42 +++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 499745a8fef..0b16cec18c4 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -92,10 +92,10 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %type <value> scalar_value path_primary expr array_accessor any_path accessor_op key predicate delimited_predicate index_elem starts_with_initial expr_or_predicate - datetime_template opt_datetime_template csv_elem - datetime_precision opt_datetime_precision + str_elem opt_str_arg int_elem + uint_elem opt_uint_arg -%type <elems> accessor_expr csv_list opt_csv_list +%type <elems> accessor_expr int_list opt_int_list %type <indexs> index_list @@ -254,7 +254,7 @@ accessor_op: | '.' any_path { $$ = $2; } | '.' method '(' ')' { $$ = makeItemType($2); } | '?' '(' predicate ')' { $$ = makeItemUnary(jpiFilter, $3); } - | '.' DECIMAL_P '(' opt_csv_list ')' + | '.' DECIMAL_P '(' opt_int_list ')' { if (list_length($4) == 0) $$ = makeItemBinary(jpiDecimal, NULL, NULL); @@ -268,19 +268,19 @@ accessor_op: errmsg("invalid input syntax for type %s", "jsonpath"), errdetail(".decimal() can only have an optional precision[,scale]."))); } - | '.' DATETIME_P '(' opt_datetime_template ')' + | '.' DATETIME_P '(' opt_str_arg ')' { $$ = makeItemUnary(jpiDatetime, $4); } - | '.' TIME_P '(' opt_datetime_precision ')' + | '.' TIME_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTime, $4); } - | '.' TIME_TZ_P '(' opt_datetime_precision ')' + | '.' TIME_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimeTz, $4); } - | '.' TIMESTAMP_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestamp, $4); } - | '.' TIMESTAMP_TZ_P '(' opt_datetime_precision ')' + | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } ; -csv_elem: +int_elem: INT_P { $$ = makeItemNumeric(&$1); } | '+' INT_P %prec UMINUS @@ -289,31 +289,31 @@ csv_elem: { $$ = makeItemUnary(jpiMinus, makeItemNumeric(&$2)); } ; -csv_list: - csv_elem { $$ = list_make1($1); } - | csv_list ',' csv_elem { $$ = lappend($1, $3); } +int_list: + int_elem { $$ = list_make1($1); } + | int_list ',' int_elem { $$ = lappend($1, $3); } ; -opt_csv_list: - csv_list { $$ = $1; } +opt_int_list: + int_list { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_precision: +uint_elem: INT_P { $$ = makeItemNumeric(&$1); } ; -opt_datetime_precision: - datetime_precision { $$ = $1; } +opt_uint_arg: + uint_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -datetime_template: +str_elem: STRING_P { $$ = makeItemString(&$1); } ; -opt_datetime_template: - datetime_template { $$ = $1; } +opt_str_arg: + str_elem { $$ = $1; } | /* EMPTY */ { $$ = NULL; } ; -- 2.49.0 [application/octet-stream] v12-0002-Add-additional-jsonpath-string-methods.patch (48.5K, 3-v12-0002-Add-additional-jsonpath-string-methods.patch) download | inline diff: From 694cf349f3bfa7e6e8c0d99f91cdc4a9a362d8e0 Mon Sep 17 00:00:00 2001 From: Florents Tselai <[email protected]> Date: Fri, 11 Jul 2025 14:04:28 -0400 Subject: [PATCH v12 2/2] Add additional jsonpath string methods Add the following jsonpath methods: * l/r/btrim() * lower(), upper() * initcap() * replace() * split_part() Each simply dispatches to the standard string processing functions. These depend on the locale, but since it's set at `initdb`, they can be considered immutable and therefore allowed in any jsonpath expression. --- doc/src/sgml/func.sgml | 140 +++++++ src/backend/utils/adt/jsonpath.c | 117 +++++- src/backend/utils/adt/jsonpath_exec.c | 207 ++++++++++ src/backend/utils/adt/jsonpath_gram.y | 28 +- src/backend/utils/adt/jsonpath_scan.l | 8 + src/include/utils/jsonpath.h | 8 + src/test/regress/expected/jsonb_jsonpath.out | 399 +++++++++++++++++++ src/test/regress/expected/jsonpath.out | 137 +++++++ src/test/regress/sql/jsonb_jsonpath.sql | 109 +++++ src/test/regress/sql/jsonpath.sql | 31 ++ 10 files changed, 1179 insertions(+), 5 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c28aa71f570..23710921869 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -18637,6 +18637,146 @@ ERROR: jsonpath member accessor can only be applied to an object <returnvalue>[{"id": 0, "key": "x", "value": "20"}, {"id": 0, "key": "y", "value": 32}]</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>lower()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all lower case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"TOM"', '$.lower()')</literal> + <returnvalue>"tom"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>upper()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String converted to all upper case according to the rules of the database's locale. + </para> + <para> + <literal>jsonb_path_query('"tom"', '$.upper()')</literal> + <returnvalue>"TOM"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>initcap()</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the first letter of each word converted to upper case + according to the rules of the database's locale. Words are sequences + of alphanumeric characters separated by non-alphanumeric characters. + </para> + <para> + <literal>jsonb_path_query('"hi THOMAS"', '$.initcap()')</literal> + <returnvalue>"Hi Thomas"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>replace(<replaceable>from</replaceable>, <replaceable>to</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with all occurrences of substring from replaced with substring to. + </para> + <para> + <literal>jsonb_path_query('"abcdefabcdef"', '$.replace("cd", "XX")')</literal> + <returnvalue>"abXXefabXXef"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>split_part(<replaceable>delimiter</replaceable>, <replaceable>n</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String split at occurrences of <replaceable>delimiter</replaceable> + and returns the <replaceable>n</replaceable>'th field (counting from + one) or, when <replaceable>n</replaceable> is negative, returns the + |<replaceable>n</replaceable>|'th-from-last field. + </para> + <para> + <literal>jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)')</literal> + <returnvalue>"def"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", 2)')</literal> + <returnvalue>"ghi"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>ltrim(<replaceable>characters</replaceable>)</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start of <replaceable>string</replaceable> + </para> + <para> + <literal> jsonb_path_query('" hello"', '$.ltrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"zzzytest"', '$.ltrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>rtrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('"hello "', '$.rtrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"testxxzx"', '$.rtrim("xyz")')</literal> + <returnvalue>"test"</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <replaceable>string</replaceable> <literal>.</literal> <literal>btrim([ <replaceable>characters</replaceable> ])</literal> + <returnvalue><replaceable>string</replaceable></returnvalue> + </para> + <para> + String with the longest string containing only spaces or the + characters in <replaceable>characters</replaceable> removed from the + start and end of <replaceable>string</replaceable> + </para> + <para> + <literal>jsonb_path_query('" hello "', '$.btrim()')</literal> + <returnvalue>"hello"</returnvalue> + </para> + <para> + <literal>jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")')</literal> + <returnvalue>"trim"</returnvalue> + </para></entry> + </row> + </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c index 762f7e8a09d..f3f2119a4a8 100644 --- a/src/backend/utils/adt/jsonpath.c +++ b/src/backend/utils/adt/jsonpath.c @@ -298,6 +298,8 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: { /* * First, reserve place for left/right arg's positions, then @@ -362,6 +364,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: { int32 arg = reserveSpaceForItemPointer(buf); @@ -457,6 +462,9 @@ flattenJsonPathParseItem(StringInfo buf, int *result, struct Node *escontext, case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; default: elog(ERROR, "unrecognized jsonpath item type: %d", item->type); @@ -831,6 +839,60 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey, } appendStringInfoChar(buf, ')'); break; + case jpiStrReplace: + appendStringInfoString(buf, ".replace("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrSplitPart: + appendStringInfoString(buf, ".split_part("); + jspGetLeftArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ','); + jspGetRightArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + appendStringInfoChar(buf, ')'); + break; + case jpiStrLower: + appendStringInfoString(buf, ".lower()"); + break; + case jpiStrUpper: + appendStringInfoString(buf, ".upper()"); + break; + case jpiStrInitcap: + appendStringInfoString(buf, ".initcap()"); + break; + case jpiStrLtrim: + appendStringInfoString(buf, ".ltrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrRtrim: + appendStringInfoString(buf, ".rtrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; + case jpiStrBtrim: + appendStringInfoString(buf, ".btrim("); + if (v->content.arg) + { + jspGetArg(v, &elem); + printJsonPathItem(buf, &elem, false, false); + } + appendStringInfoChar(buf, ')'); + break; default: elog(ERROR, "unrecognized jsonpath item type: %d", v->type); } @@ -906,6 +968,12 @@ jspOperationName(JsonPathItemType type) return "number"; case jpiStringFunc: return "string"; + case jpiStrReplace: + return "replace"; + case jpiStrLower: + return "lower"; + case jpiStrUpper: + return "upper"; case jpiTime: return "time"; case jpiTimeTz: @@ -914,6 +982,16 @@ jspOperationName(JsonPathItemType type) return "timestamp"; case jpiTimestampTz: return "timestamp_tz"; + case jpiStrLtrim: + return "ltrim"; + case jpiStrRtrim: + return "rtrim"; + case jpiStrBtrim: + return "btrim"; + case jpiStrInitcap: + return "initcap"; + case jpiStrSplitPart: + return "split_part"; default: elog(ERROR, "unrecognized jsonpath item type: %d", type); return NULL; @@ -1016,6 +1094,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrLower: + case jpiStrUpper: + case jpiStrInitcap: break; case jpiString: case jpiKey: @@ -1041,6 +1122,8 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiMod: case jpiStartsWith: case jpiDecimal: + case jpiStrReplace: + case jpiStrSplitPart: read_int32(v->content.args.left, base, pos); read_int32(v->content.args.right, base, pos); break; @@ -1055,6 +1138,9 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos) case jpiTimeTz: case jpiTimestamp: case jpiTimestampTz: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: read_int32(v->content.arg, base, pos); break; case jpiIndexArray: @@ -1090,7 +1176,10 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim); jspInitByBuffer(a, v->base, v->content.arg); } @@ -1149,10 +1238,18 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a) v->type == jpiInteger || v->type == jpiNumber || v->type == jpiStringFunc || + v->type == jpiStrReplace || + v->type == jpiStrLower || + v->type == jpiStrUpper || v->type == jpiTime || v->type == jpiTimeTz || v->type == jpiTimestamp || - v->type == jpiTimestampTz); + v->type == jpiTimestampTz || + v->type == jpiStrLtrim || + v->type == jpiStrRtrim || + v->type == jpiStrBtrim || + v->type == jpiStrInitcap || + v->type == jpiStrSplitPart); if (a) jspInitByBuffer(a, v->base, v->nextPos); @@ -1179,7 +1276,9 @@ jspGetLeftArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.left); } @@ -1201,7 +1300,9 @@ jspGetRightArg(JsonPathItem *v, JsonPathItem *a) v->type == jpiDiv || v->type == jpiMod || v->type == jpiStartsWith || - v->type == jpiDecimal); + v->type == jpiDecimal || + v->type == jpiStrReplace || + v->type == jpiStrSplitPart); jspInitByBuffer(a, v->base, v->content.args.right); } @@ -1501,6 +1602,14 @@ jspIsMutableWalker(JsonPathItem *jpi, struct JsonPathMutableContext *cxt) case jpiInteger: case jpiNumber: case jpiStringFunc: + case jpiStrReplace: + case jpiStrLower: + case jpiStrUpper: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: status = jpdsNonDateTime; break; diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c index dbab24737ef..98418bee9b7 100644 --- a/src/backend/utils/adt/jsonpath_exec.c +++ b/src/backend/utils/adt/jsonpath_exec.c @@ -301,6 +301,8 @@ static JsonPathExecResult executeNumericItemMethod(JsonPathExecContext *cxt, JsonValueList *found); static JsonPathExecResult executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); +static JsonPathExecResult executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found); static JsonPathExecResult executeKeyValueMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, JsonbValue *jb, JsonValueList *found); static JsonPathExecResult appendBoolResult(JsonPathExecContext *cxt, @@ -1659,6 +1661,23 @@ executeItemOptUnwrapTarget(JsonPathExecContext *cxt, JsonPathItem *jsp, } break; + case jpiStrLtrim: + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + { + if (unwrap && JsonbType(jb) == jbvArray) + return executeItemUnwrapTargetArray(cxt, jsp, jb, found, false); + + return executeStringInternalMethod(cxt, jsp, jb, found); + } + break; + + default: elog(ERROR, "unrecognized jsonpath item type: %d", jsp->type); } @@ -2792,6 +2811,194 @@ executeDateTimeMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); } +/* + * Implementation of .upper(), lower() et. al. methods, + * that forward their actual implementation to internal functions. + */ +static JsonPathExecResult +executeStringInternalMethod(JsonPathExecContext *cxt, JsonPathItem *jsp, + JsonbValue *jb, JsonValueList *found) +{ + JsonbValue jbvbuf; + bool hasNext; + JsonPathExecResult res = jperNotFound; + JsonPathItem elem; + Datum str; /* Datum representation for the current string + * value. The first argument to internal + * functions */ + char *tmp = NULL; + char *resStr = NULL; + + Assert(jsp->type == jpiStrLower || + jsp->type == jpiStrUpper || + jsp->type == jpiStrReplace || + jsp->type == jpiStrLtrim || + jsp->type == jpiStrRtrim || + jsp->type == jpiStrBtrim || + jsp->type == jpiStrInitcap || + jsp->type == jpiStrSplitPart); + + if (!(jb = getScalar(jb, jbvString))) + RETURN_ERROR(ereport(ERROR, + (errcode(ERRCODE_INVALID_ARGUMENT_FOR_SQL_JSON_DATETIME_FUNCTION), + errmsg("jsonpath item method .%s() can only be applied to a string", + jspOperationName(jsp->type))))); + + tmp = pnstrdup(jb->val.string.val, jb->val.string.len); + str = CStringGetTextDatum(tmp); + + /* Internal string functions that accept no arguments */ + switch (jsp->type) + { + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + { + char *characters_str; + int characters_len; + PGFunction func = NULL; + + if (jsp->content.arg) + { + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim; + break; + case jpiStrRtrim: + func = rtrim; + break; + case jpiStrBtrim: + func = btrim; + break; + default:; + } + jspGetArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .%s() argument", jspOperationName(jsp->type)); + + characters_str = jspGetString(&elem, &characters_len); + resStr = TextDatumGetCString(DirectFunctionCall2Coll(func, + DEFAULT_COLLATION_OID, str, + CStringGetTextDatum(characters_str))); + break; + } + + switch (jsp->type) + { + case jpiStrLtrim: + func = ltrim1; + break; + case jpiStrRtrim: + func = rtrim1; + break; + case jpiStrBtrim: + func = btrim1; + break; + default:; + } + resStr = TextDatumGetCString(DirectFunctionCall1Coll(func, + DEFAULT_COLLATION_OID, str)); + break; + } + + case jpiStrLower: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(lower, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrUpper: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(upper, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrInitcap: + resStr = TextDatumGetCString(DirectFunctionCall1Coll(initcap, DEFAULT_COLLATION_OID, str)); + break; + case jpiStrReplace: + { + char *from_str, + *to_str; + int from_len, + to_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() from"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .replace() to"); + + to_str = jspGetString(&elem, &to_len); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(replace_text, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + CStringGetTextDatum(to_str))); + break; + } + case jpiStrSplitPart: + { + char *from_str; + Numeric n; + int from_len; + + jspGetLeftArg(jsp, &elem); + if (elem.type != jpiString) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + from_str = jspGetString(&elem, &from_len); + + jspGetRightArg(jsp, &elem); + if (elem.type != jpiNumeric) + elog(ERROR, "invalid jsonpath item type for .split_part()"); + + n = jspGetNumeric(&elem); + + resStr = TextDatumGetCString(DirectFunctionCall3Coll(split_part, + C_COLLATION_OID, + CStringGetTextDatum(tmp), + CStringGetTextDatum(from_str), + DirectFunctionCall1(numeric_int4, NumericGetDatum(n)))); + break; + } + default: + elog(ERROR, "unsupported jsonpath item type: %d", jsp->type); + } + + if (resStr) + res = jperOk; + + hasNext = jspGetNext(jsp, &elem); + + if (!hasNext && !found) + return res; + + jb = hasNext ? &jbvbuf : palloc(sizeof(*jb)); + + /* Create the appropriate jb value to return */ + switch (jsp->type) + { + /* Cases for functions that return text */ + case jpiStrLower: + case jpiStrUpper: + case jpiStrReplace: + case jpiStrLtrim: + case jpiStrRtrim: + case jpiStrBtrim: + case jpiStrInitcap: + case jpiStrSplitPart: + jb->type = jbvString; + jb->val.string.val = resStr; + jb->val.string.len = strlen(jb->val.string.val); + default: + ; + /* cant' happen */ + } + + return executeNextItem(cxt, jsp, &elem, jb, found, hasNext); +} + /* * Implementation of .keyvalue() method. * diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y index 0b16cec18c4..53bf779e152 100644 --- a/src/backend/utils/adt/jsonpath_gram.y +++ b/src/backend/utils/adt/jsonpath_gram.y @@ -86,6 +86,8 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, %token <str> DATETIME_P %token <str> BIGINT_P BOOLEAN_P DATE_P DECIMAL_P INTEGER_P NUMBER_P %token <str> STRINGFUNC_P TIME_P TIME_TZ_P TIMESTAMP_P TIMESTAMP_TZ_P +%token <str> STR_REPLACE_P STR_LOWER_P STR_UPPER_P STR_LTRIM_P STR_RTRIM_P STR_BTRIM_P + STR_INITCAP_P STR_SPLIT_PART_P %type <result> result @@ -95,7 +97,7 @@ static bool makeItemLikeRegex(JsonPathParseItem *expr, str_elem opt_str_arg int_elem uint_elem opt_uint_arg -%type <elems> accessor_expr int_list opt_int_list +%type <elems> accessor_expr int_list opt_int_list str_int_args str_str_args %type <indexs> index_list @@ -278,6 +280,16 @@ accessor_op: { $$ = makeItemUnary(jpiTimestamp, $4); } | '.' TIMESTAMP_TZ_P '(' opt_uint_arg ')' { $$ = makeItemUnary(jpiTimestampTz, $4); } + | '.' STR_REPLACE_P '(' str_str_args ')' + { $$ = makeItemBinary(jpiStrReplace, linitial($4), lsecond($4)); } + | '.' STR_SPLIT_PART_P '(' str_int_args ')' + { $$ = makeItemBinary(jpiStrSplitPart, linitial($4), lsecond($4)); } + | '.' STR_LTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrLtrim, $4); } + | '.' STR_RTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrRtrim, $4); } + | '.' STR_BTRIM_P '(' opt_str_arg ')' + { $$ = makeItemUnary(jpiStrBtrim, $4); } ; int_elem: @@ -317,6 +329,14 @@ opt_str_arg: | /* EMPTY */ { $$ = NULL; } ; +str_int_args: + str_elem ',' int_elem { $$ = list_make2($1, $3); } + ; + +str_str_args: + str_elem ',' str_elem { $$ = list_make2($1, $3); } + ; + key: key_name { $$ = makeItemKey(&$1); } ; @@ -357,6 +377,9 @@ key_name: | TIME_TZ_P | TIMESTAMP_P | TIMESTAMP_TZ_P + | STR_LTRIM_P + | STR_RTRIM_P + | STR_BTRIM_P ; method: @@ -373,6 +396,9 @@ method: | INTEGER_P { $$ = jpiInteger; } | NUMBER_P { $$ = jpiNumber; } | STRINGFUNC_P { $$ = jpiStringFunc; } + | STR_LOWER_P { $$ = jpiStrLower; } + | STR_UPPER_P { $$ = jpiStrUpper; } + | STR_INITCAP_P { $$ = jpiStrInitcap; } ; %% diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/backend/utils/adt/jsonpath_scan.l index c7aab83eeb4..69c61f98500 100644 --- a/src/backend/utils/adt/jsonpath_scan.l +++ b/src/backend/utils/adt/jsonpath_scan.l @@ -413,8 +413,13 @@ static const JsonPathKeyword keywords[] = { {4, true, TRUE_P, "true"}, {4, false, TYPE_P, "type"}, {4, false, WITH_P, "with"}, + {5, false, STR_BTRIM_P, "btrim"}, {5, true, FALSE_P, "false"}, {5, false, FLOOR_P, "floor"}, + {5, false, STR_LOWER_P, "lower"}, + {5, false, STR_LTRIM_P, "ltrim"}, + {5, false, STR_RTRIM_P, "rtrim"}, + {5, false, STR_UPPER_P, "upper"}, {6, false, BIGINT_P, "bigint"}, {6, false, DOUBLE_P, "double"}, {6, false, EXISTS_P, "exists"}, @@ -425,13 +430,16 @@ static const JsonPathKeyword keywords[] = { {7, false, BOOLEAN_P, "boolean"}, {7, false, CEILING_P, "ceiling"}, {7, false, DECIMAL_P, "decimal"}, + {7, false, STR_INITCAP_P, "initcap"}, {7, false, INTEGER_P, "integer"}, + {7, false, STR_REPLACE_P, "replace"}, {7, false, TIME_TZ_P, "time_tz"}, {7, false, UNKNOWN_P, "unknown"}, {8, false, DATETIME_P, "datetime"}, {8, false, KEYVALUE_P, "keyvalue"}, {9, false, TIMESTAMP_P, "timestamp"}, {10, false, LIKE_REGEX_P, "like_regex"}, + {10,false, STR_SPLIT_PART_P, "split_part"}, {12, false, TIMESTAMP_TZ_P, "timestamp_tz"}, }; diff --git a/src/include/utils/jsonpath.h b/src/include/utils/jsonpath.h index 23a76d233e9..2d0c53f7b06 100644 --- a/src/include/utils/jsonpath.h +++ b/src/include/utils/jsonpath.h @@ -115,6 +115,14 @@ typedef enum JsonPathItemType jpiTimeTz, /* .time_tz() item method */ jpiTimestamp, /* .timestamp() item method */ jpiTimestampTz, /* .timestamp_tz() item method */ + jpiStrReplace, /* .replace() item method */ + jpiStrLower, /* .lower() item method */ + jpiStrUpper, /* .upper() item method */ + jpiStrLtrim, /* .ltrim() item method */ + jpiStrRtrim, /* .rtrim() item method */ + jpiStrBtrim, /* .btrim() item method */ + jpiStrInitcap, /* .initcap() item method */ + jpiStrSplitPart, /* .split_part() item method */ } JsonPathItemType; /* XQuery regex mode flags for LIKE_REGEX predicate */ diff --git a/src/test/regress/expected/jsonb_jsonpath.out b/src/test/regress/expected/jsonb_jsonpath.out index 4bcd4e91a29..cea539717f9 100644 --- a/src/test/regress/expected/jsonb_jsonpath.out +++ b/src/test/regress/expected/jsonb_jsonpath.out @@ -2723,6 +2723,405 @@ select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); (1 row) rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim(" ")'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('" hello "', '$.ltrim()'); + jsonb_path_query +------------------ + "hello " +(1 row) + +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('null', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('null', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.ltrim()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('{}', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.ltrim()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('"1.23"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('true', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('1234', '$.ltrim().type()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query('[2, true]', '$.ltrim()'); +ERROR: jsonpath item method .ltrim() can only be applied to a string +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); + jsonb_path_query_array +-------------------------- + ["maybe ", "yes", "no"] +(1 row) + +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); + jsonb_path_query +------------------ + "test" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +select jsonb_path_query('" hello "', '$.rtrim()'); + jsonb_path_query +------------------ + " hello" +(1 row) + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); + jsonb_path_query +------------------ + "trim" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +select jsonb_path_query('" hello "', '$.btrim()'); + jsonb_path_query +------------------ + "hello" +(1 row) + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('null', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.lower()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('{}', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[]', 'strict $.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.lower()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('"1.23"', '$.lower()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.lower()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('true', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('1234', '$.lower().type()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query('[2, true]', '$.lower()'); +ERROR: jsonpath item method .lower() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); + jsonb_path_query_array +------------------------ + ["maybe", "yes", "no"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('null', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.upper()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('{}', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[]', 'strict $.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.upper()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('"1.23"', '$.upper()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.upper()'); + jsonb_path_query +------------------ + "1.23AAA" +(1 row) + +select jsonb_path_query('1234', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('true', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('1234', '$.upper().type()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query('[2, true]', '$.upper()'); +ERROR: jsonpath item method .upper() can only be applied to a string +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); + jsonb_path_query_array +------------------------ + ["MAYBE", "YES", "NO"] +(1 row) + +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + jsonb_path_query_array +-------------------------------- + ["string", "string", "string"] +(1 row) + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('null', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', '$.initcap()'); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('[]', 'strict $.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('{}', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.initcap()', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('"1.23"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23" +(1 row) + +select jsonb_path_query('"1.23aaa"', '$.initcap()'); + jsonb_path_query +------------------ + "1.23aaa" +(1 row) + +select jsonb_path_query('1234', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('true', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('1234', '$.initcap().type()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('[2, true]', '$.initcap()'); +ERROR: jsonpath item method .initcap() can only be applied to a string +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + jsonb_path_query +------------------ + "Maybe Yes" + "Probably No" +(2 rows) + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); + jsonb_path_query +------------------ + "bye" + "y" + "z" +(3 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); + jsonb_path_query +------------------ +(0 rows) + +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +ERROR: jsonpath item method .replace() can only be applied to a string +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); + jsonb_path_query +------------------ + "def" +(1 row) + +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + jsonb_path_query +------------------ + "ghi" +(1 row) + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); + jsonb_path_query +------------------ + "bye world" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); + jsonb_path_query +------------------ + "BYE WORLD" +(1 row) + +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + jsonb_path_query +------------------ + true +(1 row) + -- Test .time() select jsonb_path_query('null', '$.time()'); ERROR: jsonpath item method .time() can only be applied to a string diff --git a/src/test/regress/expected/jsonpath.out b/src/test/regress/expected/jsonpath.out index fd9bd755f52..a13022feabd 100644 --- a/src/test/regress/expected/jsonpath.out +++ b/src/test/regress/expected/jsonpath.out @@ -435,6 +435,143 @@ select '$.string()'::jsonpath; $.string() (1 row) +select '$.replace("hello","bye")'::jsonpath; + jsonpath +-------------------------- + $.replace("hello","bye") +(1 row) + +select '$.lower()'::jsonpath; + jsonpath +----------- + $.lower() +(1 row) + +select '$.upper()'::jsonpath; + jsonpath +----------- + $.upper() +(1 row) + +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; + jsonpath +-------------------------------------------------- + $.lower().upper().lower().replace("hello","bye") +(1 row) + +select '$.ltrim()'::jsonpath; + jsonpath +----------- + $.ltrim() +(1 row) + +select '$.ltrim("xyz")'::jsonpath; + jsonpath +---------------- + $.ltrim("xyz") +(1 row) + +select '$.rtrim()'::jsonpath; + jsonpath +----------- + $.rtrim() +(1 row) + +select '$.rtrim("xyz")'::jsonpath; + jsonpath +---------------- + $.rtrim("xyz") +(1 row) + +select '$.btrim()'::jsonpath; + jsonpath +----------- + $.btrim() +(1 row) + +select '$.btrim("xyz")'::jsonpath; + jsonpath +---------------- + $.btrim("xyz") +(1 row) + +select '$.initcap()'::jsonpath; + jsonpath +------------- + $.initcap() +(1 row) + +select '$.split_part("~@~", 2)'::jsonpath; + jsonpath +----------------------- + $.split_part("~@~",2) +(1 row) + +-- Parse errors +select '$.replace("hello")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace("hello")'::jsonpath; + ^ +select '$.replace()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.replace()'::jsonpath; + ^ +select '$.replace("hello","bye","extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.replace("hello","bye","extra")'::jsonpath; + ^ +select '$.split_part("~@~")'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part("~@~")'::jsonpath; + ^ +select '$.split_part()'::jsonpath; +ERROR: syntax error at or near ")" of jsonpath input +LINE 1: select '$.split_part()'::jsonpath; + ^ +select '$.split_part("~@~", "hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.split_part("~@~", "hi")'::jsonpath; + ^ +select '$.split_part("~@~", 2, "extra")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.split_part("~@~", 2, "extra")'::jsonpath; + ^ +select '$.lower("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.lower("hi")'::jsonpath; + ^ +select '$.upper("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.upper("hi")'::jsonpath; + ^ +select '$.initcap("hi")'::jsonpath; +ERROR: syntax error at or near """ of jsonpath input +LINE 1: select '$.initcap("hi")'::jsonpath; + ^ +select '$.ltrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.ltrim(42)'::jsonpath; + ^ +select '$.ltrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.ltrim("x", "y")'::jsonpath; + ^ +select '$.rtrim(42)'::jsonpath; +ERROR: syntax error at or near "42" of jsonpath input +LINE 1: select '$.rtrim(42)'::jsonpath; + ^ +select '$.rtrim("x", "y")'::jsonpath; +ERROR: syntax error at or near "," of jsonpath input +LINE 1: select '$.rtrim("x", "y")'::jsonpath; + ^ +select '$.trim(42)'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim(42)'::jsonpath; + ^ +select '$.trim("x", "y")'::jsonpath; +ERROR: syntax error at or near "(" of jsonpath input +LINE 1: select '$.trim("x", "y")'::jsonpath; + ^ select '$.time()'::jsonpath; jsonpath ---------- diff --git a/src/test/regress/sql/jsonb_jsonpath.sql b/src/test/regress/sql/jsonb_jsonpath.sql index 3e8929a5269..8169719343e 100644 --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -623,6 +623,115 @@ select jsonb_path_query('"2023-08-15 12:34:56 +5:30"', '$.timestamp_tz().string( select jsonb_path_query('"2023-08-15 12:34:56"', '$.timestamp().string()'); rollback; +-- test .ltrim() +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim(" ")'); +select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); +select jsonb_path_query('null', '$.ltrim()'); +select jsonb_path_query('null', '$.ltrim()', silent => true); +select jsonb_path_query('[]', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()'); +select jsonb_path_query('{}', '$.ltrim()'); +select jsonb_path_query('[]', 'strict $.ltrim()', silent => true); +select jsonb_path_query('{}', '$.ltrim()', silent => true); +select jsonb_path_query('1.23', '$.ltrim()'); +select jsonb_path_query('"1.23"', '$.ltrim()'); +select jsonb_path_query('"1.23aaa"', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim()'); +select jsonb_path_query('true', '$.ltrim()'); +select jsonb_path_query('1234', '$.ltrim().type()'); +select jsonb_path_query('[2, true]', '$.ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim()'); +select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().type()'); + +-- test .rtrim() +select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); +select jsonb_path_query('" hello "', '$.rtrim()'); +select jsonb_path_query('" hello "', '$.rtrim()'); + +-- test .btrim() +select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); +select jsonb_path_query('" hello "', '$.btrim()'); +select jsonb_path_query('" hello "', '$.btrim()'); + +-- test .lower() +select jsonb_path_query('null', '$.lower()'); +select jsonb_path_query('null', '$.lower()', silent => true); +select jsonb_path_query('[]', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()'); +select jsonb_path_query('{}', '$.lower()'); +select jsonb_path_query('[]', 'strict $.lower()', silent => true); +select jsonb_path_query('{}', '$.lower()', silent => true); +select jsonb_path_query('1.23', '$.lower()'); +select jsonb_path_query('"1.23"', '$.lower()'); +select jsonb_path_query('"1.23aaa"', '$.lower()'); +select jsonb_path_query('1234', '$.lower()'); +select jsonb_path_query('true', '$.lower()'); +select jsonb_path_query('1234', '$.lower().type()'); +select jsonb_path_query('[2, true]', '$.lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].lower().type()'); + +-- test .upper() +select jsonb_path_query('null', '$.upper()'); +select jsonb_path_query('null', '$.upper()', silent => true); +select jsonb_path_query('[]', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()'); +select jsonb_path_query('{}', '$.upper()'); +select jsonb_path_query('[]', 'strict $.upper()', silent => true); +select jsonb_path_query('{}', '$.upper()', silent => true); +select jsonb_path_query('1.23', '$.upper()'); +select jsonb_path_query('"1.23"', '$.upper()'); +select jsonb_path_query('"1.23aaa"', '$.upper()'); +select jsonb_path_query('1234', '$.upper()'); +select jsonb_path_query('true', '$.upper()'); +select jsonb_path_query('1234', '$.upper().type()'); +select jsonb_path_query('[2, true]', '$.upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper()'); +select jsonb_path_query_array('["maybe", "yes", "no"]', '$[*].upper().type()'); + +-- test .initcap() +select jsonb_path_query('null', '$.initcap()'); +select jsonb_path_query('null', '$.initcap()', silent => true); +select jsonb_path_query('[]', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()'); +select jsonb_path_query('{}', '$.initcap()'); +select jsonb_path_query('[]', 'strict $.initcap()', silent => true); +select jsonb_path_query('{}', '$.initcap()', silent => true); +select jsonb_path_query('1.23', '$.initcap()'); +select jsonb_path_query('"1.23"', '$.initcap()'); +select jsonb_path_query('"1.23aaa"', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap()'); +select jsonb_path_query('true', '$.initcap()'); +select jsonb_path_query('1234', '$.initcap().type()'); +select jsonb_path_query('[2, true]', '$.initcap()'); +select jsonb_path_query('["maybe yes", "probably no"]', '$.initcap()'); + +-- Test .replace() +select jsonb_path_query('null', '$.replace("x", "bye")'); +select jsonb_path_query('null', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('["x", "y", "z"]', '$.replace("x", "bye")'); +select jsonb_path_query('{}', '$.replace("x", "bye")'); +select jsonb_path_query('[]', 'strict $.replace("x", "bye")', silent => true); +select jsonb_path_query('{}', '$.replace("x", "bye")', silent => true); +select jsonb_path_query('1.23', '$.replace("x", "bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye")'); +select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with "bye"'); + +-- Test .split_part() +select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); + +-- Test string methods play nicely together +select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye")'); +select jsonb_path_query('"hElLo WorlD"', '$.upper().lower().upper().replace("HELLO", "BYE")'); +select jsonb_path_query('"hElLo WorlD"', '$.lower().upper().lower().replace("hello","bye") starts with "bye"'); +select jsonb_path_query('" hElLo WorlD "', '$.btrim().lower().upper().lower().replace("hello","bye") starts with "bye"'); + -- Test .time() select jsonb_path_query('null', '$.time()'); select jsonb_path_query('true', '$.time()'); diff --git a/src/test/regress/sql/jsonpath.sql b/src/test/regress/sql/jsonpath.sql index 61a5270d4e8..7dd2e57e7a6 100644 --- a/src/test/regress/sql/jsonpath.sql +++ b/src/test/regress/sql/jsonpath.sql @@ -78,6 +78,37 @@ select '$.boolean()'::jsonpath; select '$.date()'::jsonpath; select '$.decimal(4,2)'::jsonpath; select '$.string()'::jsonpath; +select '$.replace("hello","bye")'::jsonpath; +select '$.lower()'::jsonpath; +select '$.upper()'::jsonpath; +select '$.lower().upper().lower().replace("hello","bye")'::jsonpath; +select '$.ltrim()'::jsonpath; +select '$.ltrim("xyz")'::jsonpath; +select '$.rtrim()'::jsonpath; +select '$.rtrim("xyz")'::jsonpath; +select '$.btrim()'::jsonpath; +select '$.btrim("xyz")'::jsonpath; +select '$.initcap()'::jsonpath; +select '$.split_part("~@~", 2)'::jsonpath; + +-- Parse errors +select '$.replace("hello")'::jsonpath; +select '$.replace()'::jsonpath; +select '$.replace("hello","bye","extra")'::jsonpath; +select '$.split_part("~@~")'::jsonpath; +select '$.split_part()'::jsonpath; +select '$.split_part("~@~", "hi")'::jsonpath; +select '$.split_part("~@~", 2, "extra")'::jsonpath; +select '$.lower("hi")'::jsonpath; +select '$.upper("hi")'::jsonpath; +select '$.initcap("hi")'::jsonpath; +select '$.ltrim(42)'::jsonpath; +select '$.ltrim("x", "y")'::jsonpath; +select '$.rtrim(42)'::jsonpath; +select '$.rtrim("x", "y")'::jsonpath; +select '$.trim(42)'::jsonpath; +select '$.trim("x", "y")'::jsonpath; + select '$.time()'::jsonpath; select '$.time(6)'::jsonpath; select '$.time_tz()'::jsonpath; -- 2.49.0 [application/pgp-signature] signature.asc (833B, 5-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-07-12 04:07 Florents Tselai <[email protected]> parent: David E. Wheeler <[email protected]> 0 siblings, 1 reply; 39+ messages in thread From: Florents Tselai @ 2025-07-12 04:07 UTC (permalink / raw) To: David E. Wheeler <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Fri, Jul 11, 2025 at 9:48 PM David E. Wheeler <[email protected]> wrote: > On Jul 10, 2025, at 19:23, David E. Wheeler <[email protected]> wrote: > > > Now with the `ISO C90 forbids mixed declarations and code` warning > cleared up. > > > > Weird that there’s a failure on Bookworm with Meson [1] (pg_regress > diffs [2]) but not Bookworm with Configure [3]. Collation issue, perhaps? > > David Johnson noticed that this build is 32-bit. I looked at the > split_path function and after trying a couple of things, realized that it > was passing an int8 when the SQL function in Marlena.c passes an int4. This > change got the test passing in my clone (indentation reduced): Occasionally I've noticed myself some inconsistencies wrt to compiler warnings between meson & make . But cirrus seems generally happy now https://cirrus-ci.com/build/4964687915253760 To recap so far; - I like your changes and renames on the parser/lexer; it indeed looks much cleaner now and will help with future improvements. - I also like the addition of executeStringInternalMethod ; it'll help us add more stuff in the future (reminder that for the original patch I implemented the methods I'd like more, but string operations are quite more). - AFAICT no test cases / results have changed with your versions; is this correct ? ^ permalink raw reply [nested|flat] 39+ messages in thread
* Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part @ 2025-07-12 18:24 David E. Wheeler <[email protected]> parent: Florents Tselai <[email protected]> 0 siblings, 0 replies; 39+ messages in thread From: David E. Wheeler @ 2025-07-12 18:24 UTC (permalink / raw) To: Florents Tselai <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Robert Haas <[email protected]>; Alexander Korotkov <[email protected]>; pgsql-hackers <[email protected]>; Andrew Dunstan <[email protected]> On Jul 12, 2025, at 00:07, Florents Tselai <[email protected]> wrote: > To recap so far; > > - I like your changes and renames on the parser/lexer; it indeed looks much cleaner now and will help with future improvements. Thanks! > - I also like the addition of executeStringInternalMethod ; it'll help us add more stuff in the future (reminder that for the original patch I implemented the methods I'd like more, but string operations are quite more). Agreed. > - AFAICT no test cases / results have changed with your versions; is this correct ? I made some minor changes, notably to test alternate trim values and a negative position passed to split_part(): ```patch --- a/src/test/regress/sql/jsonb_jsonpath.sql +++ b/src/test/regress/sql/jsonb_jsonpath.sql @@ -627,7 +627,7 @@ rollback; select jsonb_path_query('" hello "', '$.ltrim(" ")'); select jsonb_path_query('" hello "', '$.ltrim(" ")'); select jsonb_path_query('" hello "', '$.ltrim()'); -select jsonb_path_query('" hello "', '$.ltrim()'); +select jsonb_path_query('"zzzytest"', '$.ltrim("xyz")'); select jsonb_path_query('null', '$.ltrim()'); select jsonb_path_query('null', '$.ltrim()', silent => true); select jsonb_path_query('[]', '$.ltrim()'); @@ -647,13 +647,13 @@ select jsonb_path_query_array('[" maybe ", " yes", " no"]', '$[*].ltrim().ty -- test .rtrim() select jsonb_path_query('" hello "', '$.rtrim(" ")'); -select jsonb_path_query('" hello "', '$.rtrim(" ")'); +select jsonb_path_query('"testxxzx"', '$.rtrim("xyz")'); select jsonb_path_query('" hello "', '$.rtrim()'); select jsonb_path_query('" hello "', '$.rtrim()'); -- test .btrim() select jsonb_path_query('" hello "', '$.btrim(" ")'); -select jsonb_path_query('" hello "', '$.btrim(" ")'); +select jsonb_path_query('"xyxtrimyyx"', '$.btrim("xyz")'); select jsonb_path_query('" hello "', '$.btrim()'); select jsonb_path_query('" hello "', '$.btrim()'); @@ -723,6 +723,7 @@ select jsonb_path_query('"hello world"', '$.replace("hello","bye") starts with " -- Test .split_part() select jsonb_path_query('"abc~@~def~@~ghi"', '$.split_part("~@~", 2)'); +select jsonb_path_query('"abc,def,ghi,jkl"', '$.split_part(",", -2)'); -- Test string methods play nicely together select jsonb_path_query('"hello world"', '$.replace("hello","bye").upper()'); ``` Best, David Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 39+ messages in thread
end of thread, other threads:[~2025-07-12 18:24 UTC | newest] Thread overview: 39+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-09-26 10:55 Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part Alexander Korotkov <[email protected]> 2024-09-26 12:59 ` Florents Tselai <[email protected]> 2024-09-27 09:45 ` David E. Wheeler <[email protected]> 2024-09-27 10:28 ` Florents Tselai <[email protected]> 2025-03-05 19:29 ` Florents Tselai <[email protected]> 2025-05-09 19:50 ` Robert Haas <[email protected]> 2025-05-13 18:07 ` David E. Wheeler <[email protected]> 2025-05-13 20:24 ` Florents Tselai <[email protected]> 2025-05-14 03:00 ` David E. Wheeler <[email protected]> 2025-05-14 15:33 ` Florents Tselai <[email protected]> 2025-05-21 18:06 ` Robert Haas <[email protected]> 2025-05-21 18:13 ` David E. Wheeler <[email protected]> 2025-05-21 18:31 ` Tom Lane <[email protected]> 2025-05-22 14:05 ` Robert Haas <[email protected]> 2025-05-22 16:38 ` Florents Tselai <[email protected]> 2025-05-22 18:08 ` David E. Wheeler <[email protected]> 2025-05-22 20:56 ` Peter Eisentraut <[email protected]> 2025-05-23 13:56 ` Robert Haas <[email protected]> 2025-05-23 17:06 ` Florents Tselai <[email protected]> 2025-05-23 17:52 ` Tom Lane <[email protected]> 2025-05-24 16:08 ` David E. Wheeler <[email protected]> 2025-05-24 16:51 ` Florents Tselai <[email protected]> 2025-05-24 21:39 ` David E. Wheeler <[email protected]> 2025-05-24 21:46 ` David E. Wheeler <[email protected]> 2025-05-24 21:55 ` David E. Wheeler <[email protected]> 2025-05-24 22:01 ` David E. Wheeler <[email protected]> 2025-05-25 04:16 ` Florents Tselai <[email protected]> 2025-05-26 22:00 ` David E. Wheeler <[email protected]> 2025-06-03 19:02 ` David E. Wheeler <[email protected]> 2025-06-03 19:10 ` David E. Wheeler <[email protected]> 2025-06-04 15:27 ` David E. Wheeler <[email protected]> 2025-06-14 15:08 ` David E. Wheeler <[email protected]> 2025-07-10 17:41 ` Florents Tselai <[email protected]> 2025-07-10 18:13 ` David E. Wheeler <[email protected]> 2025-07-10 22:40 ` David E. Wheeler <[email protected]> 2025-07-10 23:23 ` David E. Wheeler <[email protected]> 2025-07-11 18:48 ` David E. Wheeler <[email protected]> 2025-07-12 04:07 ` Florents Tselai <[email protected]> 2025-07-12 18:24 ` David E. Wheeler <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox