public inbox for [email protected]help / color / mirror / Atom feed
[PATCH] Add XMLText function (SQL/XML X038) 21+ messages / 7 participants [nested] [flat]
* [PATCH] Add XMLText function (SQL/XML X038) @ 2023-03-25 11:49 Jim Jones <[email protected]> 0 siblings, 2 replies; 21+ messages in thread From: Jim Jones @ 2023-03-25 11:49 UTC (permalink / raw) To: PostgreSQL Hackers <[email protected]> Hi, This small patch proposes the implementation of the standard SQL/XML function XMLText (X038). It basically converts a text parameter into an xml text node. It uses the libxml2 function xmlEncodeSpecialChars[1] to escape possible predefined entities. This patch also contains documentation and regression tests. Any thoughts? Best, Jim 1 - https://gnome.pages.gitlab.gnome.org/libxml2/devhelp/libxml2-entities.html#xmlEncodeSpecialChars Attachments: [text/x-patch] v1-0001-Add-XMLText-function-SQL-XML-X038.patch (8.7K, ../../[email protected]/2-v1-0001-Add-XMLText-function-SQL-XML-X038.patch) download | inline diff: From 84d6e026724d7e3869f28b09c686e2fc4da67873 Mon Sep 17 00:00:00 2001 From: Jim Jones <[email protected]> Date: Sat, 25 Mar 2023 12:24:49 +0100 Subject: [PATCH v1] Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predifined entites (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. This patch includes also documentation and regression tests. --- doc/src/sgml/func.sgml | 30 +++++++++++++++++++++++ src/backend/catalog/sql_features.txt | 2 +- src/backend/utils/adt/xml.c | 30 +++++++++++++++++++++++ src/include/catalog/pg_proc.dat | 3 +++ src/test/regress/expected/xml.out | 36 ++++++++++++++++++++++++++++ src/test/regress/expected/xml_1.out | 23 ++++++++++++++++++ src/test/regress/expected/xml_2.out | 36 ++++++++++++++++++++++++++++ src/test/regress/sql/xml.sql | 7 ++++++ 8 files changed, 166 insertions(+), 1 deletion(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index a3a13b895f..45195cf05b 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -14001,6 +14001,36 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple documents for processing in client applications. </para> + <sect3 id="functions-producing-xml-xmltext"> + <title><literal>xmltext</literal></title> + + <indexterm> + <primary>xmltext</primary> + </indexterm> + +<synopsis> +<function>xmltext</function> ( <type>text</type> ) <returnvalue>xml</returnvalue> +</synopsis> + + <para> + The function <function>xmltext</function> returns an XML value with a single + text node containing the input argument as its content. Predefined entities + like ampersand (<literal><![CDATA[&]]></literal>), left and right angle brackets + (<literal><![CDATA[< >]]></literal>), and quotation marks (<literal><![CDATA[""]]></literal>) + are escaped. + </para> + + <para> + Example: +<screen><![CDATA[ +SELECT xmltext('< foo & bar >'); + xmltext +------------------------- + < foo & bar > +]]></screen> + </para> + </sect3> + <sect3 id="functions-producing-xml-xmlcomment"> <title><literal>xmlcomment</literal></title> diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt index bb4c135a7f..82e6ab8258 100644 --- a/src/backend/catalog/sql_features.txt +++ b/src/backend/catalog/sql_features.txt @@ -595,7 +595,7 @@ X034 XMLAgg YES X035 XMLAgg: ORDER BY option YES X036 XMLComment YES X037 XMLPI YES -X038 XMLText NO +X038 XMLText YES X040 Basic table mapping YES X041 Basic table mapping: null absent YES X042 Basic table mapping: null as nil YES diff --git a/src/backend/utils/adt/xml.c b/src/backend/utils/adt/xml.c index 15adbd6a01..6621d69219 100644 --- a/src/backend/utils/adt/xml.c +++ b/src/backend/utils/adt/xml.c @@ -47,6 +47,7 @@ #ifdef USE_LIBXML #include <libxml/chvalid.h> +#include <libxml/entities.h> #include <libxml/parser.h> #include <libxml/parserInternals.h> #include <libxml/tree.h> @@ -505,6 +506,10 @@ xmlcomment(PG_FUNCTION_ARGS) appendStringInfoText(&buf, arg); appendStringInfoString(&buf, "-->"); + + + + PG_RETURN_XML_P(stringinfo_to_xmltype(&buf)); #else NO_XML_SUPPORT(); @@ -5006,3 +5011,28 @@ XmlTableDestroyOpaque(TableFuncScanState *state) NO_XML_SUPPORT(); #endif /* not USE_LIBXML */ } + +Datum +xmltext(PG_FUNCTION_ARGS) +{ +#ifdef USE_LIBXML + + text *arg = PG_GETARG_TEXT_PP(0); + text *result; + xmlChar *xmlbuf = NULL; + + xmlbuf = xmlEncodeSpecialChars(NULL,xml_text2xmlChar(arg)); + + Assert(xmlbuf); + + result = cstring_to_text_with_len((const char *) xmlbuf, xmlStrlen(xmlbuf)); + + xmlFree(xmlbuf); + + PG_RETURN_XML_P(result); + +#else + NO_XML_SUPPORT(); + return 0; +#endif +} \ No newline at end of file diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 5cf87aeb2c..293ae66adc 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8772,6 +8772,9 @@ { oid => '2922', descr => 'serialize an XML value to a character string', proname => 'text', prorettype => 'text', proargtypes => 'xml', prosrc => 'xmltotext' }, +{ oid => '3813', descr => 'generate XML text node', + proname => 'xmltext', proisstrict => 't', prorettype => 'xml', + proargtypes => 'text', prosrc => 'xmltext' }, { oid => '2923', descr => 'map table contents to XML', proname => 'table_to_xml', procost => '100', provolatile => 's', diff --git a/src/test/regress/expected/xml.out b/src/test/regress/expected/xml.out index 398345ca67..93cbe4edf6 100644 --- a/src/test/regress/expected/xml.out +++ b/src/test/regress/expected/xml.out @@ -1785,3 +1785,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo ´`$_-+?=*^%!|/\()[]{}°'); + xmltext +---------------------------- + foo ´`$_-+?=*^%!|/\()[]{}° +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/expected/xml_1.out b/src/test/regress/expected/xml_1.out index 63b779470f..904c665b54 100644 --- a/src/test/regress/expected/xml_1.out +++ b/src/test/regress/expected/xml_1.out @@ -1402,3 +1402,26 @@ DETAIL: This functionality requires the server to be built with libxml support. SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); ERROR: unsupported XML feature DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(' '); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo ´`$_-+?=*^%!|/\()[]{}°'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo & <"bar">'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); +ERROR: unsupported XML feature +LINE 1: SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j':... + ^ +DETAIL: This functionality requires the server to be built with libxml support. diff --git a/src/test/regress/expected/xml_2.out b/src/test/regress/expected/xml_2.out index 43c2558352..0af16dcca4 100644 --- a/src/test/regress/expected/xml_2.out +++ b/src/test/regress/expected/xml_2.out @@ -1765,3 +1765,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo ´`$_-+?=*^%!|/\()[]{}°'); + xmltext +---------------------------- + foo ´`$_-+?=*^%!|/\()[]{}° +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/sql/xml.sql b/src/test/regress/sql/xml.sql index a591eea2e5..c189ba6531 100644 --- a/src/test/regress/sql/xml.sql +++ b/src/test/regress/sql/xml.sql @@ -660,3 +660,10 @@ SELECT * FROM XMLTABLE('*' PASSING '<e>pre<!--c1--><?pi arg?><![CDATA[&ent1]]><n \x SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); + +SELECT xmltext(NULL); +SELECT xmltext(''); +SELECT xmltext(' '); +SELECT xmltext('foo ´`$_-+?=*^%!|/\()[]{}°'); +SELECT xmltext('foo & <"bar">'); +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); \ No newline at end of file -- 2.25.1 ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-03-25 11:53 Pavel Stehule <[email protected]> parent: Jim Jones <[email protected]> 1 sibling, 1 reply; 21+ messages in thread From: Pavel Stehule @ 2023-03-25 11:53 UTC (permalink / raw) To: Jim Jones <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> so 25. 3. 2023 v 12:49 odesílatel Jim Jones <[email protected]> napsal: > Hi, > > This small patch proposes the implementation of the standard SQL/XML > function XMLText (X038). It basically converts a text parameter into an > xml text node. It uses the libxml2 function xmlEncodeSpecialChars[1] to > escape possible predefined entities. > > This patch also contains documentation and regression tests. > > Any thoughts? > +1 Pavel > Best, Jim > > 1 - > > https://gnome.pages.gitlab.gnome.org/libxml2/devhelp/libxml2-entities.html#xmlEncodeSpecialChars > ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-03-25 12:25 Jim Jones <[email protected]> parent: Pavel Stehule <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Jim Jones @ 2023-03-25 12:25 UTC (permalink / raw) To: Pavel Stehule <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On 25.03.23 12:53, Pavel Stehule wrote: > > so 25. 3. 2023 v 12:49 odesílatel Jim Jones > <[email protected]> napsal: > > Hi, > > This small patch proposes the implementation of the standard SQL/XML > function XMLText (X038). It basically converts a text parameter > into an > xml text node. It uses the libxml2 function > xmlEncodeSpecialChars[1] to > escape possible predefined entities. > > This patch also contains documentation and regression tests. > > Any thoughts? > > > +1 > > Pavel Thanks! I just realized that I forgot to add a few examples to my last message :D postgres=# SELECT xmltext('foo ´/[({bar?})]\`'); xmltext -------------------- foo ´/[({bar?})]\` (1 row) postgres=# SELECT xmltext('foo & <bar>'); xmltext ----------------------- foo & <bar> (1 row) ^ permalink raw reply [nested|flat] 21+ messages in thread
* [PATCH v3 2/7] Row pattern recognition patch (parse/analysis). @ 2023-07-26 10:49 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 21+ messages in thread From: Tatsuo Ishii @ 2023-07-26 10:49 UTC (permalink / raw) --- src/backend/parser/parse_agg.c | 7 ++ src/backend/parser/parse_clause.c | 190 +++++++++++++++++++++++++++++- src/backend/parser/parse_expr.c | 4 + src/backend/parser/parse_func.c | 3 + 4 files changed, 203 insertions(+), 1 deletion(-) diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c index 85cd47b7ae..aa7a1cee80 100644 --- a/src/backend/parser/parse_agg.c +++ b/src/backend/parser/parse_agg.c @@ -564,6 +564,10 @@ check_agglevels_and_constraints(ParseState *pstate, Node *expr) errkind = true; break; + case EXPR_KIND_RPR_DEFINE: + errkind = true; + break; + /* * There is intentionally no default: case here, so that the * compiler will warn if we add a new ParseExprKind without @@ -953,6 +957,9 @@ transformWindowFuncCall(ParseState *pstate, WindowFunc *wfunc, case EXPR_KIND_CYCLE_MARK: errkind = true; break; + case EXPR_KIND_RPR_DEFINE: + errkind = true; + break; /* * There is intentionally no default: case here, so that the diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c index 334b9b42bd..ea2decc579 100644 --- a/src/backend/parser/parse_clause.c +++ b/src/backend/parser/parse_clause.c @@ -100,7 +100,10 @@ static WindowClause *findWindowClause(List *wclist, const char *name); static Node *transformFrameOffset(ParseState *pstate, int frameOptions, Oid rangeopfamily, Oid rangeopcintype, Oid *inRangeFunc, Node *clause); - +static void transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef); +static List *transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef); +static void transformPatternClause(ParseState *pstate, WindowClause *wc, WindowDef *windef); +static List *transformMeasureClause(ParseState *pstate, WindowClause *wc, WindowDef *windef); /* * transformFromClause - @@ -2950,6 +2953,10 @@ transformWindowDefinitions(ParseState *pstate, rangeopfamily, rangeopcintype, &wc->endInRangeFunc, windef->endOffset); + + /* Process Row Pattern Recognition related clauses */ + transformRPR(pstate, wc, windef); + wc->runCondition = NIL; wc->winref = winref; @@ -3815,3 +3822,184 @@ transformFrameOffset(ParseState *pstate, int frameOptions, return node; } + +/* + * transformRPR + * Process Row Pattern Recognition related clauses + */ +static void +transformRPR(ParseState *pstate, WindowClause *wc, WindowDef *windef) +{ + /* Check Frame option. Frame must start at current row */ + + /* + * Window definition exists? + */ + if (windef == NULL) + return; + + /* + * Row Pattern Common Syntax clause exists? + */ + if (windef->rpCommonSyntax == NULL) + return; + + if ((wc->frameOptions & FRAMEOPTION_START_CURRENT_ROW) == 0) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("FRAME must start at current row when row patttern recognition is used"))); + + /* Transform AFTER MACH SKIP TO clause */ + wc->rpSkipTo = windef->rpCommonSyntax->rpSkipTo; + + /* Transform SEEK or INITIAL clause */ + wc->initial = windef->rpCommonSyntax->initial; + + /* Transform DEFINE clause into list of TargetEntry's */ + wc->defineClause = transformDefineClause(pstate, wc, windef); + + /* Check PATTERN clause and copy to patternClause */ + transformPatternClause(pstate, wc, windef); + + /* Transform MEASURE clause */ + transformMeasureClause(pstate, wc, windef); +} + +/* + * transformDefineClause Process DEFINE clause and transform ResTarget into + * list of TargetEntry. + * + * XXX we only support column reference in row pattern definition search + * condition, e.g. "price". <row pattern definition variable name>.<column + * reference> is not supported, e.g. "A.price". + */ +static List * +transformDefineClause(ParseState *pstate, WindowClause *wc, WindowDef *windef) +{ + ListCell *lc; + ResTarget *restarget, *r; + List *restargets; + + + /* + * If Row Definition Common Syntax exists, DEFINE clause must exist. + * (the raw parser should have already checked it.) + */ + Assert(windef->rpCommonSyntax->rpDefs != NULL); + + /* + * Check for duplicate row pattern definition variables. The standard + * requires that no two row pattern definition variable names shall be + * equivalent. + */ + restargets = NIL; + foreach(lc, windef->rpCommonSyntax->rpDefs) + { + char *name; + ListCell *l; + + restarget = (ResTarget *)lfirst(lc); + name = restarget->name; + + /* + * Make sure that row pattern definition search condition is a boolean + * expression. + */ + transformWhereClause(pstate, restarget->val, + EXPR_KIND_RPR_DEFINE, "DEFINE"); + + foreach(l, restargets) + { + char *n; + + r = (ResTarget *) lfirst(l); + n = r->name; + + if (!strcmp(n, name)) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("row pattern definition variable name \"%s\" appears more than once in DEFINE clause", + name), + parser_errposition(pstate, exprLocation((Node *)r)))); + } + restargets = lappend(restargets, restarget); + } + list_free(restargets); + + return transformTargetList(pstate, windef->rpCommonSyntax->rpDefs, + EXPR_KIND_RPR_DEFINE); +} + +/* + * transformPatternClause + * Process PATTERN clause and return PATTERN clause in the raw parse tree + */ +static void +transformPatternClause(ParseState *pstate, WindowClause *wc, WindowDef *windef) +{ + ListCell *lc, *l; + + /* + * Row Pattern Common Syntax clause exists? + */ + if (windef->rpCommonSyntax == NULL) + return; + + /* + * Primary row pattern variable names in PATTERN clause must appear in + * DEFINE clause as row pattern definition variable names. + */ + wc->patternVariable = NIL; + wc->patternRegexp = NIL; + foreach(lc, windef->rpCommonSyntax->rpPatterns) + { + A_Expr *a; + char *name; + char *regexp; + bool found = false; + + if (!IsA(lfirst(lc), A_Expr)) + ereport(ERROR, + errmsg("node type is not A_Expr")); + + a = (A_Expr *)lfirst(lc); + name = strVal(a->lexpr); + + foreach(l, windef->rpCommonSyntax->rpDefs) + { + ResTarget *restarget = (ResTarget *)lfirst(l); + + if (!strcmp(restarget->name, name)) + { + found = true; + break; + } + } + + if (!found) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("primary row pattern variable name \"%s\" does not appear in DEFINE clause", + name), + parser_errposition(pstate, exprLocation((Node *)a)))); + wc->patternVariable = lappend(wc->patternVariable, makeString(pstrdup(name))); + regexp = strVal(lfirst(list_head(a->name))); + wc->patternRegexp = lappend(wc->patternRegexp, makeString(pstrdup(regexp))); + } +} + +/* + * transformMeasureClause + * Process MEASURE clause + */ +static List * +transformMeasureClause(ParseState *pstate, WindowClause *wc, WindowDef *windef) +{ + if (windef->rowPatternMeasures == NIL) + return NIL; + + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("%s","MEASURE clause is not supported yet"), + parser_errposition(pstate, exprLocation((Node *)windef->rowPatternMeasures)))); +} diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index fed8e4d089..8921b7ae01 100644 --- a/src/backend/parser/parse_expr.c +++ b/src/backend/parser/parse_expr.c @@ -557,6 +557,7 @@ transformColumnRef(ParseState *pstate, ColumnRef *cref) case EXPR_KIND_COPY_WHERE: case EXPR_KIND_GENERATED_COLUMN: case EXPR_KIND_CYCLE_MARK: + case EXPR_KIND_RPR_DEFINE: /* okay */ break; @@ -1770,6 +1771,7 @@ transformSubLink(ParseState *pstate, SubLink *sublink) case EXPR_KIND_VALUES: case EXPR_KIND_VALUES_SINGLE: case EXPR_KIND_CYCLE_MARK: + case EXPR_KIND_RPR_DEFINE: /* okay */ break; case EXPR_KIND_CHECK_CONSTRAINT: @@ -3149,6 +3151,8 @@ ParseExprKindName(ParseExprKind exprKind) return "GENERATED AS"; case EXPR_KIND_CYCLE_MARK: return "CYCLE"; + case EXPR_KIND_RPR_DEFINE: + return "DEFINE"; /* * There is intentionally no default: case here, so that the diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c index b3f0b6a137..2ff3699538 100644 --- a/src/backend/parser/parse_func.c +++ b/src/backend/parser/parse_func.c @@ -2656,6 +2656,9 @@ check_srf_call_placement(ParseState *pstate, Node *last_srf, int location) case EXPR_KIND_CYCLE_MARK: errkind = true; break; + case EXPR_KIND_RPR_DEFINE: + errkind = true; + break; /* * There is intentionally no default: case here, so that the -- 2.25.1 ----Next_Part(Wed_Jul_26_21_21_34_2023_317)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v3-0003-Row-pattern-recognition-patch-planner.patch" ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 08:27 Jim Jones <[email protected]> parent: Jim Jones <[email protected]> 0 siblings, 0 replies; 21+ messages in thread From: Jim Jones @ 2023-08-25 08:27 UTC (permalink / raw) To: [email protected] On 25.03.23 13:25, I wrote: > I just realized that I forgot to add a few examples to my last message :D > > postgres=# SELECT xmltext('foo ´/[({bar?})]\`'); > xmltext > -------------------- > foo ´/[({bar?})]\` > (1 row) > > postgres=# SELECT xmltext('foo & <bar>'); > xmltext > ----------------------- > foo & <bar> > (1 row) > It seems that an encoding issue appears in the regression tests on Debian + Meson, 32 bit. ´ > ´ ° > ° v2 attached updates the regression tests to fix it. Jim Attachments: [text/x-patch] v2-0001-Add-XMLText-function-SQL-XML-X038.patch (8.9K, ../../[email protected]/3-v2-0001-Add-XMLText-function-SQL-XML-X038.patch) download | inline diff: From 348e8952de0939c34e40283c7860aa44b66182f5 Mon Sep 17 00:00:00 2001 From: Jim Jones <[email protected]> Date: Fri, 25 Aug 2023 10:14:24 +0200 Subject: [PATCH v2] Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predifined entites (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. This patch includes also documentation and regression tests. --- .gitignore | 5 ++++ doc/src/sgml/func.sgml | 30 +++++++++++++++++++++++ src/backend/catalog/sql_features.txt | 2 +- src/backend/utils/adt/xml.c | 30 +++++++++++++++++++++++ src/include/catalog/pg_proc.dat | 3 +++ src/test/regress/expected/xml.out | 36 ++++++++++++++++++++++++++++ src/test/regress/expected/xml_1.out | 23 ++++++++++++++++++ src/test/regress/expected/xml_2.out | 36 ++++++++++++++++++++++++++++ src/test/regress/sql/xml.sql | 7 ++++++ 9 files changed, 171 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 4e911395fe..e724ef42a5 100644 --- a/.gitignore +++ b/.gitignore @@ -43,3 +43,8 @@ lib*.pc /Release/ /tmp_install/ /portlock/ + + + +.vscode/ +redeploy-testdb.sh diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7a0d4b9134..2f01a2c25d 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -14058,6 +14058,36 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple documents for processing in client applications. </para> + <sect3 id="functions-producing-xml-xmltext"> + <title><literal>xmltext</literal></title> + + <indexterm> + <primary>xmltext</primary> + </indexterm> + +<synopsis> +<function>xmltext</function> ( <type>text</type> ) <returnvalue>xml</returnvalue> +</synopsis> + + <para> + The function <function>xmltext</function> returns an XML value with a single + text node containing the input argument as its content. Predefined entities + like ampersand (<literal><![CDATA[&]]></literal>), left and right angle brackets + (<literal><![CDATA[< >]]></literal>), and quotation marks (<literal><![CDATA[""]]></literal>) + are escaped. + </para> + + <para> + Example: +<screen><![CDATA[ +SELECT xmltext('< foo & bar >'); + xmltext +------------------------- + < foo & bar > +]]></screen> + </para> + </sect3> + <sect3 id="functions-producing-xml-xmlcomment"> <title><literal>xmlcomment</literal></title> diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt index b33065d7bf..680d541673 100644 --- a/src/backend/catalog/sql_features.txt +++ b/src/backend/catalog/sql_features.txt @@ -633,7 +633,7 @@ X034 XMLAgg YES X035 XMLAgg: ORDER BY option YES X036 XMLComment YES X037 XMLPI YES -X038 XMLText NO +X038 XMLText YES X040 Basic table mapping YES X041 Basic table mapping: null absent YES X042 Basic table mapping: null as nil YES diff --git a/src/backend/utils/adt/xml.c b/src/backend/utils/adt/xml.c index 866d0d649a..592b804e36 100644 --- a/src/backend/utils/adt/xml.c +++ b/src/backend/utils/adt/xml.c @@ -47,6 +47,7 @@ #ifdef USE_LIBXML #include <libxml/chvalid.h> +#include <libxml/entities.h> #include <libxml/parser.h> #include <libxml/parserInternals.h> #include <libxml/tree.h> @@ -505,6 +506,10 @@ xmlcomment(PG_FUNCTION_ARGS) appendStringInfoText(&buf, arg); appendStringInfoString(&buf, "-->"); + + + + PG_RETURN_XML_P(stringinfo_to_xmltype(&buf)); #else NO_XML_SUPPORT(); @@ -5006,3 +5011,28 @@ XmlTableDestroyOpaque(TableFuncScanState *state) NO_XML_SUPPORT(); #endif /* not USE_LIBXML */ } + +Datum +xmltext(PG_FUNCTION_ARGS) +{ +#ifdef USE_LIBXML + + text *arg = PG_GETARG_TEXT_PP(0); + text *result; + xmlChar *xmlbuf = NULL; + + xmlbuf = xmlEncodeSpecialChars(NULL,xml_text2xmlChar(arg)); + + Assert(xmlbuf); + + result = cstring_to_text_with_len((const char *) xmlbuf, xmlStrlen(xmlbuf)); + + xmlFree(xmlbuf); + + PG_RETURN_XML_P(result); + +#else + NO_XML_SUPPORT(); + return 0; +#endif +} \ No newline at end of file diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9805bc6118..ff00c6365d 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8789,6 +8789,9 @@ { oid => '2922', descr => 'serialize an XML value to a character string', proname => 'text', prorettype => 'text', proargtypes => 'xml', prosrc => 'xmltotext' }, +{ oid => '3813', descr => 'generate XML text node', + proname => 'xmltext', proisstrict => 't', prorettype => 'xml', + proargtypes => 'text', prosrc => 'xmltext' }, { oid => '2923', descr => 'map table contents to XML', proname => 'table_to_xml', procost => '100', provolatile => 's', diff --git a/src/test/regress/expected/xml.out b/src/test/regress/expected/xml.out index 398345ca67..13e4296bf8 100644 --- a/src/test/regress/expected/xml.out +++ b/src/test/regress/expected/xml.out @@ -1785,3 +1785,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/expected/xml_1.out b/src/test/regress/expected/xml_1.out index 63b779470f..eb9c6f2ed4 100644 --- a/src/test/regress/expected/xml_1.out +++ b/src/test/regress/expected/xml_1.out @@ -1402,3 +1402,26 @@ DETAIL: This functionality requires the server to be built with libxml support. SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); ERROR: unsupported XML feature DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(' '); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo & <"bar">'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); +ERROR: unsupported XML feature +LINE 1: SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j':... + ^ +DETAIL: This functionality requires the server to be built with libxml support. diff --git a/src/test/regress/expected/xml_2.out b/src/test/regress/expected/xml_2.out index 43c2558352..c8ed8e0cfa 100644 --- a/src/test/regress/expected/xml_2.out +++ b/src/test/regress/expected/xml_2.out @@ -1765,3 +1765,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/sql/xml.sql b/src/test/regress/sql/xml.sql index a591eea2e5..bd4a4e7acd 100644 --- a/src/test/regress/sql/xml.sql +++ b/src/test/regress/sql/xml.sql @@ -660,3 +660,10 @@ SELECT * FROM XMLTABLE('*' PASSING '<e>pre<!--c1--><?pi arg?><![CDATA[&ent1]]><n \x SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); + +SELECT xmltext(NULL); +SELECT xmltext(''); +SELECT xmltext(' '); +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +SELECT xmltext('foo & <"bar">'); +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); \ No newline at end of file -- 2.34.1 ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 10:05 Vik Fearing <[email protected]> parent: Jim Jones <[email protected]> 1 sibling, 1 reply; 21+ messages in thread From: Vik Fearing @ 2023-08-25 10:05 UTC (permalink / raw) To: Jim Jones <[email protected]>; PostgreSQL Hackers <[email protected]> On 3/25/23 12:49, Jim Jones wrote: > Hi, > > This small patch proposes the implementation of the standard SQL/XML > function XMLText (X038). It basically converts a text parameter into an > xml text node. It uses the libxml2 function xmlEncodeSpecialChars[1] to > escape possible predefined entities. > > This patch also contains documentation and regression tests. > > Any thoughts? I am replying to this email, but my comments are based on the v2 patch. Thank you for working on this, and I think this is a valuable addition. However, I have two issues with it. 1) There seems to be several spurious blank lines added that I do not think are warranted. 2) This patch does nothing to address the <XML returning clause> so we can't claim to implement X038 without a disclaimer. Upon further review, the same is true of XMLCOMMENT() so maybe that is okay for this patch, and a more comprehensive patch for our xml features is necessary. -- Vik Fearing ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 12:42 Jim Jones <[email protected]> parent: Vik Fearing <[email protected]> 0 siblings, 2 replies; 21+ messages in thread From: Jim Jones @ 2023-08-25 12:42 UTC (permalink / raw) To: Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]> Hi Vik Thanks for reviewing my patch! On 25.08.23 12:05, Vik Fearing wrote: > I am replying to this email, but my comments are based on the v2 patch. > > Thank you for working on this, and I think this is a valuable > addition. However, I have two issues with it. > > 1) There seems to be several spurious blank lines added that I do not > think are warranted. I tried to copy the aesthetics of other functions, but it seems I failed :) I removed a few blank lines. I hope it's fine now. Is there any tool like pgindent to take care of it automatically? > > 2) This patch does nothing to address the <XML returning clause> so we > can't claim to implement X038 without a disclaimer. Upon further > review, the same is true of XMLCOMMENT() so maybe that is okay for > this patch, and a more comprehensive patch for our xml features is > necessary. If we decide to not address this point here, I can take a look at it and work in a separated patch. v3 attached. Thanks Jim Attachments: [text/x-patch] v3-0001-Add-XMLText-function-SQL-XML-X038.patch (8.6K, ../../[email protected]/2-v3-0001-Add-XMLText-function-SQL-XML-X038.patch) download | inline diff: From 5a5302c8c8a16bf7cf26ade70a40f9e016d23bbf Mon Sep 17 00:00:00 2001 From: Jim Jones <[email protected]> Date: Fri, 25 Aug 2023 14:05:39 +0200 Subject: [PATCH v3] Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predifined entites (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. This patch includes also documentation and regression tests. --- doc/src/sgml/func.sgml | 30 +++++++++++++++++++++++ src/backend/catalog/sql_features.txt | 2 +- src/backend/utils/adt/xml.c | 26 ++++++++++++++++++++ src/include/catalog/pg_proc.dat | 3 +++ src/test/regress/expected/xml.out | 36 ++++++++++++++++++++++++++++ src/test/regress/expected/xml_1.out | 23 ++++++++++++++++++ src/test/regress/expected/xml_2.out | 36 ++++++++++++++++++++++++++++ src/test/regress/sql/xml.sql | 7 ++++++ 8 files changed, 162 insertions(+), 1 deletion(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7a0d4b9134..2f01a2c25d 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -14058,6 +14058,36 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple documents for processing in client applications. </para> + <sect3 id="functions-producing-xml-xmltext"> + <title><literal>xmltext</literal></title> + + <indexterm> + <primary>xmltext</primary> + </indexterm> + +<synopsis> +<function>xmltext</function> ( <type>text</type> ) <returnvalue>xml</returnvalue> +</synopsis> + + <para> + The function <function>xmltext</function> returns an XML value with a single + text node containing the input argument as its content. Predefined entities + like ampersand (<literal><![CDATA[&]]></literal>), left and right angle brackets + (<literal><![CDATA[< >]]></literal>), and quotation marks (<literal><![CDATA[""]]></literal>) + are escaped. + </para> + + <para> + Example: +<screen><![CDATA[ +SELECT xmltext('< foo & bar >'); + xmltext +------------------------- + < foo & bar > +]]></screen> + </para> + </sect3> + <sect3 id="functions-producing-xml-xmlcomment"> <title><literal>xmlcomment</literal></title> diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt index b33065d7bf..680d541673 100644 --- a/src/backend/catalog/sql_features.txt +++ b/src/backend/catalog/sql_features.txt @@ -633,7 +633,7 @@ X034 XMLAgg YES X035 XMLAgg: ORDER BY option YES X036 XMLComment YES X037 XMLPI YES -X038 XMLText NO +X038 XMLText YES X040 Basic table mapping YES X041 Basic table mapping: null absent YES X042 Basic table mapping: null as nil YES diff --git a/src/backend/utils/adt/xml.c b/src/backend/utils/adt/xml.c index 866d0d649a..dd8b453b71 100644 --- a/src/backend/utils/adt/xml.c +++ b/src/backend/utils/adt/xml.c @@ -47,6 +47,7 @@ #ifdef USE_LIBXML #include <libxml/chvalid.h> +#include <libxml/entities.h> #include <libxml/parser.h> #include <libxml/parserInternals.h> #include <libxml/tree.h> @@ -505,6 +506,10 @@ xmlcomment(PG_FUNCTION_ARGS) appendStringInfoText(&buf, arg); appendStringInfoString(&buf, "-->"); + + + + PG_RETURN_XML_P(stringinfo_to_xmltype(&buf)); #else NO_XML_SUPPORT(); @@ -5006,3 +5011,24 @@ XmlTableDestroyOpaque(TableFuncScanState *state) NO_XML_SUPPORT(); #endif /* not USE_LIBXML */ } + +Datum +xmltext(PG_FUNCTION_ARGS) +{ +#ifdef USE_LIBXML + text *arg = PG_GETARG_TEXT_PP(0); + text *result; + xmlChar *xmlbuf = NULL; + + xmlbuf = xmlEncodeSpecialChars(NULL,xml_text2xmlChar(arg)); + + Assert(xmlbuf); + + result = cstring_to_text_with_len((const char *) xmlbuf, xmlStrlen(xmlbuf)); + xmlFree(xmlbuf); + PG_RETURN_XML_P(result); +#else + NO_XML_SUPPORT(); + return 0; +#endif +} \ No newline at end of file diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9805bc6118..ff00c6365d 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8789,6 +8789,9 @@ { oid => '2922', descr => 'serialize an XML value to a character string', proname => 'text', prorettype => 'text', proargtypes => 'xml', prosrc => 'xmltotext' }, +{ oid => '3813', descr => 'generate XML text node', + proname => 'xmltext', proisstrict => 't', prorettype => 'xml', + proargtypes => 'text', prosrc => 'xmltext' }, { oid => '2923', descr => 'map table contents to XML', proname => 'table_to_xml', procost => '100', provolatile => 's', diff --git a/src/test/regress/expected/xml.out b/src/test/regress/expected/xml.out index 398345ca67..13e4296bf8 100644 --- a/src/test/regress/expected/xml.out +++ b/src/test/regress/expected/xml.out @@ -1785,3 +1785,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/expected/xml_1.out b/src/test/regress/expected/xml_1.out index 63b779470f..eb9c6f2ed4 100644 --- a/src/test/regress/expected/xml_1.out +++ b/src/test/regress/expected/xml_1.out @@ -1402,3 +1402,26 @@ DETAIL: This functionality requires the server to be built with libxml support. SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); ERROR: unsupported XML feature DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(' '); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo & <"bar">'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); +ERROR: unsupported XML feature +LINE 1: SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j':... + ^ +DETAIL: This functionality requires the server to be built with libxml support. diff --git a/src/test/regress/expected/xml_2.out b/src/test/regress/expected/xml_2.out index 43c2558352..c8ed8e0cfa 100644 --- a/src/test/regress/expected/xml_2.out +++ b/src/test/regress/expected/xml_2.out @@ -1765,3 +1765,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/sql/xml.sql b/src/test/regress/sql/xml.sql index a591eea2e5..bd4a4e7acd 100644 --- a/src/test/regress/sql/xml.sql +++ b/src/test/regress/sql/xml.sql @@ -660,3 +660,10 @@ SELECT * FROM XMLTABLE('*' PASSING '<e>pre<!--c1--><?pi arg?><![CDATA[&ent1]]><n \x SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); + +SELECT xmltext(NULL); +SELECT xmltext(''); +SELECT xmltext(' '); +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +SELECT xmltext('foo & <"bar">'); +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); \ No newline at end of file -- 2.34.1 ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 12:44 Daniel Gustafsson <[email protected]> parent: Jim Jones <[email protected]> 1 sibling, 0 replies; 21+ messages in thread From: Daniel Gustafsson @ 2023-08-25 12:44 UTC (permalink / raw) To: Jim Jones <[email protected]>; +Cc: Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]> > On 25 Aug 2023, at 14:42, Jim Jones <[email protected]> wrote: > Is there any tool like pgindent to take care of it automatically? No, pgindent doesn't address whitespace, only indentation of non-whitespace. -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 14:49 Vik Fearing <[email protected]> parent: Jim Jones <[email protected]> 1 sibling, 2 replies; 21+ messages in thread From: Vik Fearing @ 2023-08-25 14:49 UTC (permalink / raw) To: Jim Jones <[email protected]>; PostgreSQL Hackers <[email protected]> On 8/25/23 14:42, Jim Jones wrote: > Hi Vik > > Thanks for reviewing my patch! Thank you for writing it! > On 25.08.23 12:05, Vik Fearing wrote: >> I am replying to this email, but my comments are based on the v2 patch. >> >> Thank you for working on this, and I think this is a valuable >> addition. However, I have two issues with it. >> >> 1) There seems to be several spurious blank lines added that I do not >> think are warranted. > > I tried to copy the aesthetics of other functions, but it seems I failed > :) I removed a few blank lines. I hope it's fine now. I am talking specifically about this: @@ -505,6 +506,10 @@ xmlcomment(PG_FUNCTION_ARGS) appendStringInfoText(&buf, arg); appendStringInfoString(&buf, "-->"); + + + + PG_RETURN_XML_P(stringinfo_to_xmltype(&buf)); #else NO_XML_SUPPORT(); >> 2) This patch does nothing to address the <XML returning clause> so we >> can't claim to implement X038 without a disclaimer. Upon further >> review, the same is true of XMLCOMMENT() so maybe that is okay for >> this patch, and a more comprehensive patch for our xml features is >> necessary. > > If we decide to not address this point here, I can take a look at it and > work in a separated patch. I do not think this should be addressed in this patch because there are quite a lot of functions that need to handle this. -- Vik Fearing ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 15:40 Jim Jones <[email protected]> parent: Vik Fearing <[email protected]> 1 sibling, 1 reply; 21+ messages in thread From: Jim Jones @ 2023-08-25 15:40 UTC (permalink / raw) To: Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]> On 25.08.23 16:49, Vik Fearing wrote: > > I am talking specifically about this: > > @@ -505,6 +506,10 @@ xmlcomment(PG_FUNCTION_ARGS) > appendStringInfoText(&buf, arg); > appendStringInfoString(&buf, "-->"); > > + > + > + > + > PG_RETURN_XML_P(stringinfo_to_xmltype(&buf)); > #else > NO_XML_SUPPORT(); I have no idea how xmlcomment() got changed in this patch :D nice catch! > > I do not think this should be addressed in this patch because there > are quite a lot of functions that need to handle this. v4 attached. Jim Attachments: [text/x-patch] v4-0001-Add-XMLText-function-SQL-XML-X038.patch (8.4K, ../../[email protected]/2-v4-0001-Add-XMLText-function-SQL-XML-X038.patch) download | inline diff: From ea812791ffdce22d5f5615da361f67c801089f91 Mon Sep 17 00:00:00 2001 From: Jim Jones <[email protected]> Date: Fri, 25 Aug 2023 14:05:39 +0200 Subject: [PATCH v4] Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predifined entites (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. This patch includes also documentation and regression tests. --- doc/src/sgml/func.sgml | 30 +++++++++++++++++++++++ src/backend/catalog/sql_features.txt | 2 +- src/backend/utils/adt/xml.c | 22 +++++++++++++++++ src/include/catalog/pg_proc.dat | 3 +++ src/test/regress/expected/xml.out | 36 ++++++++++++++++++++++++++++ src/test/regress/expected/xml_1.out | 23 ++++++++++++++++++ src/test/regress/expected/xml_2.out | 36 ++++++++++++++++++++++++++++ src/test/regress/sql/xml.sql | 7 ++++++ 8 files changed, 158 insertions(+), 1 deletion(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7a0d4b9134..2f01a2c25d 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -14058,6 +14058,36 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple documents for processing in client applications. </para> + <sect3 id="functions-producing-xml-xmltext"> + <title><literal>xmltext</literal></title> + + <indexterm> + <primary>xmltext</primary> + </indexterm> + +<synopsis> +<function>xmltext</function> ( <type>text</type> ) <returnvalue>xml</returnvalue> +</synopsis> + + <para> + The function <function>xmltext</function> returns an XML value with a single + text node containing the input argument as its content. Predefined entities + like ampersand (<literal><![CDATA[&]]></literal>), left and right angle brackets + (<literal><![CDATA[< >]]></literal>), and quotation marks (<literal><![CDATA[""]]></literal>) + are escaped. + </para> + + <para> + Example: +<screen><![CDATA[ +SELECT xmltext('< foo & bar >'); + xmltext +------------------------- + < foo & bar > +]]></screen> + </para> + </sect3> + <sect3 id="functions-producing-xml-xmlcomment"> <title><literal>xmlcomment</literal></title> diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt index b33065d7bf..680d541673 100644 --- a/src/backend/catalog/sql_features.txt +++ b/src/backend/catalog/sql_features.txt @@ -633,7 +633,7 @@ X034 XMLAgg YES X035 XMLAgg: ORDER BY option YES X036 XMLComment YES X037 XMLPI YES -X038 XMLText NO +X038 XMLText YES X040 Basic table mapping YES X041 Basic table mapping: null absent YES X042 Basic table mapping: null as nil YES diff --git a/src/backend/utils/adt/xml.c b/src/backend/utils/adt/xml.c index 866d0d649a..7f6984dd88 100644 --- a/src/backend/utils/adt/xml.c +++ b/src/backend/utils/adt/xml.c @@ -47,6 +47,7 @@ #ifdef USE_LIBXML #include <libxml/chvalid.h> +#include <libxml/entities.h> #include <libxml/parser.h> #include <libxml/parserInternals.h> #include <libxml/tree.h> @@ -5006,3 +5007,24 @@ XmlTableDestroyOpaque(TableFuncScanState *state) NO_XML_SUPPORT(); #endif /* not USE_LIBXML */ } + +Datum +xmltext(PG_FUNCTION_ARGS) +{ +#ifdef USE_LIBXML + text *arg = PG_GETARG_TEXT_PP(0); + text *result; + xmlChar *xmlbuf = NULL; + + xmlbuf = xmlEncodeSpecialChars(NULL,xml_text2xmlChar(arg)); + + Assert(xmlbuf); + + result = cstring_to_text_with_len((const char *) xmlbuf, xmlStrlen(xmlbuf)); + xmlFree(xmlbuf); + PG_RETURN_XML_P(result); +#else + NO_XML_SUPPORT(); + return 0; +#endif +} \ No newline at end of file diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9805bc6118..ff00c6365d 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8789,6 +8789,9 @@ { oid => '2922', descr => 'serialize an XML value to a character string', proname => 'text', prorettype => 'text', proargtypes => 'xml', prosrc => 'xmltotext' }, +{ oid => '3813', descr => 'generate XML text node', + proname => 'xmltext', proisstrict => 't', prorettype => 'xml', + proargtypes => 'text', prosrc => 'xmltext' }, { oid => '2923', descr => 'map table contents to XML', proname => 'table_to_xml', procost => '100', provolatile => 's', diff --git a/src/test/regress/expected/xml.out b/src/test/regress/expected/xml.out index 398345ca67..13e4296bf8 100644 --- a/src/test/regress/expected/xml.out +++ b/src/test/regress/expected/xml.out @@ -1785,3 +1785,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/expected/xml_1.out b/src/test/regress/expected/xml_1.out index 63b779470f..eb9c6f2ed4 100644 --- a/src/test/regress/expected/xml_1.out +++ b/src/test/regress/expected/xml_1.out @@ -1402,3 +1402,26 @@ DETAIL: This functionality requires the server to be built with libxml support. SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); ERROR: unsupported XML feature DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(' '); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo & <"bar">'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); +ERROR: unsupported XML feature +LINE 1: SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j':... + ^ +DETAIL: This functionality requires the server to be built with libxml support. diff --git a/src/test/regress/expected/xml_2.out b/src/test/regress/expected/xml_2.out index 43c2558352..c8ed8e0cfa 100644 --- a/src/test/regress/expected/xml_2.out +++ b/src/test/regress/expected/xml_2.out @@ -1765,3 +1765,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/sql/xml.sql b/src/test/regress/sql/xml.sql index a591eea2e5..bd4a4e7acd 100644 --- a/src/test/regress/sql/xml.sql +++ b/src/test/regress/sql/xml.sql @@ -660,3 +660,10 @@ SELECT * FROM XMLTABLE('*' PASSING '<e>pre<!--c1--><?pi arg?><![CDATA[&ent1]]><n \x SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); + +SELECT xmltext(NULL); +SELECT xmltext(''); +SELECT xmltext(' '); +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +SELECT xmltext('foo & <"bar">'); +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); \ No newline at end of file -- 2.34.1 ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 15:56 Chapman Flack <[email protected]> parent: Vik Fearing <[email protected]> 1 sibling, 2 replies; 21+ messages in thread From: Chapman Flack @ 2023-08-25 15:56 UTC (permalink / raw) To: Vik Fearing <[email protected]>; +Cc: Jim Jones <[email protected]>; PostgreSQL Hackers <[email protected]> On 2023-08-25 10:49, Vik Fearing wrote: > I do not think this should be addressed in this patch because > there are quite a lot of functions that need to handle this. Indeed, as described in [0], we still largely provide the SQL/XML:2003 notion of a single XML datatype, not the distinguishable XML(DOCUMENT), XML(CONTENT), XML(SEQUENCE) types from :2006 and later, which has a number of adverse consequences for developers[1], and that wiki page proposed a couple possible ways forward[2]. Regards, -Chap [0] https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL/XML_Standards [1] https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL/XML_Standards#Obstacles_to_improving_conformance [2] https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL/XML_Standards#Possible_ways_forward ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-25 16:40 Vik Fearing <[email protected]> parent: Chapman Flack <[email protected]> 1 sibling, 0 replies; 21+ messages in thread From: Vik Fearing @ 2023-08-25 16:40 UTC (permalink / raw) To: Chapman Flack <[email protected]>; +Cc: Jim Jones <[email protected]>; PostgreSQL Hackers <[email protected]> On 8/25/23 17:56, Chapman Flack wrote: > [0] https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL/XML_Standards I was not aware of this page. What a wealth of information! -- Vik Fearing ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-08-26 17:02 Alvaro Herrera <[email protected]> parent: Chapman Flack <[email protected]> 1 sibling, 0 replies; 21+ messages in thread From: Alvaro Herrera @ 2023-08-26 17:02 UTC (permalink / raw) To: Chapman Flack <[email protected]>; +Cc: Vik Fearing <[email protected]>; Jim Jones <[email protected]>; PostgreSQL Hackers <[email protected]> On 2023-Aug-25, Chapman Flack wrote: > On 2023-08-25 10:49, Vik Fearing wrote: > > I do not think this should be addressed in this patch because > > there are quite a lot of functions that need to handle this. > > Indeed, as described in [0], we still largely provide the SQL/XML:2003 > notion of a single XML datatype, not the distinguishable XML(DOCUMENT), > XML(CONTENT), XML(SEQUENCE) types from :2006 and later, which has a > number of adverse consequences for developers[1], and that wiki page > proposed a couple possible ways forward[2]. Sadly, all the projects seem to have been pretty much abandoned in the meantime. Zorba has been dead for 9 years, xqilla for 6. Even XQC, the API they claim to implement, is dead. It sounds unlikely that there is *any* way forward here. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "El Maquinismo fue proscrito so pena de cosquilleo hasta la muerte" (Ijon Tichy en Viajes, Stanislaw Lem) ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-03 15:30 Daniel Gustafsson <[email protected]> parent: Jim Jones <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Daniel Gustafsson @ 2023-11-03 15:30 UTC (permalink / raw) To: Jim Jones <[email protected]>; +Cc: Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]> > On 25 Aug 2023, at 17:40, Jim Jones <[email protected]> wrote: > On 25.08.23 16:49, Vik Fearing wrote: >> I do not think this should be addressed in this patch because there are quite a lot of functions that need to handle this. > > v4 attached. I had a look at v4 of this patch and apart from pgindenting and moving the function within xml.c next to xmlcomment() I think this is ready. Just like Vik says upthread we can't really claim X038 conformance without a disclaimer, so I've added a 0002 which adds this to the XML spec conformance page in the docs. The attached v5 contains the above mentioned changes. I've marked this ready for committer in the CF app as well. -- Daniel Gustafsson Attachments: [application/octet-stream] v5-0001-Add-XMLText-function-SQL-XML-X038.patch (8.5K, ../../[email protected]/2-v5-0001-Add-XMLText-function-SQL-XML-X038.patch) download | inline diff: From 2062e70d66ffb68051c193ed3fff431412944443 Mon Sep 17 00:00:00 2001 From: Daniel Gustafsson <[email protected]> Date: Fri, 3 Nov 2023 14:53:09 +0100 Subject: [PATCH v5 1/2] Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predifined entites (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. Author: Jim Jones <[email protected]> Discussion: https://postgr.es/m/[email protected] --- doc/src/sgml/func.sgml | 30 +++++++++++++++++++++++ src/backend/catalog/sql_features.txt | 2 +- src/backend/utils/adt/xml.c | 22 +++++++++++++++++ src/include/catalog/pg_proc.dat | 3 +++ src/test/regress/expected/xml.out | 36 ++++++++++++++++++++++++++++ src/test/regress/expected/xml_1.out | 23 ++++++++++++++++++ src/test/regress/expected/xml_2.out | 36 ++++++++++++++++++++++++++++ src/test/regress/sql/xml.sql | 7 ++++++ 8 files changed, 158 insertions(+), 1 deletion(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 5d5ad7ee6a..564a8a0f5e 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -14180,6 +14180,36 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple documents for processing in client applications. </para> + <sect3 id="functions-producing-xml-xmltext"> + <title><literal>xmltext</literal></title> + + <indexterm> + <primary>xmltext</primary> + </indexterm> + +<synopsis> +<function>xmltext</function> ( <type>text</type> ) <returnvalue>xml</returnvalue> +</synopsis> + + <para> + The function <function>xmltext</function> returns an XML value with a single + text node containing the input argument as its content. Predefined entities + like ampersand (<literal><![CDATA[&]]></literal>), left and right angle brackets + (<literal><![CDATA[< >]]></literal>), and quotation marks (<literal><![CDATA[""]]></literal>) + are escaped. + </para> + + <para> + Example: +<screen><![CDATA[ +SELECT xmltext('< foo & bar >'); + xmltext +------------------------- + < foo & bar > +]]></screen> + </para> + </sect3> + <sect3 id="functions-producing-xml-xmlcomment"> <title><literal>xmlcomment</literal></title> diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt index b33065d7bf..680d541673 100644 --- a/src/backend/catalog/sql_features.txt +++ b/src/backend/catalog/sql_features.txt @@ -633,7 +633,7 @@ X034 XMLAgg YES X035 XMLAgg: ORDER BY option YES X036 XMLComment YES X037 XMLPI YES -X038 XMLText NO +X038 XMLText YES X040 Basic table mapping YES X041 Basic table mapping: null absent YES X042 Basic table mapping: null as nil YES diff --git a/src/backend/utils/adt/xml.c b/src/backend/utils/adt/xml.c index 2300c7ebf3..c401e7b821 100644 --- a/src/backend/utils/adt/xml.c +++ b/src/backend/utils/adt/xml.c @@ -47,6 +47,7 @@ #ifdef USE_LIBXML #include <libxml/chvalid.h> +#include <libxml/entities.h> #include <libxml/parser.h> #include <libxml/parserInternals.h> #include <libxml/tree.h> @@ -513,6 +514,27 @@ xmlcomment(PG_FUNCTION_ARGS) } +Datum +xmltext(PG_FUNCTION_ARGS) +{ +#ifdef USE_LIBXML + text *arg = PG_GETARG_TEXT_PP(0); + text *result; + xmlChar *xmlbuf = NULL; + + xmlbuf = xmlEncodeSpecialChars(NULL, xml_text2xmlChar(arg)); + + Assert(xmlbuf); + + result = cstring_to_text_with_len((const char *) xmlbuf, xmlStrlen(xmlbuf)); + xmlFree(xmlbuf); + PG_RETURN_XML_P(result); +#else + NO_XML_SUPPORT(); + return 0; +#endif /* not USE_LIBXML */ +} + /* * TODO: xmlconcat needs to merge the notations and unparsed entities diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 091f7e343c..f14aed422a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8793,6 +8793,9 @@ { oid => '2922', descr => 'serialize an XML value to a character string', proname => 'text', prorettype => 'text', proargtypes => 'xml', prosrc => 'xmltotext' }, +{ oid => '3813', descr => 'generate XML text node', + proname => 'xmltext', proisstrict => 't', prorettype => 'xml', + proargtypes => 'text', prosrc => 'xmltext' }, { oid => '2923', descr => 'map table contents to XML', proname => 'table_to_xml', procost => '100', provolatile => 's', diff --git a/src/test/regress/expected/xml.out b/src/test/regress/expected/xml.out index 398345ca67..13e4296bf8 100644 --- a/src/test/regress/expected/xml.out +++ b/src/test/regress/expected/xml.out @@ -1785,3 +1785,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/expected/xml_1.out b/src/test/regress/expected/xml_1.out index 63b779470f..eb9c6f2ed4 100644 --- a/src/test/regress/expected/xml_1.out +++ b/src/test/regress/expected/xml_1.out @@ -1402,3 +1402,26 @@ DETAIL: This functionality requires the server to be built with libxml support. SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); ERROR: unsupported XML feature DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(' '); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo & <"bar">'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); +ERROR: unsupported XML feature +LINE 1: SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j':... + ^ +DETAIL: This functionality requires the server to be built with libxml support. diff --git a/src/test/regress/expected/xml_2.out b/src/test/regress/expected/xml_2.out index 43c2558352..c8ed8e0cfa 100644 --- a/src/test/regress/expected/xml_2.out +++ b/src/test/regress/expected/xml_2.out @@ -1765,3 +1765,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/sql/xml.sql b/src/test/regress/sql/xml.sql index a591eea2e5..bd4a4e7acd 100644 --- a/src/test/regress/sql/xml.sql +++ b/src/test/regress/sql/xml.sql @@ -660,3 +660,10 @@ SELECT * FROM XMLTABLE('*' PASSING '<e>pre<!--c1--><?pi arg?><![CDATA[&ent1]]><n \x SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); + +SELECT xmltext(NULL); +SELECT xmltext(''); +SELECT xmltext(' '); +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +SELECT xmltext('foo & <"bar">'); +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); \ No newline at end of file -- 2.32.1 (Apple Git-133) [application/octet-stream] v5-0002-doc-Add-a-note-about-not-supporting-XML-SEQUENCE.patch (1.3K, ../../[email protected]/3-v5-0002-doc-Add-a-note-about-not-supporting-XML-SEQUENCE.patch) download | inline diff: From f2c31866ade71401870f5dba1ffe757f3fb047a9 Mon Sep 17 00:00:00 2001 From: Daniel Gustafsson <[email protected]> Date: Fri, 3 Nov 2023 14:53:19 +0100 Subject: [PATCH v5 2/2] doc: Add a note about not supporting XML(SEQUENCE) The SQL specification defines a RETURNING clause to a set of XML functions, where RETURNING CONTENT or RETURNING SEQUENCE can be defined. Since PostgreSQL doesn't support XML(SEQUENCE) all of these functions operate with an implicit RETURNING CONTENT. Discussion: https://postgr.es/m/[email protected] --- doc/src/sgml/features.sgml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/src/sgml/features.sgml b/doc/src/sgml/features.sgml index 575afa3476..966fd39882 100644 --- a/doc/src/sgml/features.sgml +++ b/doc/src/sgml/features.sgml @@ -199,6 +199,15 @@ standard. </para> </listitem> + + <listitem> + <para> + <productname>PostgreSQL</productname> does not support the + <literal>RETURNING CONTENT</literal> or <literal>RETURNING SEQUENCE</literal> + clauses, functions which are defined to have these in the specification + are implicitly returning content. + </para> + </listitem> </itemizedlist> </para> -- 2.32.1 (Apple Git-133) ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-03 15:45 Vik Fearing <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Vik Fearing @ 2023-11-03 15:45 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; Jim Jones <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On 11/3/23 16:30, Daniel Gustafsson wrote: >> On 25 Aug 2023, at 17:40, Jim Jones <[email protected]> wrote: > > Just like Vik says upthread we can't really claim X038 conformance without a > disclaimer, so I've added a 0002 which adds this to the XML spec conformance > page in the docs. We should put a short version of the disclaimer in sql_features.txt as well. -- Vik Fearing ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-03 16:14 Jim Jones <[email protected]> parent: Vik Fearing <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Jim Jones @ 2023-11-03 16:14 UTC (permalink / raw) To: Vik Fearing <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> Hi Daniel, hi Vik, Thanks a lot for the review! On 03.11.23 16:45, Vik Fearing wrote: > We should put a short version of the disclaimer in sql_features.txt as > well. You mean to add a disclaimer in the X038 entry? Something along these lines perhaps? X038 XMLText YES It does not address the <literal><XML returning clause></literal>, as it is not supported in <productname>PostgreSQL</productname>. Jim ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-03 18:05 Vik Fearing <[email protected]> parent: Jim Jones <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Vik Fearing @ 2023-11-03 18:05 UTC (permalink / raw) To: Jim Jones <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On 11/3/23 17:14, Jim Jones wrote: > Hi Daniel, hi Vik, > > Thanks a lot for the review! > > On 03.11.23 16:45, Vik Fearing wrote: >> We should put a short version of the disclaimer in sql_features.txt as >> well. > You mean to add a disclaimer in the X038 entry? Something along these > lines perhaps? > > X038 XMLText YES It does not address the <literal><XML > returning clause></literal>, as it is not supported in > <productname>PostgreSQL</productname>. I was thinking of something much shorter than that. Such as X038 XMLText YES supported except for RETURNING -- Vik Fearing ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-03 20:28 Jim Jones <[email protected]> parent: Vik Fearing <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Jim Jones @ 2023-11-03 20:28 UTC (permalink / raw) To: Vik Fearing <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On 03.11.23 19:05, Vik Fearing wrote: > I was thinking of something much shorter than that. Such as > > X038 XMLText YES supported except for RETURNING v6 attached includes this change and the doc addition from Daniel. Thanks! -- Jim Attachments: [text/x-patch] v6-0001-Add-XMLText-function-SQL-XML-X038.patch (9.6K, ../../[email protected]/2-v6-0001-Add-XMLText-function-SQL-XML-X038.patch) download | inline diff: From 703c882c254826f10f7bc076a2071741d086e8f6 Mon Sep 17 00:00:00 2001 From: Jim Jones <[email protected]> Date: Fri, 3 Nov 2023 21:15:21 +0100 Subject: [PATCH v6] Add XMLText function (SQL/XML X038) This function implements the standard XMLTest function, which converts text into xml text nodes. It uses the libxml2 function xmlEncodeSpecialChars to escape predifined entites (&"<>), so that those do not cause any conflict when concatenating the text node output with existing xml documents. This also adds a note in features.sgml about not supporting XML(SEQUENCE). The SQL specification defines a RETURNING clause to a set of XML functions, where RETURNING CONTENT or RETURNING SEQUENCE can be defined. Since PostgreSQL doesn't support XML(SEQUENCE) all of these functions operate with an implicit RETURNING CONTENT. Author: Jim Jones <[email protected]> Reviewed-by: Daniel Gustafsson <[email protected]> Reviewed-by: Vik Fearing <[email protected]> Discussion: https://postgr.es/m/[email protected] --- doc/src/sgml/features.sgml | 9 +++++++ doc/src/sgml/func.sgml | 30 +++++++++++++++++++++++ src/backend/catalog/sql_features.txt | 2 +- src/backend/utils/adt/xml.c | 22 +++++++++++++++++ src/include/catalog/pg_proc.dat | 3 +++ src/test/regress/expected/xml.out | 36 ++++++++++++++++++++++++++++ src/test/regress/expected/xml_1.out | 23 ++++++++++++++++++ src/test/regress/expected/xml_2.out | 36 ++++++++++++++++++++++++++++ src/test/regress/sql/xml.sql | 7 ++++++ 9 files changed, 167 insertions(+), 1 deletion(-) diff --git a/doc/src/sgml/features.sgml b/doc/src/sgml/features.sgml index 575afa3476..966fd39882 100644 --- a/doc/src/sgml/features.sgml +++ b/doc/src/sgml/features.sgml @@ -199,6 +199,15 @@ standard. </para> </listitem> + + <listitem> + <para> + <productname>PostgreSQL</productname> does not support the + <literal>RETURNING CONTENT</literal> or <literal>RETURNING SEQUENCE</literal> + clauses, functions which are defined to have these in the specification + are implicitly returning content. + </para> + </listitem> </itemizedlist> </para> diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index a6fcac0824..d963f0a0a0 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -14180,6 +14180,36 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple documents for processing in client applications. </para> + <sect3 id="functions-producing-xml-xmltext"> + <title><literal>xmltext</literal></title> + + <indexterm> + <primary>xmltext</primary> + </indexterm> + +<synopsis> +<function>xmltext</function> ( <type>text</type> ) <returnvalue>xml</returnvalue> +</synopsis> + + <para> + The function <function>xmltext</function> returns an XML value with a single + text node containing the input argument as its content. Predefined entities + like ampersand (<literal><![CDATA[&]]></literal>), left and right angle brackets + (<literal><![CDATA[< >]]></literal>), and quotation marks (<literal><![CDATA[""]]></literal>) + are escaped. + </para> + + <para> + Example: +<screen><![CDATA[ +SELECT xmltext('< foo & bar >'); + xmltext +------------------------- + < foo & bar > +]]></screen> + </para> + </sect3> + <sect3 id="functions-producing-xml-xmlcomment"> <title><literal>xmlcomment</literal></title> diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt index b33065d7bf..80c40eaf57 100644 --- a/src/backend/catalog/sql_features.txt +++ b/src/backend/catalog/sql_features.txt @@ -633,7 +633,7 @@ X034 XMLAgg YES X035 XMLAgg: ORDER BY option YES X036 XMLComment YES X037 XMLPI YES -X038 XMLText NO +X038 XMLText YES supported except for RETURNING X040 Basic table mapping YES X041 Basic table mapping: null absent YES X042 Basic table mapping: null as nil YES diff --git a/src/backend/utils/adt/xml.c b/src/backend/utils/adt/xml.c index 2300c7ebf3..c401e7b821 100644 --- a/src/backend/utils/adt/xml.c +++ b/src/backend/utils/adt/xml.c @@ -47,6 +47,7 @@ #ifdef USE_LIBXML #include <libxml/chvalid.h> +#include <libxml/entities.h> #include <libxml/parser.h> #include <libxml/parserInternals.h> #include <libxml/tree.h> @@ -513,6 +514,27 @@ xmlcomment(PG_FUNCTION_ARGS) } +Datum +xmltext(PG_FUNCTION_ARGS) +{ +#ifdef USE_LIBXML + text *arg = PG_GETARG_TEXT_PP(0); + text *result; + xmlChar *xmlbuf = NULL; + + xmlbuf = xmlEncodeSpecialChars(NULL, xml_text2xmlChar(arg)); + + Assert(xmlbuf); + + result = cstring_to_text_with_len((const char *) xmlbuf, xmlStrlen(xmlbuf)); + xmlFree(xmlbuf); + PG_RETURN_XML_P(result); +#else + NO_XML_SUPPORT(); + return 0; +#endif /* not USE_LIBXML */ +} + /* * TODO: xmlconcat needs to merge the notations and unparsed entities diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 091f7e343c..f14aed422a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8793,6 +8793,9 @@ { oid => '2922', descr => 'serialize an XML value to a character string', proname => 'text', prorettype => 'text', proargtypes => 'xml', prosrc => 'xmltotext' }, +{ oid => '3813', descr => 'generate XML text node', + proname => 'xmltext', proisstrict => 't', prorettype => 'xml', + proargtypes => 'text', prosrc => 'xmltext' }, { oid => '2923', descr => 'map table contents to XML', proname => 'table_to_xml', procost => '100', provolatile => 's', diff --git a/src/test/regress/expected/xml.out b/src/test/regress/expected/xml.out index 398345ca67..13e4296bf8 100644 --- a/src/test/regress/expected/xml.out +++ b/src/test/regress/expected/xml.out @@ -1785,3 +1785,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/expected/xml_1.out b/src/test/regress/expected/xml_1.out index 63b779470f..eb9c6f2ed4 100644 --- a/src/test/regress/expected/xml_1.out +++ b/src/test/regress/expected/xml_1.out @@ -1402,3 +1402,26 @@ DETAIL: This functionality requires the server to be built with libxml support. SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); ERROR: unsupported XML feature DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext(' '); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('foo & <"bar">'); +ERROR: unsupported XML feature +DETAIL: This functionality requires the server to be built with libxml support. +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); +ERROR: unsupported XML feature +LINE 1: SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j':... + ^ +DETAIL: This functionality requires the server to be built with libxml support. diff --git a/src/test/regress/expected/xml_2.out b/src/test/regress/expected/xml_2.out index 43c2558352..c8ed8e0cfa 100644 --- a/src/test/regress/expected/xml_2.out +++ b/src/test/regress/expected/xml_2.out @@ -1765,3 +1765,39 @@ SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH <foo/> | <foo/> (1 row) +SELECT xmltext(NULL); + xmltext +--------- + +(1 row) + +SELECT xmltext(''); + xmltext +--------- + +(1 row) + +SELECT xmltext(' '); + xmltext +--------- + +(1 row) + +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); + xmltext +-------------------------- + foo `$_-+?=*^%!|/\()[]{} +(1 row) + +SELECT xmltext('foo & <"bar">'); + xmltext +----------------------------------- + foo & <"bar"> +(1 row) + +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); + xmltext +--------------------------------- + x<P>73</P>0.42truej +(1 row) + diff --git a/src/test/regress/sql/xml.sql b/src/test/regress/sql/xml.sql index a591eea2e5..bd4a4e7acd 100644 --- a/src/test/regress/sql/xml.sql +++ b/src/test/regress/sql/xml.sql @@ -660,3 +660,10 @@ SELECT * FROM XMLTABLE('*' PASSING '<e>pre<!--c1--><?pi arg?><![CDATA[&ent1]]><n \x SELECT * FROM XMLTABLE('.' PASSING XMLELEMENT(NAME a) columns a varchar(20) PATH '"<foo/>"', b xml PATH '"<foo/>"'); + +SELECT xmltext(NULL); +SELECT xmltext(''); +SELECT xmltext(' '); +SELECT xmltext('foo `$_-+?=*^%!|/\()[]{}'); +SELECT xmltext('foo & <"bar">'); +SELECT xmltext('x'|| '<P>73</P>'::xml || .42 || true || 'j'::char); \ No newline at end of file -- 2.34.1 ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-04 14:01 Vik Fearing <[email protected]> parent: Jim Jones <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Vik Fearing @ 2023-11-04 14:01 UTC (permalink / raw) To: Jim Jones <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On 11/3/23 21:28, Jim Jones wrote: > > On 03.11.23 19:05, Vik Fearing wrote: >> I was thinking of something much shorter than that. Such as >> >> X038 XMLText YES supported except for RETURNING > > v6 attached includes this change and the doc addition from Daniel. There are some typos in the commit message, but otherwise this looks commitable to me. -- Vik Fearing ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-06 10:49 Daniel Gustafsson <[email protected]> parent: Vik Fearing <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Daniel Gustafsson @ 2023-11-06 10:49 UTC (permalink / raw) To: Vik Fearing <[email protected]>; +Cc: Jim Jones <[email protected]>; PostgreSQL Hackers <[email protected]> > On 4 Nov 2023, at 15:01, Vik Fearing <[email protected]> wrote: > > On 11/3/23 21:28, Jim Jones wrote: >> On 03.11.23 19:05, Vik Fearing wrote: >>> I was thinking of something much shorter than that. Such as >>> >>> X038 XMLText YES supported except for RETURNING >> v6 attached includes this change and the doc addition from Daniel. > > There are some typos in the commit message, but otherwise this looks commitable to me. I took another look at this today, fixes the above mentioned typos and some tiny cosmetic things and pushed it. -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: [PATCH] Add XMLText function (SQL/XML X038) @ 2023-11-06 11:15 Jim Jones <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 0 replies; 21+ messages in thread From: Jim Jones @ 2023-11-06 11:15 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; Vik Fearing <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On 06.11.23 11:49, Daniel Gustafsson wrote: > I took another look at this today, fixes the above mentioned typos and some > tiny cosmetic things and pushed it. > > -- > Daniel Gustafsson > Awesome! Thanks Daniel and Vik for reviewing and pushing this patch :) -- Jim ^ permalink raw reply [nested|flat] 21+ messages in thread
end of thread, other threads:[~2023-11-06 11:15 UTC | newest] Thread overview: 21+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-03-25 11:49 [PATCH] Add XMLText function (SQL/XML X038) Jim Jones <[email protected]> 2023-03-25 11:53 ` Pavel Stehule <[email protected]> 2023-03-25 12:25 ` Jim Jones <[email protected]> 2023-08-25 08:27 ` Jim Jones <[email protected]> 2023-08-25 10:05 ` Vik Fearing <[email protected]> 2023-08-25 12:42 ` Jim Jones <[email protected]> 2023-08-25 12:44 ` Daniel Gustafsson <[email protected]> 2023-08-25 14:49 ` Vik Fearing <[email protected]> 2023-08-25 15:40 ` Jim Jones <[email protected]> 2023-11-03 15:30 ` Daniel Gustafsson <[email protected]> 2023-11-03 15:45 ` Vik Fearing <[email protected]> 2023-11-03 16:14 ` Jim Jones <[email protected]> 2023-11-03 18:05 ` Vik Fearing <[email protected]> 2023-11-03 20:28 ` Jim Jones <[email protected]> 2023-11-04 14:01 ` Vik Fearing <[email protected]> 2023-11-06 10:49 ` Daniel Gustafsson <[email protected]> 2023-11-06 11:15 ` Jim Jones <[email protected]> 2023-08-25 15:56 ` Chapman Flack <[email protected]> 2023-08-25 16:40 ` Vik Fearing <[email protected]> 2023-08-26 17:02 ` Alvaro Herrera <[email protected]> 2023-07-26 10:49 [PATCH v3 2/7] Row pattern recognition patch (parse/analysis). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox