Subject: WIP - xmlvalidate implementation from TODO list

From: Marcos Magueta <[email-masked]>
Date: Sat, 6 Dec 2025 20:38:24 -0300

Hello!

I am likely one of the few people still using XML, but I noticed XSD schema
validation is still a TODO on postgresql, which I have some personal use
cases for.

In this patch I attempt to implement XMLVALIDATE with the already in use
libxml following a version I got my hands on of the SQL/XML standard of
2016.

In short, I had to add the ACCORDING word to comply with it and completely
ignored the version that was already in the src that fetches arbitrary
schemas (it refers to validations of dtds, which is another more
troublesome TODO).

I had problems running the regression tests on my machine, so I could only
test the feature by spawning a modified instance of postgresql and issuing
queries through psql, therefore I am marking it as WIP. If anyone can
assert the tests pass, I would be glad.

Also, this is my first patch, so I might have not followed standard
practices as best as I could, so please pay particular attention to that on
review.

Cheers,
Marcos Magueta.

---

From: Kirill Reshke <[email-masked]>
Date: Sun, 7 Dec 2025 12:34:22 +0500

On Sun, 7 Dec 2025 at 04:38, Marcos Magueta <[email-masked]> wrote:
>
> Hello!
>
> I am likely one of the few people still using XML, but I noticed XSD schema validation is still a TODO on postgresql, which I have some personal use cases for.
>
> In this patch I attempt to implement XMLVALIDATE with the already in use libxml following a version I got my hands on of the SQL/XML standard of 2016.
>
> In short, I had to add the ACCORDING word to comply with it and completely ignored the version that was already in the src that fetches arbitrary schemas (it refers to validations of dtds, which is another more troublesome TODO).
>
> I had problems running the regression tests on my machine, so I could only test the feature by spawning a modified instance of postgresql and issuing queries through psql, therefore I am marking it as WIP. If anyone can assert the tests pass, I would be glad.
>
> Also, this is my first patch, so I might have not followed standard practices as best as I could, so please pay particular attention to that on review.
>
> Cheers,
> Marcos Magueta.

HI!

1)
> + // Default case since nothing got returned
> + // out of the normal path for validation calls to libxml

PostgreSQL uses /**/ comments style.

2)
XML regression test suite fails, see attached. By the way, what are
your issues with running `make check` ?

3)
By the way, in [0] we have this

`
The function in PostgreSQL produces an “unimplemented” error, because
PostgreSQL does not have any implementation of the mechanism assumed
in the standard for registering schemas in advance, which is necessary
to address the security implications of a function that could refer to
schemas from arbitrary locations.
`

How does your patch resolve this? I did not find any change in this area

4)
Also I want to mention that without a doc, the patch is not in a
commitable shape.

5) I am a bit surprised by this grammar rule:

>  XMLVALIDATE '(' document_or_content a_expr ACCORDING TO XMLSCHEMA a_expr ')'

this allow a wide class of expressions accepted by parser, like

`SELECT xmlvalidate(DOCUMENT  (select oid from pg_class) ACCORDING TO
XMLSCHEMA (select 32)) AS is_valid FROM xml_validation_test;`

Is this expected? a_expr is way more than string constants and column
references..  If yes, the regression test that you added, does not
test this..

p.s. I failed to find in google SQL/XML standard of 2016. So, I cannot
double-check if this feature is compliant with it...

[0] https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL/XML_Standards

-- 
Best regards,
Kirill Reshke

---

From: Marcos Magueta <[email-masked]>
Date: Sun, 7 Dec 2025 16:43:40 -0300

Thank you for your kind review!

Before I continue with the implementation, I would like to address
your concerns and discuss it further and see if it's worth carrying
on.

1) Will do!

2) The issue is the following:
```
./../../src/test/regress/pg_regress --temp-instance=./tmp_check
--inputdir=. --bindir=     --dlpath=. --max-concurrent-tests=20
 --schedule=./parallel_schedule
# +++ regress check in src/test/regress +++
# initializing database system by copying initdb template
# could not exec "sh": No such file or directory
Bail out!# postmaster failed, examine
```

This is likely due to my setup on Nix. If any command assumes paths on
conventional *nix I am often in trouble. I am checking that with a
friend, but any insights are welcome. The out file I generated was
while being completely blindfolded.

By the way, the diff you sent is assuming global paths for some
reason, so I couldn't apply it without manually changing them.

3) From what I understand, that refers to ISO/IEC 9075-14:2016, chapter and
section 11.6, page 245:
 - Parts of the grammar that reference an URI: <XML valid according to
what> ::= <XML valid according to URI> | <XML valid according to identifier>
 - NO NAMESPACE, which unspecifies the qualification but can access
something through a LOCATI ON
 - ID which should allow access to a registered schema
So they amount to:
  ACCORDING TO XMLSCHEMA URI <uri> [LOCATION <uri>]
  ACCORDING TO XMLSCHEMA NO NAMESPACE [LOCATION <uri>]
  ACCORDING TO XMLSCHEMA ID <registered_schema_name>

What I did is rely on the protection mechanisms that are already
implemented to just side-step the issue of arbitrary retrieval.

`PgXmlErrorContext * pg_xml_init(PgXmlStrictness strictness)` starts
the xml error context preventing any attempt by libxml2 to load an
external entity (DTD, XSD from URL, local file, etc.) returns an empty
string instead. Check around the line 1420 of xml.c:

```
/*
 * Also, install an entity loader to prevent unwanted fetches of external
 * files and URLs.
 */
errcxt->saved_entityfunc = xmlGetExternalEntityLoader();
xmlSetExternalEntityLoader(xmlPgEntityLoader);
```

So since I am relying on a TEXT for the schema, there should be no
issues of that sort. It does however cut part of the grammar that
handles locations, which is part of the standard, and would require
this feature to be much bigger in scope...

4) I suppose you refer to doc/src/sgml/func/func-xml.sgml. Will do

5) Hmm my intent was to simply handle TEXT on the xmlschema portion,
so the expr rule on that side is indeed an oversight. Now about the
first argument, that is just following the pattern already specified
in other xml functions, like XMLPARSE and XMLSERIALIZE, which have the
same <XML Value Expression> specified in the grammar. So that might
have alraedy diverged from the standard a while back...

This is currently grammatically valid, for example:
```
select xmlparse(DOCUMENT  (select oid from pg_class limit 1));
ERROR:  invalid XML document
DETAIL:  line 1: Start tag expected, '<' not found
2619
^
```

As a summary, it does not fully implement schemas as first-class
objects, as that would require extra parts of the grammar specified in
the standard, so I capitulated to use schemas as provided text. That's
already safe in my understanding given the shielding in place. If we
are to implement the rest, I think other serious concerns would arise,
like role management, how to store schemas, etc. And when it comes
to that, is it worth all the trouble just for xml? I would like this
feature and I think the solution of relying on text is decent, since
the cost of complying 100% seems very high for low returns.

Em dom., 7 de dez. de 2025 às 04:34, Kirill Reshke <[email-masked]>
escreveu:

> > I am likely one of the few people still using XML, but I noticed XSD
> schema validation is still a TODO on postgresql, which I have some personal
> use cases for.
> >
> > In short, I had to add the ACCORDING word to comply with it and
> completely ignored the version that was already in the src that fetches
> arbitrary schemas (it refers to validations of dtds, which is another more
> troublesome TODO).
> >
> > I had problems running the regression tests on my machine, so I could
> only test the feature by spawning a modified instance of postgresql and
> issuing queries through psql, therefore I am marking it as WIP. If anyone
> can assert the tests pass, I would be glad.
> >

---

From: Marcos Magueta <[email-masked]>
Date: Fri, 19 Dec 2025 00:25:51 -0300

Hello again!

I took some time to actually finish this feature. I think the answers
for the previous questions are now clearer. I checked the
initialization and the protections are indeed in place since commit
a4b0c0aaf093a015bebe83a24c183e10a66c8c39, which specifically states:

> Prevent access to external files/URLs via XML entity references.

> xml_parse() would attempt to fetch external files or URLs as needed to
> resolve DTD and entity references in an XML value, thus allowing
> unprivileged database users to attempt to fetch data with the privileges
> of the database server.  While the external data wouldn't get returned
> directly to the user, portions of it could be exposed in error messages
> if the data didn't parse as valid XML; and in any case the mere ability
> to check existence of a file might be useful to an attacker.
>
> The ideal solution to this would still allow fetching of references that
> are listed in the host system's XML catalogs, so that documents can be
> validated according to installed DTDs.  However, doing that with the
> available libxml2 APIs appears complex and error-prone, so we're not going
> to risk it in a security patch that necessarily hasn't gotten wide review.
> So this patch merely shuts off all access, causing any external fetch to
> silently expand to an empty string.  A future patch may improve this.

With that, the obvious affordance on the xmlvalidate implementation
was to not rely on external schema sources on the host
catalog. Therefore the implementation relies solely on expressions
that necessarily evaluate to a schema in plain text.

I added the requested documentation and a bunch of tests for each
scenario. I would appreciate another round of reviews whenever someone
has the time and patience.

At last, to nourish the curiosity: I had issues with make check, as
stated above on the e-mail thread. These got resolved when I changed
`execl` to `execlp` on `pg_regress.c`. I of course did not commit
such, but more people I know have had the very same issue while
relying on immutable package managers.

---

From: Marcos Magueta <[email-masked]>
Date: Wed, 31 Dec 2025 17:26:55 -0300

Hello again!

Is there any interest in this? I understand PostgreSQL has bigger fish to
fry, but I would like to at least know; in case this was just forgotten.

Regards!

---

From: Jim Jones <[email-masked]>
Date: Sun, 22 Feb 2026 12:44:01 +0100

Hi Marcos

On 22/02/2026 06:11, Marcos Magueta wrote:
>> Unfortunately, the patch no longer applies.[1]
> Trying to understand why it doesn't, we can get back to that.

I think the cfbot is confused about the ordering of your patch set. You
labelled the files as

* v5-0002-Add-XMLVALIDATE.patch
* v6-0001-Add-CREATE-XMLSCHEMA.patch

which generates two issues:

1) both of them should have the save version, i.e v5 or v6.
2) the XMLVALIDATE patch depends on CREATE XMLSCHEMA, and the files
order right now suggest quite the opposite. The next iteration should be:

* v7-0001-Add-CREATE-XMLSCHEMA.patch
* v7-0002-Add-XMLVALIDATE.patch

Best, Jim

---

From: Marcos Magueta <[email-masked]>
Date: Sun, 15 Mar 2026 01:25:22 -0300

Hey again Jim,

Sorry for the long pause on this.

I was thinking about the idea of managing the catalogs for read and write,
and I'm coming around to the idea of predefined roles after all. Relying on
conventional namespace-level ACLs for this turns out to be impractical.
With the normal ACL, a schema is object agnostic, so there's no clean way
to selectively restrict XML schema creation without also affecting other
objects in the sam enamespace. A simple scenario like limiting who can
write already gets messy. I did consider RLS on the catalog, but that would
be unprecedented for a pg_* table and would break assumptions throughout
the system, like pg_dump, dependency tracking, syscache lookups... blah!

That said, I'd like to hear from more people on this before committing to
an approach, assuming there's still legitimate interest in moving this work
forward.

On the potential CPU burn from validation: I think in practice it's
comparable to what you'd get from a complex index, heavy check constraint,
or trigger function. However, the nature of the input (and I mean the XML
schema definitions as plain text here), likely coming from the application
layer, sets a warrant for extra caution I guess. Limiting the depth and
size of both the schema and the document being validated would reduce
compatibility, but goes a long way in preventing resource exhaustion, so
it's a fairly trivial option to implement.

I tested the changes on my own instance with Cirrus, so it might be the
case of the version mismatch indeed. I simply reattached both as v6 once
again (no changes, still working on master).

Regards, Marcos.

---

From: Pavel Stehule <[email-masked]>
Date: Sun, 15 Mar 2026 16:20:42 +0100

ne 15. 3. 2026 v 13:58 odesílatel Jim Jones <[email-masked]>
napsal:

> Hi Marcos
>
> On 15/03/2026 05:25, Marcos Magueta wrote:
> > I was thinking about the idea of managing the catalogs for read and
> > write, and I'm coming around to the idea of predefined roles after all.
> > Relying on conventional namespace-level ACLs for this turns out to be
> > impractical. With the normal ACL, a schema is object agnostic, so
> > there's no clean way to selectively restrict XML schema creation without
> > also affecting other objects in the sam enamespace. A simple scenario
> > like limiting who can write already gets messy. I did consider RLS on
> > the catalog, but that would be unprecedented for a pg_* table and would
> > break assumptions throughout the system, like pg_dump, dependency
> > tracking, syscache lookups... blah!
> >
> > That said, I'd like to hear from more people on this before committing
> > to an approach, assuming there's still legitimate interest in moving
> > this work forward.
>
>
> I guess we can assume that everything added to the official todo list is
> of interest for the community -- at least I do :).
>
>
> > comparable to what you'd get from a complex index, heavy check
> > constraint, or trigger function. However, the nature of the input (and I
> > mean the XML schema definitions as plain text here), likely coming from
> > the application layer, sets a warrant for extra caution I guess.
> > Limiting the depth and size of both the schema and the document being
> > validated would reduce compatibility, but goes a long way in preventing
> > resource exhaustion, so it's a fairly trivial option to implement.
>
>
> I took the liberty to add Pavel to this thread. He has way more
> experience than me in this part of the code, and perhaps he can share
> his opinion on the predefined roles for XML schemas and his impressions
> on the patch as a whole.
>

I have no opinion about this now.  I need to read both variants.

---

From: Pavel Stehule <[email-masked]>
Date: Mon, 30 Mar 2026 22:28:39 +0200

Hi

> his opinion on the predefined roles for XML schemas and his impressions

I checked db2 doc, and if I understand their doc, the XML schema is
identified by "relational identifier" SQLschema.name

So taking XML schema as catalog object is the correct analogy and using acl
looks to me correct.

But what is different (patch and db2), one relational identifier can
identify a group of XML schemas. So there is relation 1:N not 1:1. You can
see the REGISTER XMLSCHEMA command.

The schema registration is different on MSSQL where it is more similar to
some local cache, and schema is identified only by uri. In this case using
ACL can be messy, and I can imagine having some dedicated role that can
register a new xml schema. But MSSQL doesn't support SQL/XML XMLVALIDATE
function.

Both concepts are workable, and I have no strong preference for one or
second (maybe DB2 concept is better for Postgres, probably DB2 concept is
closer to SQL/XML). But if we use relational identifiers, then it should be
consistent with other usage of relational identifiers - there ACL should be
used.

Regards

Pavel

---

From: Jim Jones <[email-masked]>
Date: Tue, 31 Mar 2026 11:29:38 +0200

On 30/03/2026 22:28, Pavel Stehule wrote:
> So taking XML schema as catalog object is the correct analogy and using
> acl looks to me correct.

+1

> identify a group of XML schemas. So there is relation 1:N not 1:1. You
> can see the REGISTER XMLSCHEMA command.

I wonder what are the applications for this 1:N relationship between xml
schemas and their identifiers. Sounds cool though.

> The schema registration is different on MSSQL where it is more similar
> to some local cache, and schema is identified only by uri. In this case
> using ACL can be messy, and I can imagine having some dedicated role
> that can register a new xml schema. But MSSQL doesn't support SQL/XML
> XMLVALIDATE function.
> 
> second (maybe DB2 concept is better for Postgres, probably DB2 concept
> is closer to SQL/XML). But if we use relational identifiers, then it
> should be consistent with other usage of relational identifiers - there
> ACL should be used.

A few other things I noticed in v6

== event triggers (event_trigger.c) ==

OBJECT_XMLSCHEMA is in the elog(ERROR) path of stringify_grant_objtype()
and stringify_adefprivs_objtype(). Since the grammar does support
GRANT/REVOKE ... ON XMLSCHEMA, any event trigger on ddl_command_end that
fires during a GRANT on an XML schema will raise a elog(ERROR,
"unsupported object type").

An additional test here would be nice as well.

== corrupt dump ==

pg_dump is ignoring GRANT statements. Since the feature supports
GRANT/REVOKE on XMLSCHEMAS, omitting ACL dump means privileges are
silently lost during dump/restore.

== inconsistent DDL statements ==

The docs suggest that ALTER XMLSCHEMA ... IF EXISTS is supported, which
isn't the case. It should be either implemented or just removed from the
docs.

== different error message for build without libxml ==

In xml.c the error messag used is "unsupported XML feature", but you're
using "xmlschema support requires libxml" in pg_xmlschema.c. I think we
should be consistent here.

Thanks!

Best, Jim

---

From: Pavel Stehule <[email-masked]>
Date: Tue, 31 Mar 2026 12:57:49 +0200

> schemas and their identifiers. Sounds cool though.

I found
https://www.developpez.net/forums/d540201/bases-donnees/db2/validation-xml-db2-v9-5-a/

---

From: Jim Jones <[email-masked]>
Date: Wed, 1 Apr 2026 09:47:01 +0200

On 31/03/2026 12:57, Pavel Stehule wrote:
> I found https://www.developpez.net/forums/d540201/bases-donnees/db2/
> validation-xml-db2-v9-5-a/ <https://www.developpez.net/forums/d540201/
> bases-donnees/db2/validation-xml-db2-v9-5-a/>

Interesting. In DB2 it looks indeed more like a schema composition.
Perhaps something to consider once the current approach is ready.

Best, Jim

---

From: Pavel Stehule <[email-masked]>
Date: Wed, 1 Apr 2026 10:10:56 +0200

> Perhaps something to consider once the current approach is ready.

It should be considered from the beginning - it means that the relational
identifier is not unique.  It can complicate upgrades. And it can
complicate possible ports from db2.

maybe it can be implemented just as an extension - we don't need DDL
commands CREATE XMLSCHEMA - there can be some function. If I understand
correctly, the SQL/XML knows just XMLVALIDATE command - and in the first
step - this can be just a function too.

Theoretically there can be different access - we can introduce a generic
XMLVALIDATE command, that can call some callback. This callback can be
implemented in an extension. One extension can XML storage identified by
relation identifier, another can implement some shared storage on the disc
outside the database etc. So the organization of the patch can be different

1. creating API for extensions for implementing loading the schema from
some sources.
2. implementation XMLVALIDATE that uses this API
3. creating an extension in contrib, that implements XML storage identified
by relational API.

Because we don't have an extensible parser, the XMLVALIDATE command should
be in core, but other parts are not.

XML is a little bit outdated - but the proposed infrastructure can be used
for any documents, so it can theoretically supports json, jsonb too, and
this can helps with accepting XMLVALIDATE to core. I expect something like
XMLVALIDATE will be JSON standard in a few years.

Regards

Pavel

---

From: Jim Jones <[email-masked]>
Date: Wed, 1 Apr 2026 16:39:56 +0200

> step - this can be just a function too. 

Yeah, the spec does not dictate how the schemas are stored or how
identifiers are mapped to XSDs. The way I see it, in the current state
of this patch, we miss the option to add XSDs to an existing XMLSCHEMA
identifier (something like ALTER XMLSCHEMA .. ADD ...), or even the
option to pass multiple XSDs in CREATE XMLSCHEMA.

It makes the implementation of CREATE XMLSCHEMA (or similar DDL) a bit
more challenging, as it must always resolve all root's dependencies to
ensure that XMLVALIDATE has all the necessary info to validate a given XML.

> relation identifier, another can implement some shared storage on the
> disc outside the database etc. So the organization of the patch can be
> different 
> 
> 3. creating an extension in contrib, that implements XML storage
> identified by relational API.
> 
> Because we don't have an extensible parser, the XMLVALIDATE command
> should be in core, but other parts are not.

How would XMLVALIDATE without this contrib extension look like?

Best, Jim

---

From: Pavel Stehule <[email-masked]>
Date: Wed, 1 Apr 2026 18:57:24 +0200

> How would XMLVALIDATE without this contrib extension look like?

The API can be similar to PLpgSQL plugin API - and rendezvous variable

Pavel

---

From: Marcos Magueta <[email-masked]>
Date: Thu, 2 Apr 2026 15:22:32 -0300

Hey Jim and Pavel, thanks for demonstrating interest.

> XML is a little bit outdated - but the proposed infrastructure can be
used for any documents, so it can theoretically supports json, jsonb too
[...]. I expect something like XMLVALIDATE will be JSON standard in a few
years.

That doesn't sound like a bad idea to me. But I suspect I would need to
almost completely revamp the implementation in the patch so far, so I would
like to confirm a few things before committing to anything on that front.

I am still unsure how the permissions would be managed properly with the
API approach. It seems to me we could rely on the ACL like before, but I
sense there's something more arcane around it when it comes to the
extension.

I didn't realize, but DB2 actually gives you decomposition over an xml
hierarchy into tables, which is an insanely valuable feature. I wonder how
we could go with that here using the current catalog approach.

Is there anything else that might be troublesome with changing approaches?

Regards, Marcos.

---

From: Pavel Stehule <[email-masked]>
Date: Thu, 2 Apr 2026 20:57:41 +0200

Hi

> like to confirm a few things before committing to anything on that front.

It is usual when you design a feature that is not fully described by
standard :-/. And then it is good to start primary from part (command),
that is well described by standard.

> sense there's something more arcane around it when it comes to the

There are two different things - 1. possibility to registrate xml schema -
you need just to CREATE right for related pg_namespace, 2. possibility to
use xml schema - and you need to USAGE right - and this can be checked
inside extension. Probably you cannot use all ACL related macros, and maybe
you have to design some function instead of GRANT and REVOKE.  Maybe some
form of RLS can be used. When you separate storage of XML schemas to an
extension, then you can implement just some minimalistic implementation
(just what is necessary for regress tests) - and some more complex storage
of XML schema can be designed (developed) outside the main branch.
Hypothetical storage from contrib can be enhanced step by step.

Regards

Pavel