Subject: Re: Fix XML handling with DOCTYPE
To: Tom Lane <tgl@sss.pgh.pa.us>
References: 
 <CAN-V+g-6JqUQEQZ55Q3toXEN6d5Ez5uvzL4VR+8KtvJKj31taw@mail.gmail.com>
 <5C8D5D16.7090206@anastigmatix.net> <23319.1552769738@sss.pgh.pa.us>
 <5C8D6681.4070504@anastigmatix.net> <24203.1552771272@sss.pgh.pa.us>
 <5C8D79AF.7010204@anastigmatix.net> <5146.1552837534@sss.pgh.pa.us>
 <5C8E7FC0.8040401@anastigmatix.net> <31664.1552842991@sss.pgh.pa.us>
 <5C8E8E2F.9050600@anastigmatix.net> <25228.1552849567@sss.pgh.pa.us>
 <5C8ECAA4.3090301@anastigmatix.net>
Cc: Ryan Lambert <ryan@rustprooflabs.com>, pgsql-hackers@postgresql.org
From: Chapman Flack <chap@anastigmatix.net>
Message-ID: <5C8FD4EE.7090107@anastigmatix.net>
Date: Mon, 18 Mar 2019 13:27:10 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.7.0
MIME-Version: 1.0
In-Reply-To: <5C8ECAA4.3090301@anastigmatix.net>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Precedence: bulk

There might be too many different email threads on this with patches,
but in case it went under the radar, xml-content-2006-3.patch appeared
in my previous message on this thread[1].

It is based on a simple pre-check of the prefix of the input, determining
which form of parse to apply. That may or may not be simpler than parse-
once-save-error-parse-again-report-first-error, but IMV it's a more direct
solution and clearer (the logic is clearly about "how do I determine the way
this input should be parsed?" which is the problem on the table, rather
than "how should I save and regurgitate this libxml error?" which turns the
problem on the table to a different one).

I decided, for a first point of reference, to wear the green eyeshade and
write a pre-check that exactly implements the applicable rules. That gives
a starting point for simplifications that are probably safe.

For example, a bunch of lines at the end have to do with verifying the
content inside of a processing-instruction, after finding where it ends.
We could reasonably decide that, for the purpose of skipping it, knowing
where it ends is enough, as libxml will parse it next and report any errors
anyway.

That would slightly violate my intention of sending input to (the parser
that wasn't asked for) /only/ when it's completely clear (from the prefix
we've seen) that that's where it should go. The relaxed version could do
that in completely-clear cases and cases with an invalid PI ahead of what
looks like a DTD. But you'd pretty much expect both parsers to produce
the same message for a bad PI anyway.

That made me just want to try it now, and--surprise!--the messages from
libxml are not the same. So maybe I would lean to keeping the green-eyeshade
rules in the test, if you can stomach them, but I would understand taking
them out.

Regards,
-Chap

[1] https://www.postgresql.org/message-id/5C8ECAA4.3090301@anastigmatix.net