Received: from makus.postgresql.org (makus.postgresql.org [98.129.198.125])
	by mail.postgresql.org (Postfix) with ESMTP id 621A21FE6211
	for <pgsql-sql@postgresql.org>; Sun, 15 Jan 2012 13:16:16 -0400 (AST)
Received: from sss.pgh.pa.us ([66.207.139.130])
	by makus.postgresql.org with esmtp (Exim 4.72)
	(envelope-from <tgl@sss.pgh.pa.us>) id 1RmTgi-0000AA-77
	for pgsql-sql@postgresql.org; Sun, 15 Jan 2012 17:16:16 +0000
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.14.2/8.14.2) with ESMTP id q0FHFwJX017711;
	Sun, 15 Jan 2012 12:15:59 -0500 (EST)
To: Tim Landscheidt <tim@tim-landscheidt.de>
cc: pgsql-sql@postgresql.org
Subject: Re: Token separation 
In-reply-to: <m3ipkd1jho.fsf@passepartout.tim-landscheidt.de> 
References: <m3ipkd1jho.fsf@passepartout.tim-landscheidt.de>
Comments: In-reply-to Tim Landscheidt <tim@tim-landscheidt.de>
	message dated "Sun, 15 Jan 2012 08:02:27 +0000"
Date: Sun, 15 Jan 2012 12:15:58 -0500
Message-ID: <17710.1326647758@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
X-Pg-Spam-Score: -1.9 (-)
X-Archive-Number: 201201/57
X-Sequence-Number: 36228

Tim Landscheidt <tim@tim-landscheidt.de> writes:
> [ "0x13" is lexed as "0" then "x13" ]

> Is this behaviour really conforming to the standard?

Well, it's pretty much the universal behavior of flex-based lexers,
anyway.  A token ends when the next character can no longer sensibly
be added to it.

Possibly the documentation should be tweaked to mention the
number-followed-by-identifier case.

			regards, tom lane