Mail-Followup-To: pgsql-sql@postgresql.org
To: pgsql-sql@postgresql.org
From: Tim Landscheidt <tim@tim-landscheidt.de>
Subject: Re: Token separation
Date: Mon, 16 Jan 2012 15:12:26 +0000
Organization: <URI:http://www.tim-landscheidt.de/>
Lines: 42
Message-ID: <m3d3ajd6lh.fsf@passepartout.tim-landscheidt.de>
References: <m3ipkd1jho.fsf@passepartout.tim-landscheidt.de>
	<17710.1326647758@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Mail-Copies-To: never
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux)
Cancel-Lock: sha1:cNbG4Q3qzvFZlzzpiBmEoT4yzt8=

Tom Lane <tgl@sss.pgh.pa.us> wrote:

>> [ "0x13" is lexed as "0" then "x13" ]

>> Is this behaviour really conforming to the standard?

> Well, it's pretty much the universal behavior of flex-based lexers,
> anyway.  A token ends when the next character can no longer sensibly
> be added to it.

I know, but - off the top of my head - in most other lan-
guages "0abc" will then give a syntax error.

> Possibly the documentation should be tweaked to mention the
> number-followed-by-identifier case.

Especially if you consider such cases:

| tim=# SELECT 1D1; SELECT 1E1; SELECT 1F1;
|  d1
| ----
|   1
| (1 Zeile)

|  ?column?
| ----------
|        10
| (1 Zeile)

|  f1
| ----
|   1
| (1 Zeile)

| tim=#

I don't think it's common to hit this, but the documentation
surely could use a caveat.  I will write something up and
submit it to -docs.

Thanks,
Tim