From: Andrew Gierth <andrew@tao11.riddles.org.uk>
To: pgsql-hackers@postgresql.org
Subject: Re: A really subtle lexer bug
Date: Mon, 20 Aug 2018 17:58:49 +0100
Message-ID: <87ftz90yma.fsf@news-spur.riddles.org.uk> (raw)
In-Reply-To: <87va851ppl.fsf@news-spur.riddles.org.uk>
References: <87va851ppl.fsf@news-spur.riddles.org.uk>
>>>>> "Andrew" == Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
Andrew> select f(a =>-1); -- ERROR: column "a" does not exist
Andrew> I guess the fix is to extend the existing special case code
Andrew> that checks for one character left after removing trailing [+-]
Andrew> and also check for the two-character ops "<>" ">=" "<=" "=>"
Andrew> "!=".
Patch attached.
This fixes two bugs: first the mis-lexing of two-char ops as mentioned
originally; second, the O(N^3) lexing time of strings of - or +
characters is reduced to O(N^2) (in practice it's better than O(N^2)
once N gets large because the bison stack gets blown out, ending the
loop early).
--
Andrew (irc:RhodiumToad)
Attachments:lex.patch (text/x-patch, 2.1K) downloaddiff --git a/src/backend/parser/scan.l b/src/backend/parser/scan.lindex 0cd782827a..cc855ffb1c 100644--- a/src/backend/parser/scan.l+++ b/src/backend/parser/scan.l@@ -885,20 +885,34 @@ other .
* to forbid operator names like '?-' that could not be
* sequences of SQL operators.
*/
- while (nchars > 1 &&- (yytext[nchars - 1] == '+' ||- yytext[nchars - 1] == '-'))+ if (nchars > 1 &&+ (yytext[nchars - 1] == '+' ||+ yytext[nchars - 1] == '-'))
{
int ic;
for (ic = nchars - 2; ic >= 0; ic--)
{
- if (strchr("~!@#^&|`?%", yytext[ic]))+ char c = yytext[ic];+ if (c == '~' || c == '!' || c == '@' ||+ c == '#' || c == '^' || c == '&' ||+ c == '|' || c == '`' || c == '?' ||+ c == '%')
break;
}
- if (ic >= 0)- break; /* found a char that makes it OK */- nchars--; /* else remove the +/-, and check again */+ if (ic < 0)+ {+ /*+ * didn't find a qualifying character, so remove+ * all trailing [+-]+ */+ for (nchars--;+ nchars > 1 &&+ (yytext[nchars - 1] == '+' ||+ yytext[nchars - 1] == '-');+ nchars--)+ continue;+ }
}
SET_YYLLOC();
@@ -916,6 +930,25 @@ other .
if (nchars == 1 &&
strchr(",()[].;:+-*/%^<>=", yytext[0]))
return yytext[0];
+ /*+ * Likewise, if what we have left is two chars, and+ * those match the tokens ">=", "<=", "=>", "<>" or+ * "!=", then we must return the sppropriate token+ * rather than the generic Op.+ */+ if (nchars == 2)+ {+ if (yytext[0] == '=' && yytext[1] == '>')+ return EQUALS_GREATER;+ if (yytext[0] == '>' && yytext[1] == '=')+ return GREATER_EQUALS;+ if (yytext[0] == '<' && yytext[1] == '=')+ return LESS_EQUALS;+ if (yytext[0] == '<' && yytext[1] == '>')+ return NOT_EQUALS;+ if (yytext[0] == '!' && yytext[1] == '=')+ return NOT_EQUALS;+ }
}
/*
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andrew@tao11.riddles.org.uk
Subject: Re: A really subtle lexer bug
In-Reply-To: <87ftz90yma.fsf@news-spur.riddles.org.uk>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox