Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kZxgE-0001w5-Qe for pgsql-bugs@arkaria.postgresql.org; Tue, 03 Nov 2020 14:53:02 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kZxgD-0003NU-OD for pgsql-bugs@arkaria.postgresql.org; Tue, 03 Nov 2020 14:53:01 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kZxgD-0003NM-IS for pgsql-bugs@lists.postgresql.org; Tue, 03 Nov 2020 14:53:01 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kZxgB-0004AL-DY for pgsql-bugs@lists.postgresql.org; Tue, 03 Nov 2020 14:53:00 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 0A3Eqlm0919182; Tue, 3 Nov 2020 09:52:47 -0500 From: Tom Lane To: Heikki Linnakangas cc: James Cloos , pgsql-bugs@lists.postgresql.org Subject: Re: pg should ignore u+200b zero width space In-reply-to: References: Comments: In-reply-to Heikki Linnakangas message dated "Tue, 03 Nov 2020 15:52:45 +0200" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <919180.1604415167.1@sss.pgh.pa.us> Date: Tue, 03 Nov 2020 09:52:47 -0500 Message-ID: <919181.1604415167@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk Heikki Linnakangas writes: > On 03/11/2020 15:41, James Cloos wrote: >> pg should treat a no break space after whitespace as just more >> whitespace. > Hmm. I'm not sure if change the behavior is a good idea, but a hint in > the error message would be nice. Something like: The difficulty with doing anything in this space --- whether it be ignoring, throwing an error, or whatever --- is that it makes the lexer's behavior encoding-sensitive and potentially locale-sensitive. That's problematic for all sorts of reasons. One of the worst is that frontend programs such as psql and ecpg also have SQL lexers, and there'd be no way to keep their behavior in precise sync with the backend's. (They might not even be running in the same encoding, never mind locale.) It might even be possible to build security holes around that; recall the fun we've had with trying to lock down quoting rules in encodings where backslash can be part of a multibyte character :-(. Perhaps it'd be all right to confine the change in behavior to just modifying the error text in cases where we were going to throw an error anyway. But I think this is much harder than it sounds to do in a valid, safe way. regards, tom lane