Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kZz1l-0005Ck-Kf for pgsql-bugs@arkaria.postgresql.org; Tue, 03 Nov 2020 16:19:21 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kZz1k-0000Yq-JC for pgsql-bugs@arkaria.postgresql.org; Tue, 03 Nov 2020 16:19:20 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kZz1k-0000Yj-DB for pgsql-bugs@lists.postgresql.org; Tue, 03 Nov 2020 16:19:20 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kZz1h-0004ut-7S for pgsql-bugs@lists.postgresql.org; Tue, 03 Nov 2020 16:19:19 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 0A3GJ4Gp922843; Tue, 3 Nov 2020 11:19:05 -0500 From: Tom Lane To: Heikki Linnakangas cc: James Cloos , pgsql-bugs@lists.postgresql.org Subject: Re: pg should ignore u+200b zero width space In-reply-to: References: <919181.1604415167@sss.pgh.pa.us> Comments: In-reply-to Heikki Linnakangas message dated "Tue, 03 Nov 2020 17:13:41 +0200" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <922841.1604420344.1@sss.pgh.pa.us> Date: Tue, 03 Nov 2020 11:19:04 -0500 Message-ID: <922842.1604420344@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk Heikki Linnakangas writes: > On 03/11/2020 16:52, Tom Lane wrote: >> Perhaps it'd be all right to confine the change in behavior to >> just modifying the error text in cases where we were going to >> throw an error anyway. But I think this is much harder than >> it sounds to do in a valid, safe way. > Yeah, my thinking was to just add a hint when you're throwing a syntax > error anyway. Something simple like check if client_encoding is utf8 and > there is a U+200b in the query string, and add the hint if so. It > doesn't need to catch all cases, and rare false positives are OK too. TBH, that's exactly the kind of under-baked solution I *don't* want. For starters, &zwsp is hardly the only problem here; even more common is  , which exists in most encodings not only UTF8. And once we had that, there'd no doubt be pressure to ignore BOMs. And so on. Also I don't really want us throwing such errors when the funny space is inside a literal or comment; chasing false positives like that will cost users way more time than they could save when the hint is on-point. regards, tom lane