Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kcrpv-0002JM-TT for pgsql-bugs@arkaria.postgresql.org; Wed, 11 Nov 2020 15:15:04 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kcrpu-0001uD-Hk for pgsql-bugs@arkaria.postgresql.org; Wed, 11 Nov 2020 15:15:02 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kcrYQ-0005xs-Ih for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2020 14:56:59 +0000 Received: from wforward2-smtp.messagingengine.com ([64.147.123.31]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kcrYN-0003nx-LG for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2020 14:56:57 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailforward.west.internal (Postfix) with ESMTP id B3FBADD6; Wed, 11 Nov 2020 09:56:53 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 11 Nov 2020 09:56:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=s/wUAoz7e0vn+neMuhlwpe2nm+dZCQmjsWY+uUNlE AU=; b=M2tWgv7wk3oogu273leFlJX3DdF6yOrEWVJKt94r09+/65ctqRw0ih9/c vuie19Zsr5nXM6oBtqtv+rSww5WajUzblV4dfRdlrH1pm/ZJk8mukinih/g/4uT8 K2l3zmckeh1gLPRgSSWpO+p1UXo5HrjIim1hefRZ1cWqzq1fvAhbGZBXInqScVBx TPawOnCiIzF07xuE6E6rnuQ6pyTE9chKuIJIUv8bbHvi7MOq9EaUtRBOigyb2C/Z QH6Ej633bcsCzOz8kzkfp/UFOe103Q6RI9XLX2ivkTgtkI7jDq2EghPwl05IuFJ0 48Syj8VDEePBjDPZ8zbhNHeWlzPvw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedruddvtddgtdelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffvfhfhohfkffgfgggjtgfgsehtjeertddtfeejnecuhfhrohhmpefrvght vghrucfgihhsvghnthhrrghuthcuoehpvghtvghrrdgvihhsvghnthhrrghuthesvghnth gvrhhprhhishgvuggsrdgtohhmqeenucggtffrrghtthgvrhhnpeehheeijefgvefhvdeh leekueevhfehhedvheetgffhteefgeeggfevjeejfffgvdenucfkphepkeejrddujeejrd ejhedrleegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhepphgvthgvrhdrvghishgvnhhtrhgruhhtsegvnhhtvghrphhrihhsvggusgdrtghomh X-ME-Proxy: Received: from april.pezone.net (p57b14b5e.dip0.t-ipconnect.de [87.177.75.94]) by mail.messagingengine.com (Postfix) with ESMTPA id 71F963063083; Wed, 11 Nov 2020 09:56:51 -0500 (EST) Subject: Re: pg should ignore u+200b zero width space To: Tom Lane , Heikki Linnakangas References: <919181.1604415167@sss.pgh.pa.us> Cc: James Cloos , pgsql-bugs@lists.postgresql.org From: Peter Eisentraut Organization: EnterpriseDB Message-ID: <739c79f3-ed87-f584-b209-d17318f7fdfe@enterprisedb.com> Date: Wed, 11 Nov 2020 15:56:50 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <919181.1604415167@sss.pgh.pa.us> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 2020-11-03 15:52, Tom Lane wrote: > The difficulty with doing anything in this space --- whether it be > ignoring, throwing an error, or whatever --- is that it makes the > lexer's behavior encoding-sensitive and potentially locale-sensitive. > That's problematic for all sorts of reasons. One of the worst is > that frontend programs such as psql and ecpg also have SQL lexers, > and there'd be no way to keep their behavior in precise sync with > the backend's. (They might not even be running in the same encoding, > never mind locale.) It might even be possible to build security > holes around that; recall the fun we've had with trying to lock > down quoting rules in encodings where backslash can be part of a > multibyte character :-(. I think we could support whitespace properly if we're using Unicode, and use ASCII whitespace for all other encodings. The combinations to deal with are finite. But it is a tricky problem that needs to be approached with care.