Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o2GoC-0007md-SX for pgsql-hackers@arkaria.postgresql.org; Fri, 17 Jun 2022 18:35:05 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1o2GoB-0000B3-M1 for pgsql-hackers@arkaria.postgresql.org; Fri, 17 Jun 2022 18:35:03 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o2Gm5-0002uD-Ou for pgsql-hackers@lists.postgresql.org; Fri, 17 Jun 2022 18:32:54 +0000 Received: from out5-smtp.messagingengine.com ([66.111.4.29]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o2Gm1-0005eI-Dx for pgsql-hackers@lists.postgresql.org; Fri, 17 Jun 2022 18:32:52 +0000 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 6FF765C0143; Fri, 17 Jun 2022 14:32:48 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 17 Jun 2022 14:32:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1655490768; x=1655577168; bh=q 8zxC4tGeN3buPwfnCgWP4f3CQ4bVWQTFCCwlaY9yb8=; b=DVmQxxk5+5QX8WfpF pqMKl2Nmq1Q5dsk68AqdMgg46hX43WigD3KIjdG4Oj8AYDl3cpHwr8/XfU8/d8+N IuqfyDy42soS8MEPadD0E9UeeSLXGMVe+K3kb3cRtWz1avf1w9OUrb+QhiXd2WWS ff8UbuOWTPnw20HRtOY8IpwruG1vrpmfuZGv57G6Q5caWkFzqEoO52ogj8VqGpUR hL3UgyQ2FaT8LGIbfWzsHg0njd3NRsq04s0kpah6CghOyibD1oXADLa+iISSKB3H s5g6n7Ob3rsPhXWWn1Rt+2HqJkY3HKnDCDrYrGHGWz/66HpeHGGh7HY9h/SPsm24 8vbMA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedruddvhedguddvgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkgggtugfgjgesmh ekreertddtjeenucfhrhhomheptehlvhgrrhhoucfjvghrrhgvrhgruceorghlvhhhvghr rhgvsegrlhhvhhdrnhhoqdhiphdrohhrgheqnecuggftrfgrthhtvghrnhepudelkeekgf dttdettdekgfduvdeiffelteegjeeihfetheekveeftedtuddvhfeunecuffhomhgrihhn pegvnhhtvghrphhrihhsvggusgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurf grrhgrmhepmhgrihhlfhhrohhmpegrlhhvhhgvrhhrvgesrghlvhhhrdhnohdqihhprdho rhhg X-ME-Proxy: Feedback-ID: ia2694551:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 17 Jun 2022 14:32:47 -0400 (EDT) Received: by perhan.alvh.no-ip.org (Postfix, from userid 1000) id F037A352; Fri, 17 Jun 2022 20:31:50 +0200 (CEST) Date: Fri, 17 Jun 2022 20:31:50 +0200 From: Alvaro Herrera To: Kyotaro Horiguchi Cc: tgl@sss.pgh.pa.us, daniele.varrazzo@gmail.com, pgsql-hackers@lists.postgresql.org Subject: Re: Using PQexecQuery in pipeline mode produces unexpected Close messages Message-ID: <20220617183150.ilgokxp22mzywnhh@alvherre.pgsql> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="igg5o36yau2li7um" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20220616.120747.1289213836703016883.horikyota.ntt@gmail.com> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --igg5o36yau2li7um Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit On 2022-Jun-16, Kyotaro Horiguchi wrote: > The attached is a crude patch to separate the state for PIPELINE-IDLE > from PGASYNC_IDLE. I haven't found a better way.. Ah, yeah, this might be a way to fix this. Something very similar to a PIPELINE_IDLE mode was present in Craig's initial patch for pipeline mode. However, I fought very hard to remove it, because it seemed to me that failing to handle it correctly everywhere would lead to more bugs than not having it. (Indeed, there were some.) However, I see now that your patch would not only fix this bug, but also let us remove the ugly "notionally not-idle" bit in fe-protocol3.c, which makes me ecstatic. So let's push forward with this. However, this means we'll have to go over all places that use asyncStatus to ensure that they all handle the new value correctly. I did found one bug in your patch: in the switch for asyncStatus in PQsendQueryStart, you introduce a new error message. With the current tests, that never fires, which is telling us that our coverage is not complete. But with the right sequence of libpq calls, which the attached adds (note that it's for REL_14_STABLE), that can be hit, and it's easy to see that throwing an error there is a mistake. The right action to take there is to let the action through. Others to think about: PQisBusy (I think no changes are needed), PQfn (I think it should accept a call in PGASYNC_PIPELINE_IDLE mode; fully untested in pipeline mode), PQexitPipelineMode (I think it needs to return error; needs test case), PQsendFlushRequest (I think it should let through; ditto). I also attach a patch to make the test suite use Test::Differences, if available. It makes the diffs of the traces much easier to read, when they fail. (I wish for a simple way to set the context size, but that would need a shim routine that I'm currently too lazy to write.) -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ --igg5o36yau2li7um Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v5-0001-Avoid-going-IDLE-in-pipeline-mode.patch"