Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lhDf4-0001eX-F2 for pgsql-hackers@arkaria.postgresql.org; Thu, 13 May 2021 15:54:07 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lhDf3-0005gh-DP for pgsql-hackers@arkaria.postgresql.org; Thu, 13 May 2021 15:54:05 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lhDf2-0005gT-DI for pgsql-hackers@lists.postgresql.org; Thu, 13 May 2021 15:54:05 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lhDex-00061M-Li for pgsql-hackers@lists.postgresql.org; Thu, 13 May 2021 15:54:04 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id BAE2E282C; Thu, 13 May 2021 11:53:55 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 13 May 2021 11:53:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=3nI3uG3ZlYLtWC/XrQK/tUritzDK3xDT9RySBHYq+GU=; b=FVcrOkuu nZXliUgK5fQoDURHG6A/b1kv3p51y80aJSnyizZ9Vz+8ru/2YN4mp/QfSn6LkjZU zwZn1EnuJjzN1kGVREF9AirQvvxYT7jqI0biivzA+Tp8448l3MWz83jahLB/RqRB 8jHExKt4q37RsBlwHIDzXEvNBfnYyKSE6qETfQd2J3cujiHeqs18JU2qdnigOcA8 w9YYqRIh7drKFR8k5ibUU3ztJ+gy2bzavCnLJniIrsQaJUdfsPD0m++5g2ye2jdX /PpGIjNvpVoCF1G+M/Pks+iYfN5J+c02ZlsObI6qqaxGxxvIf+HuJoV+hmKmdsCN 4UJVGjdHpzTsOw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdehgedgleegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfggtggugfgjfgesmhekreertderudenucfhrhhomheptehlvhgr rhhoucfjvghrrhgvrhgruceorghlvhhhvghrrhgvsegrlhhvhhdrnhhoqdhiphdrohhrgh eqnecuggftrfgrthhtvghrnhepvdevvdfgffehkefhvdelfedvteefieeuffeigeeivefg hedtfedtleefgffgtdfgnecuffhomhgrihhnpehpohhsthhgrhgvshhqlhdrohhrghenuc fkphepudeltddrleehrdduledrleehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomheprghlvhhhvghrrhgvsegrlhhvhhdrnhhoqdhiphdrohhrgh X-ME-Proxy: Received: from perhan.alvh.no-ip.org (unknown [190.95.19.95]) by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 13 May 2021 11:53:54 -0400 (EDT) Received: by perhan.alvh.no-ip.org (Postfix, from userid 1000) id B4CE82A0ADC; Thu, 13 May 2021 11:53:51 -0400 (-04) Date: Thu, 13 May 2021 11:53:51 -0400 From: Alvaro Herrera To: Tom Lane Cc: Pavel Borisov , Dilip Kumar , PostgreSQL Hackers Subject: Re: OOM in spgist insert Message-ID: <20210513155351.GA7848@alvherre.pgsql> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="PEIAKu/WMn1b1Hv9" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1517578.1620919523@sss.pgh.pa.us> User-Agent: Mutt/1.10.1 (2018-07-13) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --PEIAKu/WMn1b1Hv9 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit On 2021-May-13, Tom Lane wrote: > BTW, another nasty thing I discovered while testing this is that > the CHECK_FOR_INTERRUPTS() at line 2146 is useless, because > we're holding a buffer lock there so InterruptHoldoffCount > 0. > So once you get into this loop you can't even cancel the query. > Seems like that needs a fix, too. This comment made me remember a patch I've had for a while, which splits the CHECK_FOR_INTERRUPTS() definition in two -- one of them is INTERRUPTS_PENDING_CONDITION() which let us test the condition separately; that allows the lock we hold to be released prior to actually processing the interrupts. The btree code modified was found to be an actual problem in production when a btree is corrupted in such a way that vacuum would get an infinite loop. I don't remember the exact details but I think we saw vacuum running for a couple of weeks, and had to restart the server in order to terminate it (since it wouldn't respond to signals). -- Álvaro Herrera Valdivia, Chile "I am amazed at [the pgsql-sql] mailing list for the wonderful support, and lack of hesitasion in answering a lost soul's question, I just wished the rest of the mailing list could be like this." (Fotis) (http://archives.postgresql.org/pgsql-sql/2006-06/msg00265.php) --PEIAKu/WMn1b1Hv9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-Split-CHECK_FOR_INTERRUPTS.patch" From 5a008141f135bef5ba933b1e3b65c457f58ad85a Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Thu, 13 May 2021 11:41:19 -0400 Subject: [PATCH] Split CHECK_FOR_INTERRUPTS This allows the condition to be checked even when in an interrupts-held situation, so that we can exit that (eg. release some lock we know we're holding) in order to process them. --- src/backend/access/nbtree/nbtpage.c | 7 +++++++ src/include/miscadmin.h | 20 ++++++++------------ 2 files changed, 15 insertions(+), 12 deletions(-) diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index ebec8fa5b8..00de713035 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -2397,6 +2397,13 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, { bool leftsibvalid = true; + if (INTERRUPTS_PENDING_CONDITION()) + { + _bt_relbuf(rel, leafbuf); + ProcessInterrupts(); + return false; /* should not occur */ + } + /* * Before we follow the link from the page that was the left * sibling mere moments ago, validate its right link. This diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h index 95202d37af..c5c441c2e9 100644 --- a/src/include/miscadmin.h +++ b/src/include/miscadmin.h @@ -98,23 +98,19 @@ extern PGDLLIMPORT volatile uint32 CritSectionCount; extern void ProcessInterrupts(void); #ifndef WIN32 +#define INTERRUPTS_PENDING_CONDITION() \ + (unlikely(InterruptPending)) +#else +#define INTERRUPTS_PENDING_CONDITION() \ + (unlikely(UNBLOCKED_SIGNAL_QUEUE()) ? pgwin32_dispatch_queued_signals() : 0, \ + unlikely(InterruptPending)) +#endif #define CHECK_FOR_INTERRUPTS() \ do { \ - if (unlikely(InterruptPending)) \ + if (INTERRUPTS_PENDING_CONDITION()) \ ProcessInterrupts(); \ } while(0) -#else /* WIN32 */ - -#define CHECK_FOR_INTERRUPTS() \ -do { \ - if (unlikely(UNBLOCKED_SIGNAL_QUEUE())) \ - pgwin32_dispatch_queued_signals(); \ - if (unlikely(InterruptPending)) \ - ProcessInterrupts(); \ -} while(0) -#endif /* WIN32 */ - #define HOLD_INTERRUPTS() (InterruptHoldoffCount++) -- 2.20.1 --PEIAKu/WMn1b1Hv9--