Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pCutU-0000Rl-UI for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Jan 2023 03:56:49 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pCutT-0003Ki-5R for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Jan 2023 03:56:47 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pCutR-0003KZ-7O for pgsql-hackers@lists.postgresql.org; Wed, 04 Jan 2023 03:56:46 +0000 Received: from wout3-smtp.messagingengine.com ([64.147.123.19]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pCutN-0002zW-V9 for pgsql-hackers@postgresql.org; Wed, 04 Jan 2023 03:56:44 +0000 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id A838132009AB; Tue, 3 Jan 2023 22:56:39 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 03 Jan 2023 22:56:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= cc:cc:content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm2; t=1672804599; x=1672890999; bh=PVfk9CRIp/ zFc9rD/JfQNQpbOybQm1Iu4dFKoYzhB2Q=; b=JIoLvu7lU3oj7if7l4C7vZTDh8 tU21GDV1pRZ/dOzVwWY0r6j8Nb/ftM9O7FXwLYBXMT+wQEKQJdGKkM0uGJNrGB2K 9eOI+2kvV2LflZnteFJwgzpuFJ2fE1MIYG3Pj08t+47gvfRlO/s9Uq0I8emnqdIj uB+jKR951LdqgfYnv6krqA2VuWMo47sn1/rRNY9F4hgL7jZNakkKF9Vvol+Ei2MI g15oFwAvZrU6Cv1J6j9kz08qUvxIG3aHUnhGlBMpfm6/rZ/Zc+gq+rxsgMI+ZFUl +qmbUT7bblAZZKCTJtMs38KPShIj0pk8VN/qoI68K4vJ+g5eatA6Dc9OticA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1672804599; x=1672890999; bh=PVfk9CRIp/zFc9rD/JfQNQpbOybQ m1Iu4dFKoYzhB2Q=; b=JkZLlgtX+xHvbjc9zbe1g2UFGI0YlOPh/NRBNSsR7D+k IdtE3NMuWeTBhnyHN14M2/KB/6mIVKv2c4hl/jhCQkrHsiK4wvAPNXEpCt5mIMML yOW12b+OuGWu6CczkV/JoT7E3+kvugsHS1mCVaptxY7iTB8ALiI91kLifgYM+eKu pRe7ZiypFqBHVt6Fjz4OHuXFOyl8wcywQreoBtOSvc2K4tccJz/DUwe44YxPZ7b1 1aRJnUWbVWxKGBRGUxtAHsJ5YiI9RelH38eCt9lfF83cgw2TKZCPM8zqBqOXTqzR nhvr/rmoHWLSSOF1fD2f/gZIw4RI2rwXpSJFZabvVg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrjeehgdeiiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpeffhffvvefukfhfgggtuggjsehttdertddttddvnecuhfhrohhmpeetnhgurhgv shcuhfhrvghunhguuceorghnughrvghssegrnhgrrhgriigvlhdruggvqeenucggtffrrg htthgvrhhnpedvffefvefhteevffegieetfefhtddvffejvefhueetgeeludehteevudei tedtudenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe grnhgurhgvshesrghnrghrrgiivghlrdguvg X-ME-Proxy: Feedback-ID: id4a34324:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 3 Jan 2023 22:56:38 -0500 (EST) Date: Tue, 3 Jan 2023 19:56:36 -0800 From: Andres Freund To: Peter Geoghegan Cc: pgsql-hackers@postgresql.org Subject: Re: pgsql: Delay commit status checks until freezing executes. Message-ID: <20230104035636.hy5djyr2as4gbc4q@awork3.anarazel.de> References: <20230104005433.k5aptjbqx75fksfm@awork3.anarazel.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hi, On 2023-01-03 17:54:37 -0800, Peter Geoghegan wrote: > (Pruning -committers from the list, since cross-posting to -hackers > resulted in this being held up for moderation.) I still think these moderation rules are deeply unhelpful... > On Tue, Jan 3, 2023 at 5:15 PM Peter Geoghegan wrote: > > > > On Tue, Jan 3, 2023 at 4:54 PM Andres Freund wrote: > > > There's some changes from TransactionIdDidCommit() to !TransactionIdDidAbort() > > > that don't look right to me. If the server crashed while xid X was > > > in-progress, TransactionIdDidCommit(X) will return false, but so will > > > TransactionIdDidAbort(X). So besides moving when the check happens you also > > > changed what's being checked in a more substantial way. > > > > I did point this out on the thread. I made this change with the > > intention of making the check more robust. Apparently this was > > misguided. > > > > Where is the behavior that you describe documented, if anywhere? I don't know - I think there's a explicit comment somewhere, but I couldn't find it immediately. There's a bunch of indirect references to in in heapam_visibility.c, with comments like "it must have aborted or crashed". The reason for the behaviour is that we do not have any mechanism for going through the clog and aborting all in-progress-during-crash transactions. So we'll end up with the clog for all in-progress-during-crash transaction being zero / TRANSACTION_STATUS_IN_PROGRESS. IMO it's almost always wrong to use TransactionIdDidAbort(). > When the server crashes, and we have a problem case, what does > TransactionLogFetch()/TransactionIdGetStatus() (which are the guts of > both TransactionIdDidCommit and TransactionIdDidAbort) report about > the XID? Depends a bit on the specifics, but mostly TRANSACTION_STATUS_IN_PROGRESS. > > > Also, why did you change when MarkBufferDirty() happens? Previously it > > > happened before we modify the page contents, now after. That's probably fine > > > (it's the order suggested in transam/README), but seems like a mighty subtle > > > thing to change at the same time as something unrelated, particularly without > > > even mentioning it? > > > > I changed it because the new order is idiomatic. I didn't think that > > this was particularly worth mentioning, or even subtle. The logic from > > heap_execute_freeze_tuple() only performs simple in-place > > modifications. I think changes in how WAL logging is done are just about always worth mentioning in a commit message... Greetings, Andres Freund