Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r2tDu-001Frr-MI for pgsql-hackers@arkaria.postgresql.org; Tue, 14 Nov 2023 13:12:58 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1r2tDt-007btw-BS for pgsql-hackers@arkaria.postgresql.org; Tue, 14 Nov 2023 13:12:57 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r2tDt-007bto-1i for pgsql-hackers@lists.postgresql.org; Tue, 14 Nov 2023 13:12:57 +0000 Received: from out3-smtp.messagingengine.com ([66.111.4.27]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r2tDq-005XMZ-Dj for pgsql-hackers@postgresql.org; Tue, 14 Nov 2023 13:12:55 +0000 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 925FB5C014B; Tue, 14 Nov 2023 08:12:53 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Tue, 14 Nov 2023 08:12:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1699967573; x= 1700053973; bh=amxrEkZwnaDl4us117JOyN1RCUXxJOJsCdzUUI2IfJY=; b=E xU/jIXKUoYyskXvcW+KWfkZMS6MyOdf1gn8XPswkY9SXeW8te4s8ax8ye/PNBCdC feg3n3TWkyjRYVkd7vcjzI4hEX1WAkrWggdUSD9uXpkxXIrkf+h5BCSApjriMx3j QZJfaxeMq+xItzQyGEIEMDTodTmLrQjCeYyeNlCQLht1ntbiUU4cOWUpqCulm/Q1 UOhMkl4p6MDEkjpfZ9OsiuZp1SGVg67OZ1oR3xzHupCZR0+mVQz0nsE1Af8W8hoN J8n3jouI7D0yZCf4BpMsOSOebykTunCnA8p2TUBQP1A1PhLIj5uwBJ0KFR5xnlTl F9jKQWkFJD3IsMOTOFprw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudefvddggeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkgggtugfgjgesthekredttddtjeenucfhrhhomheptehlvhgr rhhoucfjvghrrhgvrhgruceorghlvhhhvghrrhgvsegrlhhvhhdrnhhoqdhiphdrohhrgh eqnecuggftrfgrthhtvghrnhepvdektdffudfftdffffehfffhjeejhffgieeuueekjeek fffgudffhfduffffueevnecuffhomhgrihhnpegvnhhtvghrphhrihhsvggusgdrtghomh enucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpegrlhhv hhgvrhhrvgesrghlvhhhrdhnohdqihhprdhorhhg X-ME-Proxy: Feedback-ID: ia2694551:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 14 Nov 2023 08:12:52 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=alvh.no-ip.org; s=schmee; t=1699967571; bh=AywaaFfmsbELjRei3oKw5sZrk+qAo2vYaAPLqcWeZXY=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=L/F7X3/Kz8MXotze3qluyHO256b91hfHRuzKX3NDf5qCHsNa6kY0xygKYZjg7oktZ iy7FWLKrKFWi5NGfruApoAyeRslErdafpFtvcEVde0RHP2zC8dS2/po5XDraNGY2/H EbymYA9R16/uSMfymkFwaEnemL9J8cixhKy8aMD/9i+N/a83rnfNnJmxDgqz3e9mTk uq1oH/Vnrz/8DjnLUUfjFJ0q6tJbiGj5+2oMV/3taxcT9+/K8T7hqsr+Eygy/cPtjA PQgqX4nWfK8/fFgZZfGN6ujLutHD6919XGVUSNFROZa8MJq1qV1Y094JvjQUu7wITQ lh9OeZxLIRrKQ== Received: by perhan.alvh.no-ip.org (Postfix, from userid 1000) id 82E41DE; Tue, 14 Nov 2023 14:12:51 +0100 (CET) Date: Tue, 14 Nov 2023 14:12:51 +0100 From: Alvaro Herrera To: Robert Haas Cc: Andres Freund , Peter Eisentraut , Dilip Kumar , "pgsql-hackers@postgresql.org" Subject: Re: trying again to get incremental backup Message-ID: <202311141312.u4qx5gtpvfq3@alvherre.pgsql> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk 0001 looks OK to push, and since it stands on its own I would get it out of the way soon rather than waiting for the rest of the series to be further reviewed. 0002: This moves bin/pg_verifybackup/parse_manifest.c to common/parse_manifest.c, where it's not clear that it's for backup manifests (wasn't a problem in the previous location). I wonder if we're going to have anything else called "manifest", in which case I propose to rename the file to make it clear that this is about backup manifests -- maybe parse_bkp_manifest.c. This patch looks pretty uncontroversial, but there's no point in going further with this one until followup patches are closer to commit. 0003: AmWalSummarizerProcess() is unused. Remove? MaybeWalSummarizer() is called on each ServerLoop() in postmaster.c? This causes a function call to be emitted every time through. That looks odd. All other process starts have some triggering condition. GetOldestUnsummarizedLSN uses while(true), but WaitForWalSummarization and SummarizeWAL use while(1). Maybe settle on one style? Still reading this one. 0004: in PrepareForIncrementalBackup(), the logic that determines earliest_wal_range_tli and latest_wal_range_tli looks pretty weird. I think it works fine if there's a single timeline, but not otherwise. Or maybe the trick is that it relies on timelines returned by readTimeLineHistory being sorted backwards? If so, maybe add a comment about that somewhere; I don't think other callers of readTimeLineHistory make that assumption. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "Postgres is bloatware by design: it was built to house PhD theses." (Joey Hellerstein, SIGMOD annual conference 2002)