Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vVQ4w-002kIZ-10 for pgsql-hackers@arkaria.postgresql.org; Tue, 16 Dec 2025 08:06:43 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vVQ4v-005E5O-0l for pgsql-hackers@arkaria.postgresql.org; Tue, 16 Dec 2025 08:06:42 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vVQ4u-005E5F-2j for pgsql-hackers@lists.postgresql.org; Tue, 16 Dec 2025 08:06:41 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1vVQ4s-0010j7-30 for pgsql-hackers@lists.postgresql.org; Tue, 16 Dec 2025 08:06:41 +0000 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-7b22ffa2a88so4004571b3a.1 for ; Tue, 16 Dec 2025 00:06:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765872397; x=1766477197; darn=lists.postgresql.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=X1n/HC+DiHLm33yZBiz9uxxhuk4FGj/GCs1AF60saxo=; b=WOpEma+tngo0rzmnPREoxkLUn/meHBANYMWyprjGlJf9XqXJnZZocvDhRlHwXBs4g1 IfEATS5DwxIwDOKU4S6C5qPIifCjvE7tSfiksUfaR+sSH8LOynagKdsl+BSYAKDrFe7l ZBNDO/e5bjIRTCUDZQkgkhGIRLVjqOnNhALmNdZjZp+Dr2VU9cYLrG7p+sueAANlDLaJ P8h6pap+MdDCufkBEFZ/vixxZi1VSDIGgKQnNxko6LPzzWk6RZwdkcNbzsJ0OOT3R34t RuMt8FcBPNIcbFK09QCTNvdVYj4fdShrs6edlcCQXZd9hPLvtC97eBgF2R3/tLoD9/zZ Io1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765872397; x=1766477197; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-gg:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=X1n/HC+DiHLm33yZBiz9uxxhuk4FGj/GCs1AF60saxo=; b=bqJOZ2yN/bx2EDvts0ed3jAjSt1u1KM5xyxuYaD6WzLuzRI1SDcjB2Oe+LUOS0ylZ0 d+pMiafe4ryCFvO3K5iNm2Qx0lc8HjZylJulTUEaYG9LPOiY/gCJRqAAhQq0+duAPup4 5goM0bcRIsoYsVkRUKgZW7HzRfgid9bYxK+xCAaikpByAsay/b9nt2vZu+TbwX/mJrOo +2xLN5oz7LqTZ5JV/CIM0ADKWBwbeEHPQyvrj87r6Vg2288VabqI/uWISUQW+pQBdk+C zE98LugxIkm+d3yysxzwvnVJRMFWdqtk8gTWx6bUjhHQXOmiMVuiTzGkq7BT2BBOiDTm l4HA== X-Forwarded-Encrypted: i=1; AJvYcCVNXU2M50id2CzjjBB+CHLyCvzbIMmi7VPrLGrNtlkRS/66hXUyeAjj7dtEsvoU0z9TYlZ5LP5fRcnUhz0d@lists.postgresql.org X-Gm-Message-State: AOJu0YwP5OOVX5EuMFcREeLzVocAWnUZnpCYKV2567Qqeu1ZeGWAyi4g 4CJSYf1QqiycMCEivrto54Kdutsauh2o85sCfs2XSlzkEn8r4tN1enX4eR8X+15eyrg= X-Gm-Gg: AY/fxX4tI+M2glIEtpL8L+LX7+qjb6aVJ787B/by2kG3t616bbALcd50FNFon+0JgYF QMP9VFO9EZBNS3CNGPSKEMNfhbkvkBUhkk9nzGTfmwb6+S6K24RdxXFs4pJEfbr3jHtvWMMr/7L KVMWc1ZGmTI/dbyz5PrPbFFmflk/DCmZwg30FhHbnKuEmYYx6sP99eMn5UDuPm42x2Ak0ibn9np bqhrW0MxRJNgjBn3Y/VqCjYLoeQl/DOYyFID0EueIwhomuPzUzS6KXfSHrXXm8ty+9Pt03aprsZ LBaUddpXm/4//qAFDVsElnfAuLuVPGn9qqVt7KksAVGSqRJk88qMJIZgR5XYygbv2t9TbJjNG1M e2/HxWJSAttPKOP5if1GAQSfplx0WguJ+6WkKoQRVDYIil5l98HKqxqoFb8E3ykwGv3URm3Jf1b LvkErr41r8I7xvPIJN8nQ= X-Google-Smtp-Source: AGHT+IH61xpb//Ii0GjeqTRYv2zVLr0lcHgc+TUkDlO3GTcBEIhZqyTzadUcuo7d27tVMt+2Ai2+Yw== X-Received: by 2002:a05:6a21:e097:b0:334:96ed:7a75 with SMTP id adf61e73a8af0-369ae0abdafmr13172165637.24.1765872396519; Tue, 16 Dec 2025 00:06:36 -0800 (PST) Received: from smtpclient.apple ([45.32.121.103]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c0c2c582e86sm14409837a12.33.2025.12.16.00.06.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Dec 2025 00:06:35 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables From: Chao Li In-Reply-To: Date: Tue, 16 Dec 2025 16:05:56 +0800 Cc: Amul Sul , Oleg Tkachenko , pgsql-hackers@lists.postgresql.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <00FEFC88-EA1D-4271-B38F-EB741733A84A@gmail.com> To: Robert Haas X-Mailer: Apple Mail (2.3826.700.81) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk > On Dec 16, 2025, at 00:35, Robert Haas wrote: >=20 > [ sorry for not noticing this thread sooner; thanks to Andres for > pointing me to it ] >=20 > On Mon, Dec 15, 2025 at 9:01=E2=80=AFAM Amul Sul = wrote: >> Thanks for the reproducer; I can see the reported issue, but I am not >> quite sure the proposed fix is correct and might break other cases (I >> haven't tried constructed that case yet) but there is a comment >> detailing that case just before the point where you are planning to = do >> the changes: >>=20 >> /* >> * The truncation block length is the minimum length of the = reconstructed >> * file. Any block numbers below this threshold that are not = present in >> * the backup need to be fetched from the prior backup. At or = above this >> * threshold, blocks should only be included in the result if they = are >> * present in the backup. (This may require inserting zero blocks = if the >> * blocks included in the backup are non-consecutive.) >> */ >>=20 >> IIUC, we might need the original assignment logic as it is. But we >> need to ensure that truncation_block_length is not set to a value = that >> exceeds RELSEG_SIZE. >=20 > I think you're right. By way of example, let's say that the current > length of the file is 200 blocks, but the limit block is 100 blocks > into the current segment. That means that the only blocks that we can > get from any previous backup are blocks 0-99. Blocks 100-199 of the > current segment are either mentioned in the WAL summaries we're using > for this backup, or they're all zeroes. We can't set the > truncation_block_length to a value greater than 100, or we'll go > looking for the contents of any zero-filled blocks in previous > backups, will will either fail or produce the wrong answer. But Oleg > is correct that we also shouldn't set it to a value greater than > RELSEG_SIZE. So my guess is that the correct fix might be something > like the attached (untested, for discussion). >=20 > --=20 > Robert Haas > EDB: http://www.enterprisedb.com > The change looks good to me. Only nitpick is: ``` Subject: [PATCH v1] Don't set the truncation_block_length rather than RELSEG_SIZE. ``` I guess you meant to say =E2=80=9Clarger (or greater) than=E2=80=9D = instead of =E2=80=9Crather than=E2=80=9D. Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/