pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feed[BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
19+ messages / 5 participants
[nested] [flat]
Commits from this thread:
2026-03-09 ffc226ab64 9540c0e5dd 076bc57fa4 Prevent restore of incremental backup from bloating VM fork.
2026-01-26 4cbaf4dcd2 7903377d9d cf3170ff31 Reduce length of TAP test file name.
2026-01-19 ecd275718b c80b0c9d63 ad569b54a1 Don't set the truncation block length greater than RELSEG_SIZE.
* [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
@ 2025-11-14 19:43 Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Oleg Tkachenko @ 2025-11-14 19:43 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org
Hello PostgreSQL developers,
I’ve encountered a bug in the incremental backup feature that prevents restoration of backups containing relations larger than 1 GB that were vacuum-truncated.
Problem Description
When taking incremental backups of relations that span multiple segments, if the relation is truncated during VACUUM (after the base backup but before the incremental one), pg_combinebackup fails with:
```
file "%s" has truncation block length %u in excess of segment size %u
```
pg_basebackup itself completes without errors, but the resulting incremental backup cannot be restored.
Root Cause
In segmented relations, a VACUUM that truncates blocks sets a limit_block in the WAL summary. The incremental restore logic miscalculates truncation_block_length when processing segment 0…N, because it compares the segment-local size with a relation-wide limit.
In src/backend/backup/basebackup_incremental.c:
```
*truncation_block_length = size / BLCKSZ;
if (BlockNumberIsValid(limit_block))
{
unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
if (*truncation_block_length < relative_limit) /* ← problematic */
*truncation_block_length = relative_limit;
}
```
For example, if limit_block lies in segment 10, then relative_limit will be roughly 9 * RELSEG_SIZE while processing segment 0. This forces truncation_block_length far beyond the actual segment size, leading to a segment length larger than RELSEG_SIZE and eventually the restore error.
Reproduction Steps
Create a table larger than 1 GB (multiple segments).
Take a full base backup.
Delete rows that occupy the end of the relation.
Run VACUUM (VERBOSE, TRUNCATE) to ensure blocks are removed.
(optional) Confirm that the WAL summary includes a limit entry for the relation.
Take an incremental backup with pg_basebackup.
Attempt to restore using pg_combinebackup.
Observe the truncation block length error.
Patch
A patch correcting this logic is attached, and I’m happy to provide additional details or revisions if helpful.
Best regards,
Oleg Tkachenko

Attachments:
bug_truncation_block_length.patch (application/octet-stream, 548B) download
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index ceec1f44312..7c8138e0386 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -854,7 +854,7 @@ GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,
{
unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
- if (*truncation_block_length < relative_limit)
+ if (*truncation_block_length > relative_limit)
*truncation_block_length = relative_limit;
}
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
@ 2025-12-15 11:09 ` Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Oleg Tkachenko @ 2025-12-15 11:09 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org
Hello,
I am following up on my report with a patch, which did not receive any responses. I suspect the issue only manifests under specific conditions, so I am sending additional details along with a reliable reproducer.
For context:
Incremental backups cannot be restored when a relation larger than 1 GB (multiple segments) is vacuum-truncated between the base backup and the incremental backup (pg_basebackup itself completes successfully, but the resulting incremental backup is not restorable)
For segmented relations, the WAL summarizer records limit_block in the WAL summary. During incremental backup, the truncation length is computed incorrectly because a relation-wide limit is compared against a segment-local size.
Reproducer
I am attaching a bash script that reliably reproduces the issue on my system. The script:
Creates a table large enough to span multiple segments.
Takes a full base backup.
Deletes rows at the end of the relation.
Runs VACUUM (TRUNCATE) to remove blocks.
Takes an incremental backup.
Fails during pg_combinebackup.
The script is fully automated and intended to be run as-is.
A patch itself in the previous message.
I would appreciate feedback on the approach and am happy to revise it if needed.
Best regards,
Oleg Tkachenko

> On Nov 14, 2025, at 20:43, Oleg Tkachenko <oatkachenko@gmail.com> wrote:
>
> Hello PostgreSQL developers,
>
> I’ve encountered a bug in the incremental backup feature that prevents restoration of backups containing relations larger than 1 GB that were vacuum-truncated.
>
>
>
> Problem Description
>
> When taking incremental backups of relations that span multiple segments, if the relation is truncated during VACUUM (after the base backup but before the incremental one), pg_combinebackup fails with:
>
> ```
>
> file "%s" has truncation block length %u in excess of segment size %u
>
> ```
>
> pg_basebackup itself completes without errors, but the resulting incremental backup cannot be restored.
>
>
>
> Root Cause
>
> In segmented relations, a VACUUM that truncates blocks sets a limit_block in the WAL summary. The incremental restore logic miscalculates truncation_block_length when processing segment 0…N, because it compares the segment-local size with a relation-wide limit.
>
> In src/backend/backup/basebackup_incremental.c:
>
> ```
>
> *truncation_block_length = size / BLCKSZ;
>
> if (BlockNumberIsValid(limit_block))
>
> {
>
> unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
>
> if (*truncation_block_length < relative_limit) /* ← problematic */
>
> *truncation_block_length = relative_limit;
>
> }
>
> ```
>
> For example, if limit_block lies in segment 10, then relative_limit will be roughly 9 * RELSEG_SIZE while processing segment 0. This forces truncation_block_length far beyond the actual segment size, leading to a segment length larger than RELSEG_SIZE and eventually the restore error.
>
>
>
> Reproduction Steps
>
>
> Create a table larger than 1 GB (multiple segments).
>
> Take a full base backup.
>
> Delete rows that occupy the end of the relation.
>
> Run VACUUM (VERBOSE, TRUNCATE) to ensure blocks are removed.
>
> (optional) Confirm that the WAL summary includes a limit entry for the relation.
>
> Take an incremental backup with pg_basebackup.
>
> Attempt to restore using pg_combinebackup.
>
> Observe the truncation block length error.
>
>
>
>
> Patch
>
> A patch correcting this logic is attached, and I’m happy to provide additional details or revisions if helpful.
>
>
>
> Best regards,
>
> Oleg Tkachenko
>
>
> <bug_truncation_block_length.patch>
>
>
>
>
>
Attachments:
reproducer.zip (application/zip, 2.5K) download
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
@ 2025-12-15 14:01 ` Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Amul Sul @ 2025-12-15 14:01 UTC (permalink / raw)
To: Oleg Tkachenko <oatkachenko@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org
On Mon, Dec 15, 2025 at 4:39 PM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
>
> [....]
>
> A patch correcting this logic is attached, and I’m happy to provide additional details or revisions if helpful.
>
Thanks for the reproducer; I can see the reported issue, but I am not
quite sure the proposed fix is correct and might break other cases (I
haven't tried constructed that case yet) but there is a comment
detailing that case just before the point where you are planning to do
the changes:
/*
* The truncation block length is the minimum length of the reconstructed
* file. Any block numbers below this threshold that are not present in
* the backup need to be fetched from the prior backup. At or above this
* threshold, blocks should only be included in the result if they are
* present in the backup. (This may require inserting zero blocks if the
* blocks included in the backup are non-consecutive.)
*/
IIUC, we might need the original assignment logic as it is. But we
need to ensure that truncation_block_length is not set to a value that
exceeds RELSEG_SIZE.
Regards,
Amul
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
@ 2025-12-15 16:35 ` Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-16 08:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Chao Li <li.evan.chao@gmail.com>
0 siblings, 2 replies; 19+ messages in thread
From: Robert Haas @ 2025-12-15 16:35 UTC (permalink / raw)
To: Amul Sul <sulamul@gmail.com>; +Cc: Oleg Tkachenko <oatkachenko@gmail.com>; pgsql-hackers@lists.postgresql.org
[ sorry for not noticing this thread sooner; thanks to Andres for
pointing me to it ]
On Mon, Dec 15, 2025 at 9:01 AM Amul Sul <sulamul@gmail.com> wrote:
> Thanks for the reproducer; I can see the reported issue, but I am not
> quite sure the proposed fix is correct and might break other cases (I
> haven't tried constructed that case yet) but there is a comment
> detailing that case just before the point where you are planning to do
> the changes:
>
> /*
> * The truncation block length is the minimum length of the reconstructed
> * file. Any block numbers below this threshold that are not present in
> * the backup need to be fetched from the prior backup. At or above this
> * threshold, blocks should only be included in the result if they are
> * present in the backup. (This may require inserting zero blocks if the
> * blocks included in the backup are non-consecutive.)
> */
>
> IIUC, we might need the original assignment logic as it is. But we
> need to ensure that truncation_block_length is not set to a value that
> exceeds RELSEG_SIZE.
I think you're right. By way of example, let's say that the current
length of the file is 200 blocks, but the limit block is 100 blocks
into the current segment. That means that the only blocks that we can
get from any previous backup are blocks 0-99. Blocks 100-199 of the
current segment are either mentioned in the WAL summaries we're using
for this backup, or they're all zeroes. We can't set the
truncation_block_length to a value greater than 100, or we'll go
looking for the contents of any zero-filled blocks in previous
backups, will will either fail or produce the wrong answer. But Oleg
is correct that we also shouldn't set it to a value greater than
RELSEG_SIZE. So my guess is that the correct fix might be something
like the attached (untested, for discussion).
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
v1-0001-Don-t-set-the-truncation_block_length-rather-than.patch (application/octet-stream, 1.4K) download
From bac3a98cd4e4b3007e140d9db334b040cf6772c5 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 15 Dec 2025 11:34:34 -0500
Subject: [PATCH v1] Don't set the truncation_block_length rather than
RELSEG_SIZE.
Also, add some comments to explain the logic.
---
src/backend/backup/basebackup_incremental.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index 7678e4f6ec3..f4ef33f6672 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -850,8 +850,22 @@ GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,
{
unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
+ /*
+ * We can't set a truncation_block_length in excess of the limit block
+ * number (relativized to the current segment). To do so would be to
+ * treat blocks from older backups as valid current contents even if
+ * they were subsequently truncated away.
+ */
if (*truncation_block_length < relative_limit)
*truncation_block_length = relative_limit;
+
+ /*
+ * We also can't set a truncation_block_length in excess of the
+ * segment size, since the reconstructed file can't be larger than
+ * that.
+ */
+ if (*truncation_block_length > RELSEG_SIZE)
+ *truncation_block_length = RELSEG_SIZE;
}
/* Send it incrementally. */
--
2.51.0
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
@ 2025-12-15 18:46 ` Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 19+ messages in thread
From: Oleg Tkachenko @ 2025-12-15 18:46 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
Hello Robert,
Thank you for the explanation.
At first, I also thought about clamping truncation_block_length to RELSEG_SIZE. But I hesitated because I thought the reconstructed relation file couldn’t be larger than relative_limit.
After reading the reconstruction code and the comments on top of the discussed block of code (many times), I finally understood that truncation_block_length is the minimum length of the reconstructed file, not just a safety limit. It determines which blocks must be fetched from older backups. So a simple clamp could change how reconstruction works if some blocks are included in incremental backups.
I’ve tested the version with the limit enforced to RELSEG_SIZE, and it works correctly.
Also, I’ve attached a patch based on your guidance. The changes are effectively the same as your suggested approach, but I would be happy to be listed as a contributor.
Regards,
Oleg Tkachenko

> On Dec 15, 2025, at 17:35, Robert Haas <robertmhaas@gmail.com> wrote:
>
> [ sorry for not noticing this thread sooner; thanks to Andres for
> pointing me to it ]
>
> On Mon, Dec 15, 2025 at 9:01 AM Amul Sul <sulamul@gmail.com> wrote:
>> Thanks for the reproducer; I can see the reported issue, but I am not
>> quite sure the proposed fix is correct and might break other cases (I
>> haven't tried constructed that case yet) but there is a comment
>> detailing that case just before the point where you are planning to do
>> the changes:
>>
>> /*
>> * The truncation block length is the minimum length of the reconstructed
>> * file. Any block numbers below this threshold that are not present in
>> * the backup need to be fetched from the prior backup. At or above this
>> * threshold, blocks should only be included in the result if they are
>> * present in the backup. (This may require inserting zero blocks if the
>> * blocks included in the backup are non-consecutive.)
>> */
>>
>> IIUC, we might need the original assignment logic as it is. But we
>> need to ensure that truncation_block_length is not set to a value that
>> exceeds RELSEG_SIZE.
>
> I think you're right. By way of example, let's say that the current
> length of the file is 200 blocks, but the limit block is 100 blocks
> into the current segment. That means that the only blocks that we can
> get from any previous backup are blocks 0-99. Blocks 100-199 of the
> current segment are either mentioned in the WAL summaries we're using
> for this backup, or they're all zeroes. We can't set the
> truncation_block_length to a value greater than 100, or we'll go
> looking for the contents of any zero-filled blocks in previous
> backups, will will either fail or produce the wrong answer. But Oleg
> is correct that we also shouldn't set it to a value greater than
> RELSEG_SIZE. So my guess is that the correct fix might be something
> like the attached (untested, for discussion).
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com

Attachments: download all (2 files, .tar.gz)
0001-Clamp-truncation_block_length-to-RELSEG_SIZE.patch (application/octet-stream, 1.5K) download
From f8ab0228c1125e07e81b7f1aeaee6e5af2370b21 Mon Sep 17 00:00:00 2001
From: Oleg Tkachenko <oleg@elsci.io>
Date: Mon, 15 Dec 2025 19:29:50 +0100
Subject: [PATCH] Clamp truncation_block_length to RELSEG_SIZE
Fix an issue where truncation_block_length could exceed RELSEG_SIZE,
causing restore failures when reconstructing incremental backups.
---
src/backend/backup/basebackup_incremental.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index 7678e4f6ec3..f4ef33f6672 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -850,8 +850,22 @@ GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,
{
unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
+ /*
+ * We can't set a truncation_block_length in excess of the limit block
+ * number (relativized to the current segment). To do so would be to
+ * treat blocks from older backups as valid current contents even if
+ * they were subsequently truncated away.
+ */
if (*truncation_block_length < relative_limit)
*truncation_block_length = relative_limit;
+
+ /*
+ * We also can't set a truncation_block_length in excess of the
+ * segment size, since the reconstructed file can't be larger than
+ * that.
+ */
+ if (*truncation_block_length > RELSEG_SIZE)
+ *truncation_block_length = RELSEG_SIZE;
}
/* Send it incrementally. */
--
2.52.0
=
v1-0001-Don-t-set-the-truncation_block_length-rather-than.patch (application/octet-stream, 1.4K) download
From bac3a98cd4e4b3007e140d9db334b040cf6772c5 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 15 Dec 2025 11:34:34 -0500
Subject: [PATCH v1] Don't set the truncation_block_length rather than
RELSEG_SIZE.
Also, add some comments to explain the logic.
---
src/backend/backup/basebackup_incremental.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index 7678e4f6ec3..f4ef33f6672 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -850,8 +850,22 @@ GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,
{
unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
+ /*
+ * We can't set a truncation_block_length in excess of the limit block
+ * number (relativized to the current segment). To do so would be to
+ * treat blocks from older backups as valid current contents even if
+ * they were subsequently truncated away.
+ */
if (*truncation_block_length < relative_limit)
*truncation_block_length = relative_limit;
+
+ /*
+ * We also can't set a truncation_block_length in excess of the
+ * segment size, since the reconstructed file can't be larger than
+ * that.
+ */
+ if (*truncation_block_length > RELSEG_SIZE)
+ *truncation_block_length = RELSEG_SIZE;
}
/* Send it incrementally. */
--
2.51.0
=
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
@ 2025-12-15 20:13 ` Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Robert Haas @ 2025-12-15 20:13 UTC (permalink / raw)
To: Oleg Tkachenko <oatkachenko@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
On Mon, Dec 15, 2025 at 1:46 PM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
> Also, I’ve attached a patch based on your guidance. The changes are effectively the same as your suggested approach, but I would be happy to be listed as a contributor.
You'll certain be listed as the reporter for this issue when a fix is
committed. If you want to be listed as a co-author of the patch, I
think it is fair to say that it will need to contain some code written
by you. For example, maybe you would like to try writing a TAP test
case that fails without this fix and passes with it.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
@ 2025-12-16 21:55 ` Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Oleg Tkachenko @ 2025-12-16 21:55 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
Hello, Robert
I’ve created a small test that reproduces the issue. With the proposed fix applied, the test passes, and the reconstruction behaves as expected.
I’m attaching the test for review. Please let me know if this looks OK or if you would like it changed.
Regards,
Oleg

> On Dec 15, 2025, at 21:13, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Mon, Dec 15, 2025 at 1:46 PM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
>> Also, I’ve attached a patch based on your guidance. The changes are effectively the same as your suggested approach, but I would be happy to be listed as a contributor.
>
> You'll certain be listed as the reporter for this issue when a fix is
> committed. If you want to be listed as a co-author of the patch, I
> think it is fair to say that it will need to contain some code written
> by you. For example, maybe you would like to try writing a TAP test
> case that fails without this fix and passes with it.
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
Attachments:
0001-Test-the-correctness-of-the-truncation_block_length-.patch (application/octet-stream, 3.4K) download
From ad3ad6bb276bbac0e527d20d1943174667e971b9 Mon Sep 17 00:00:00 2001
From: Oleg Tkachenko <oleg@elsci.io>
Date: Tue, 16 Dec 2025 21:51:21 +0100
Subject: [PATCH] Test the correctness of the truncation_block_length value in
incremental files.
---
...050_incremental_backup_truncation_block.pl | 69 +++++++++++++++++++
1 file changed, 69 insertions(+)
create mode 100644 src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl
diff --git a/src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl b/src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl
new file mode 100644
index 00000000000..9b7a7b490e6
--- /dev/null
+++ b/src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl
@@ -0,0 +1,69 @@
+# Test case for a scenario where truncation_block is miscalculated during an incremental backup
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Initialize primary node
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init(has_archiving => 1, allows_streaming => 1);
+# Enable WAL summarization to support incremental backup
+$node->append_conf('postgresql.conf', 'summarize_wal = on');
+$node->start;
+
+# Backup locations
+my $backup_path = $node->backup_dir;
+my $full_backup = "$backup_path/full";
+
+# Create a table with wide rows so each tuple occupies almost a full page.
+# STORAGE PLAIN disables compression, allowing simple, repetitive data to be used without worrying about TOAST size.
+$node->safe_psql('postgres', q{
+ CREATE TABLE t (
+ id int,
+ data text STORAGE PLAIN
+ );
+});
+
+# Each tuple occupies almost a full page; having 135k pages guarantees that the relation spans two segments.
+$node->safe_psql('postgres', q{
+ INSERT INTO t
+ SELECT i, repeat('x', 8000)
+ FROM generate_series(1, 135000) i;
+});
+
+# This step is required because at this moment, tuples do not have hint bits set.
+# Later (for example, soon after the base backup is created), a background process may set hint bits
+# on many tuples and change many heap pages.
+# Because of this, the WAL summary may show that too many pages were changed and create a full file copy
+# instead of an incremental one, which makes the issue unproducible.
+$node->safe_psql('postgres', 'VACUUM t;');
+
+# Take a full base backup
+$node->backup('full');
+
+# Delete rows at the logical end of the table. This creates removable empty pages at the tail
+$node->safe_psql('postgres', 'DELETE FROM t WHERE id > 134900;');
+
+# VACUUM with truncation to physically remove trailing empty pages
+$node->safe_psql('postgres', 'VACUUM (TRUNCATE) t;');
+
+# Take an incremental backup based on the full backup manifest
+$node->backup('incr', backup_options => ['--incremental', "$full_backup/backup_manifest"]);
+
+# Combine full and incremental backups.
+# This step must correctly handle truncated relation segments.
+# Before the fix, this failed because the INCREMENTAL file header contained an incorrect truncation_block value.
+my $restored = PostgreSQL::Test::Cluster->new('node2');
+$restored->init_from_backup($node, 'incr', combine_with_prior => ['full']);
+$restored->start();
+
+# Check that the restored table contains the correct number of rows
+my $restored_count = $restored->safe_psql('postgres', "SELECT count(*) FROM t;");
+is($restored_count, '134900', 'Restored backup has correct row count');
+
+$node->stop;
+$restored->stop;
+
+done_testing();
--
2.52.0
=
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
@ 2025-12-18 06:05 ` Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Amul Sul @ 2025-12-18 06:05 UTC (permalink / raw)
To: Oleg Tkachenko <oatkachenko@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
On Wed, Dec 17, 2025 at 3:25 AM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
>
> Hello, Robert
>
> I’ve created a small test that reproduces the issue. With the proposed fix applied, the test passes, and the reconstruction behaves as expected.
>
> I’m attaching the test for review. Please let me know if this looks OK or if you would like it changed.
>
Test looks good to me, but I have three suggestions as follow:
1. To minimize repetition in insert: use fillfactor 10, which is the
minimal we can set for a table, so that we can minimize tuples per
page. Use a longer string and lower count in repeat(), which I believe
helps the test become a bit faster.
2. I think we could add this test to the existing pg_combinebackup's
test file instead of creating a new file with a single-test. See the
attached version; it’s a bit smaller than your original patch, but
since I haven't copied all of your comments yet, I’ve marked it as
WIP.
3. Kindly combine the code fix and tests together into a single patch.
Regards,
Amul
Attachments:
WIP-add-tests-to-pg_combinebackup.patch (application/octet-stream, 1.6K) download
commit 6cf12211a62de32762e173f74a4e09cfe9b2d842
Author: Amul Sul <sulamul@gmail.com>
Date: Thu Dec 18 11:09:44 2025 +0530
WIP2
diff --git a/src/bin/pg_combinebackup/t/002_compare_backups.pl b/src/bin/pg_combinebackup/t/002_compare_backups.pl
index a3e29c05509..d539a0ca511 100644
--- a/src/bin/pg_combinebackup/t/002_compare_backups.pl
+++ b/src/bin/pg_combinebackup/t/002_compare_backups.pl
@@ -45,6 +45,17 @@ CREATE TABLE will_change_in_ts (a int, b text) TABLESPACE ts1;
INSERT INTO will_change_in_ts VALUES (1, 'initial test row');
CREATE TABLE will_get_dropped_in_ts (a int, b text);
INSERT INTO will_get_dropped_in_ts VALUES (1, 'initial test row');
+--
+-- Create a table with minimal fillfactor to minimize tuples per page.
+-- STORAGE PLAIN prevents compression and TOASTing of repetitive data,
+-- ensuring predictable row sizes that target 1 row per page.
+-- At 8KB page size, 135k pages forces the relation to exceed 1GB,
+-- ensuring it spans at least two physical storage segments.
+--
+CREATE TABLE big_tab (a int, b text) WITH (FILLFACTOR = 10);
+INSERT INTO big_tab SELECT i, repeat('x1234567890x0987654321x', 20)
+FROM generate_series(1, 135000) i;
+VACUUM big_tab;
EOM
# Read list of tablespace OIDs. There should be just one.
@@ -83,6 +94,12 @@ INSERT INTO newly_created_in_ts VALUES (1, 'row for new table');
VACUUM FULL will_get_rewritten;
DROP DATABASE db_will_get_dropped;
CREATE DATABASE db_newly_created;
+--
+-- Delete rows at the logical end of the table. This creates removable empty
+-- pages at the tail
+--
+DELETE FROM big_tab WHERE a > 134900;
+VACUUM (TRUNCATE) big_tab;
EOM
# Take an incremental backup.
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
@ 2025-12-18 14:26 ` Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Robert Haas @ 2025-12-18 14:26 UTC (permalink / raw)
To: Amul Sul <sulamul@gmail.com>; +Cc: Oleg Tkachenko <oatkachenko@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
On Thu, Dec 18, 2025 at 1:05 AM Amul Sul <sulamul@gmail.com> wrote:
> Test looks good to me, but I have three suggestions as follow:
>
> 1. To minimize repetition in insert: use fillfactor 10, which is the
> minimal we can set for a table, so that we can minimize tuples per
> page. Use a longer string and lower count in repeat(), which I believe
> helps the test become a bit faster.
I haven't checked how big a relation the test case creates, but it's
worth keeping in mind that the CI tests run on one platform with the
segment size set to six blocks. I think we should design the test case
with that in mind i.e. don't worry about catching the bug when the
segment size is 1GB, but make sure the test fails in CI without the
bug fix. Let's not rely on fillfactor -- the cost here is the disk
space and the time to write the blocks, not how many tuples they
actually contain.
> 2. I think we could add this test to the existing pg_combinebackup's
> test file instead of creating a new file with a single-test. See the
> attached version; it’s a bit smaller than your original patch, but
> since I haven't copied all of your comments yet, I’ve marked it as
> WIP.
-1. This kind of thing tends to make the tests harder to understand.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
@ 2025-12-18 17:23 ` Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Oleg Tkachenko @ 2025-12-18 17:23 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
Hi,
Here is a refactored test.
Now, it creates data depending on the relation block size, so it works even if the segment size is not standard. I tested it locally with segment_size_blocks = 6, and it works correctly.
I would be happy to hear your comments or suggestions.
Regards,
Oleg

> On Dec 18, 2025, at 15:26, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 1:05 AM Amul Sul <sulamul@gmail.com> wrote:
>> Test looks good to me, but I have three suggestions as follow:
>>
>> 1. To minimize repetition in insert: use fillfactor 10, which is the
>> minimal we can set for a table, so that we can minimize tuples per
>> page. Use a longer string and lower count in repeat(), which I believe
>> helps the test become a bit faster.
>
> I haven't checked how big a relation the test case creates, but it's
> worth keeping in mind that the CI tests run on one platform with the
> segment size set to six blocks. I think we should design the test case
> with that in mind i.e. don't worry about catching the bug when the
> segment size is 1GB, but make sure the test fails in CI without the
> bug fix. Let's not rely on fillfactor -- the cost here is the disk
> space and the time to write the blocks, not how many tuples they
> actually contain.
>
>> 2. I think we could add this test to the existing pg_combinebackup's
>> test file instead of creating a new file with a single-test. See the
>> attached version; it’s a bit smaller than your original patch, but
>> since I haven't copied all of your comments yet, I’ve marked it as
>> WIP.
>
> -1. This kind of thing tends to make the tests harder to understand.
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
Attachments:
0001-Clamp-truncation_block_length-to-RELSEG_SIZE-when-se.patch (application/octet-stream, 6.4K) download
From 009ceb2ac252041ff84763deda8675118abb8053 Mon Sep 17 00:00:00 2001
From: Oleg Tkachenko <oleg@elsci.io>
Date: Thu, 18 Dec 2025 18:05:21 +0100
Subject: [PATCH] Clamp truncation_block_length to RELSEG_SIZE when sending
incremental files
- Fix bug where truncation_block_length could exceed RELSEG_SIZE, leading to restore failures.
- Add TAP test to verify that a relation truncated after the last base backup can be safely included in an incremental backup.
This patch follows guidance from Robert Haas.
---
src/backend/backup/basebackup_incremental.c | 14 +++
...050_incremental_backup_truncation_block.pl | 93 +++++++++++++++++++
2 files changed, 107 insertions(+)
create mode 100644 src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index 7678e4f6ec3..f4ef33f6672 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -850,8 +850,22 @@ GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,
{
unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
+ /*
+ * We can't set a truncation_block_length in excess of the limit block
+ * number (relativized to the current segment). To do so would be to
+ * treat blocks from older backups as valid current contents even if
+ * they were subsequently truncated away.
+ */
if (*truncation_block_length < relative_limit)
*truncation_block_length = relative_limit;
+
+ /*
+ * We also can't set a truncation_block_length in excess of the
+ * segment size, since the reconstructed file can't be larger than
+ * that.
+ */
+ if (*truncation_block_length > RELSEG_SIZE)
+ *truncation_block_length = RELSEG_SIZE;
}
/* Send it incrementally. */
diff --git a/src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl b/src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl
new file mode 100644
index 00000000000..e9469d25227
--- /dev/null
+++ b/src/bin/pg_basebackup/t/050_incremental_backup_truncation_block.pl
@@ -0,0 +1,93 @@
+# Test case for a scenario where truncation_block is miscalculated during an incremental backup
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Initialize primary node
+my $primary = PostgreSQL::Test::Cluster->new('primary');
+$primary->init(has_archiving => 1, allows_streaming => 1);
+# Enable WAL summarization to support incremental backup
+$primary->append_conf('postgresql.conf', 'summarize_wal = on');
+$primary->start;
+
+# Backup locations
+my $backup_path = $primary->backup_dir;
+my $full_backup = "$backup_path/full";
+
+# NB: CI tests run on one platform with the segment size set to six blocks,
+# so we can't rely on any defaults (e.g., segment size = 131072 blocks).
+# Calculate how many rows we need to insert into the table to make it span more than one physical storage segment,
+# sized so that at most one tuple fits on a page.
+my $block_size = $primary->safe_psql('postgres', "SELECT current_setting('block_size')::int;");
+my $segment_size_blocks = $primary->safe_psql('postgres',
+ "SELECT (pg_size_bytes(current_setting('segment_size')) / $block_size)::int;"
+);
+# Two blocks will be located in the second segment, and one block will still stay there after truncation.
+my $target_rows = int($segment_size_blocks + 2);
+my $rows_after_truncation = int($target_rows - 1);
+
+# Create a test table.
+# STORAGE PLAIN prevents compression and TOASTing of repetitive data, ensuring predictable row sizes.
+$primary->safe_psql('postgres', q{
+ CREATE TABLE t (
+ id int,
+ data text STORAGE PLAIN
+ );
+});
+
+# The tuple size should be enough to prevent two tuples from being on the same page.
+# Since the template string has a length of 32 bytes, it's enough to repeat it (block_size / (2*32)) times.
+$primary->safe_psql('postgres',
+ "INSERT INTO t
+ SELECT i, repeat('0123456789ABCDEF0123456789ABCDEF', ($block_size / (2*32)))
+ FROM generate_series(1, $target_rows) i;"
+);
+
+# This step is required because at this moment, tuples do not have hint bits set.
+# Later (for example, soon after the base backup is created), a background process may set hint bits
+# on many tuples and change many heap pages.
+# Because of this, the WAL summary may show that too many pages were changed and create a full file copy
+# instead of an incremental one, which makes the issue non-reproducible.
+$primary->safe_psql('postgres', 'VACUUM t;');
+
+# Verify that relation spans more than one physical storage segment
+my $t_blocks = $primary->safe_psql('postgres', "SELECT pg_relation_size('t') / current_setting('block_size')::int;");
+cmp_ok($t_blocks, '>', $segment_size_blocks, 'Relation spans more than one physical segment.');
+
+# Take a full base backup
+$primary->backup('full');
+
+# Delete rows at the logical end of the table. This creates removable empty pages at the tail
+$primary->safe_psql('postgres', "DELETE FROM t WHERE id > ($rows_after_truncation);");
+
+# Although TRUNCATE is enabled by default, here it emphasizes the expected behavior of the operation.
+$primary->safe_psql('postgres', 'VACUUM (TRUNCATE) t;');
+
+# Verify that after VACUUM relation is truncated but still spans more than one physical storage segment.
+$t_blocks = $primary->safe_psql('postgres',
+ "SELECT pg_relation_size('t') / current_setting('block_size')::int;"
+);
+is($t_blocks, $rows_after_truncation, 'Relation has expected size.');
+cmp_ok($t_blocks, '>', $segment_size_blocks, 'Relation spans more than one physical segment.');
+
+# Take an incremental backup based on the full backup manifest
+$primary->backup('incr', backup_options => [ '--incremental', "$full_backup/backup_manifest" ]);
+
+# Combine full and incremental backups.
+# This step must correctly handle truncated relation segments.
+# Before the fix, this failed because the INCREMENTAL file header contained an incorrect truncation_block value.
+my $restored = PostgreSQL::Test::Cluster->new('node2');
+$restored->init_from_backup($primary, 'incr', combine_with_prior => [ 'full' ]);
+$restored->start();
+
+# Check that the restored table contains the correct number of rows
+my $restored_count = $restored->safe_psql('postgres', "SELECT count(*) FROM t;");
+is($restored_count, $rows_after_truncation, 'Restored backup has correct row count');
+
+$primary->stop;
+$restored->stop;
+
+done_testing();
--
2.52.0
=
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
@ 2026-01-05 16:05 ` Robert Haas <robertmhaas@gmail.com>
2026-01-07 14:49 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Robert Haas @ 2026-01-05 16:05 UTC (permalink / raw)
To: Oleg Tkachenko <oatkachenko@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
On Thu, Dec 18, 2025 at 12:24 PM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
> Here is a refactored test.
>
> Now, it creates data depending on the relation block size, so it works even if the segment size is not standard. I tested it locally with segment_size_blocks = 6, and it works correctly.
>
> I would be happy to hear your comments or suggestions.
Hi Oleg,
I have been mostly on vacation since you sent this email, but here I
am back again. I tried running this on CI with and without the actual
code fix, and was pleased to see the CI failed on this test without
the code fix and passed with it. But then I noticed that you hadn't
updated meson.build in src/bin/pg_basebackup for the new test, which
means that the test was only running in configure/make builds and not
in meson/ninja builds. When I fixed that, things didn't look so good.
The test then fails:
pg_combinebackup: reconstructing
"/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_node2_data/pgdata/base/5/16384.1"
(1 blocks, checksum CRC32C)
pg_combinebackup: reconstruction plan:
0:/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/full/base/5/16384.1@0
pg_combinebackup: read 1 blocks from
"/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/full/base/5/16384.1"
pg_combinebackup: reconstructing
"/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_node2_data/pgdata/base/5/16384_vm"
(131072 blocks, checksum CRC32C)
pg_combinebackup: reconstruction plan:
0-3:/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/full/base/5/16384_vm@24576
4:/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/incr/base/5/INCREMENTAL.16384_vm@8192
5-131071:zero
pg_combinebackup: error: could not write file
"/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_node2_data/pgdata/base/5/16384_vm":
No space left on device
I'm not sure what's going on here exactly, but it seems bad. The
output implies that 16384_vm is a full 1GB in size, which doesn't
really make any sense to me at all, but the same thing also happens
when I run the test locally. The VM fork is normally quite small
compared to the data, and here the data is only one block over 1GB, so
I'd expect the VM fork to be just a few blocks. Are we somehow
confusing the length of the VM fork with the length of the main fork?
A couple of stylistic notes: All of the existing incremental backup
tests are in src/bin/pg_combinebackup/t. I suggest putting this one
there too. Normally, our TAP test names are all lower-case, so do the
same here. Try to format the test file so that things fit within 80
columns, by breaking comments and Perl statements at appropriate
points. Consider running src/tools/pgindent/pgperltidy over the script
to check that the way you've broken the Perl statements won't get
reindented.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
@ 2026-01-07 14:49 ` Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-14 15:38 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Oleg Tkachenko @ 2026-01-07 14:49 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
Hello Robert,
I checked the VM fork file and found that its incremental version has a wrong
block number in the header:
```
xxd -l 12 INCREMENTAL.16384_vm
0d1f aed3 0100 0000 0000 0200 <--- 131072 blocks (1 GB)
^^^^ ^^^^
```
This value can only come from the WAL summaries, so I checked them too.
One of the summary files contains:
```
TS 1663, DB 5, REL 16384, FORK main: limit 131073
TS 1663, DB 5, REL 16384, FORK vm: limit 131073
TS 1663, DB 5, REL 16384, FORK vm: block 4
```
Both forks have the same limit, which looks wrong.
So I checked the WAL files to see what really happened with the VM fork.
I did not find any “truncate" records for the VM file.
I only found this record for the main fork
(actually, the fork isn’t mentioned at all):
```
rmgr: Storage len (rec/tot): 46/46, tx: 759, lsn: 0/4600D318,
prev 0/4600B2C8, desc: TRUNCATE base/5/16384 to 131073 blocks flags 7
```
This suggests that the WAL summarizer may be mixing up information between
relation forks.
I also noticed this comment in basebackup_incremental.c:
```
/*
* The free-space map fork is not properly WAL-logged, so we need to
* backup the entire file every time.
*/
if (forknum == FSM_FORKNUM)
return BACK_UP_FILE_FULLY;
```
Maybe we should treat the VM fork the same way and always back it up fully?
Another option is to fix the summarizer so it handles forks correctly.
Best regards,
Oleg
> On Jan 5, 2026, at 17:05, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 12:24 PM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
>> Here is a refactored test.
>>
>> Now, it creates data depending on the relation block size, so it works even if the segment size is not standard. I tested it locally with segment_size_blocks = 6, and it works correctly.
>>
>> I would be happy to hear your comments or suggestions.
>
> Hi Oleg,
>
> I have been mostly on vacation since you sent this email, but here I
> am back again. I tried running this on CI with and without the actual
> code fix, and was pleased to see the CI failed on this test without
> the code fix and passed with it. But then I noticed that you hadn't
> updated meson.build in src/bin/pg_basebackup for the new test, which
> means that the test was only running in configure/make builds and not
> in meson/ninja builds. When I fixed that, things didn't look so good.
> The test then fails:
>
> pg_combinebackup: reconstructing
> "/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_node2_data/pgdata/base/5/16384.1"
> (1 blocks, checksum CRC32C)
> pg_combinebackup: reconstruction plan:
> 0:/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/full/base/5/16384.1@0
> pg_combinebackup: read 1 blocks from
> "/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/full/base/5/16384.1"
> pg_combinebackup: reconstructing
> "/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_node2_data/pgdata/base/5/16384_vm"
> (131072 blocks, checksum CRC32C)
> pg_combinebackup: reconstruction plan:
> 0-3:/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/full/base/5/16384_vm@24576
> 4:/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_primary_data/backup/incr/base/5/INCREMENTAL.16384_vm@8192
> 5-131071:zero
> pg_combinebackup: error: could not write file
> "/tmp/cirrus-ci-build/build/testrun/pg_basebackup/050_incremental_backup_truncation_block/data/t_050_incremental_backup_truncation_block_node2_data/pgdata/base/5/16384_vm":
> No space left on device
>
> I'm not sure what's going on here exactly, but it seems bad. The
> output implies that 16384_vm is a full 1GB in size, which doesn't
> really make any sense to me at all, but the same thing also happens
> when I run the test locally. The VM fork is normally quite small
> compared to the data, and here the data is only one block over 1GB, so
> I'd expect the VM fork to be just a few blocks. Are we somehow
> confusing the length of the VM fork with the length of the main fork?
>
> A couple of stylistic notes: All of the existing incremental backup
> tests are in src/bin/pg_combinebackup/t. I suggest putting this one
> there too. Normally, our TAP test names are all lower-case, so do the
> same here. Try to format the test file so that things fit within 80
> columns, by breaking comments and Perl statements at appropriate
> points. Consider running src/tools/pgindent/pgperltidy over the script
> to check that the way you've broken the Perl statements won't get
> reindented.
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-07 14:49 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
@ 2026-01-14 15:38 ` Robert Haas <robertmhaas@gmail.com>
2026-01-16 12:10 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Robert Haas @ 2026-01-14 15:38 UTC (permalink / raw)
To: Oleg Tkachenko <oatkachenko@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
On Wed, Jan 7, 2026 at 9:50 AM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
> Both forks have the same limit, which looks wrong.
> So I checked the WAL files to see what really happened with the VM fork.
> I did not find any “truncate" records for the VM file.
> I only found this record for the main fork
> (actually, the fork isn’t mentioned at all):
>
> rmgr: Storage len (rec/tot): 46/46, tx: 759, lsn: 0/4600D318,
> prev 0/4600B2C8, desc: TRUNCATE base/5/16384 to 131073 blocks flags 7
Flags 7 for Storage/TRUNCATE means all forks:
#define SMGR_TRUNCATE_HEAP 0x0001
#define SMGR_TRUNCATE_VM 0x0002
#define SMGR_TRUNCATE_FSM 0x0004
#define SMGR_TRUNCATE_ALL \
(SMGR_TRUNCATE_HEAP|SMGR_TRUNCATE_VM|SMGR_TRUNCATE_FSM)
I think this comes from RelationTruncate(), which does indeed set
xlrec.flags = SMGR_TRUNCATE_ALL. It seems bananas to me to use the
same count of blocks for all forks, but it seems that is the way the
code treats it. SmgrTruncate() goes on to do
smgrtruncate(RelationGetSmgr(rel), forks, nforks, old_blocks, blocks)
which iterates over all forks and uses the same block number for all
of them, smgr_redo() also does this, and SummarizeSmgrRecord() also
calls BlockRefTableSetLimitBlock() for each relevant fork with that
same block number. This really makes no sense to me unless the block
count happens to be zero, but AFAICT all the code agrees that this is
how it's supposed to work.
I think the problem here is that the incremental backup code makes the
apparently-naive assumption that the purpose of truncation is to make
things shorter. In this case, all forks were truncated to a random
length that was well in excess of the length of the VM fork, and in
pg_combinebackup, find_reconstructed_block_length() interprets that to
mean that the output file should be at least as long as the truncation
length. I am at present uncertain whether that can be safely changed
without breaking anything else. I don't think that what we're doing is
unsafe in the sense of producing corrupted data, because a bunch of
trailing blocks of zeroes are harmless, but it's obviously potentially
pretty problematic if it causes a huge disk space blowup as it did
here. So I think something should be done about this, but I think the
original issue you reported is more urgent.
So my suggestion is to change the test so that it produces a file that
is the same small size on every platform. On most platforms, this will
be 1 segment. On the CI platform where we set the segment size to 6,
it will be multiple segments, and on that platform only it will
effectively test for this bug. If you do that, then we can commit the
fix for the original problem. We (or someone else) can then look into
what needs to address the excessive zero-padding as a separate issue.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-07 14:49 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-14 15:38 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
@ 2026-01-16 12:10 ` Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-19 17:42 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Oleg Tkachenko @ 2026-01-16 12:10 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
Hi, Robert
As you suggested, I've updated the test so that it produces small files consistently across platforms.
On one specific platform (with segment size = 6 blocks), it still exercises the relevant code path.
And I've also applied the style changes you mentioned.
Best regards,
Oleg
> On Jan 14, 2026, at 16:38, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Wed, Jan 7, 2026 at 9:50 AM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
>> Both forks have the same limit, which looks wrong.
>> So I checked the WAL files to see what really happened with the VM fork.
>> I did not find any “truncate" records for the VM file.
>> I only found this record for the main fork
>> (actually, the fork isn’t mentioned at all):
>>
>> rmgr: Storage len (rec/tot): 46/46, tx: 759, lsn: 0/4600D318,
>> prev 0/4600B2C8, desc: TRUNCATE base/5/16384 to 131073 blocks flags 7
>
> Flags 7 for Storage/TRUNCATE means all forks:
>
> #define SMGR_TRUNCATE_HEAP 0x0001
> #define SMGR_TRUNCATE_VM 0x0002
> #define SMGR_TRUNCATE_FSM 0x0004
> #define SMGR_TRUNCATE_ALL \
> (SMGR_TRUNCATE_HEAP|SMGR_TRUNCATE_VM|SMGR_TRUNCATE_FSM)
>
> I think this comes from RelationTruncate(), which does indeed set
> xlrec.flags = SMGR_TRUNCATE_ALL. It seems bananas to me to use the
> same count of blocks for all forks, but it seems that is the way the
> code treats it. SmgrTruncate() goes on to do
> smgrtruncate(RelationGetSmgr(rel), forks, nforks, old_blocks, blocks)
> which iterates over all forks and uses the same block number for all
> of them, smgr_redo() also does this, and SummarizeSmgrRecord() also
> calls BlockRefTableSetLimitBlock() for each relevant fork with that
> same block number. This really makes no sense to me unless the block
> count happens to be zero, but AFAICT all the code agrees that this is
> how it's supposed to work.
>
> I think the problem here is that the incremental backup code makes the
> apparently-naive assumption that the purpose of truncation is to make
> things shorter. In this case, all forks were truncated to a random
> length that was well in excess of the length of the VM fork, and in
> pg_combinebackup, find_reconstructed_block_length() interprets that to
> mean that the output file should be at least as long as the truncation
> length. I am at present uncertain whether that can be safely changed
> without breaking anything else. I don't think that what we're doing is
> unsafe in the sense of producing corrupted data, because a bunch of
> trailing blocks of zeroes are harmless, but it's obviously potentially
> pretty problematic if it causes a huge disk space blowup as it did
> here. So I think something should be done about this, but I think the
> original issue you reported is more urgent.
>
> So my suggestion is to change the test so that it produces a file that
> is the same small size on every platform. On most platforms, this will
> be 1 segment. On the CI platform where we set the segment size to 6,
> it will be multiple segments, and on that platform only it will
> effectively test for this bug. If you do that, then we can commit the
> fix for the original problem. We (or someone else) can then look into
> what needs to address the excessive zero-padding as a separate issue.
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
Attachments:
0001-Clamp-truncation_block_length-to-RELSEG_SIZE-when-se.patch (application/octet-stream, 8.1K) download
From 98f52d6bb5c0464465ab75f48ae3dc5d9a18c99f Mon Sep 17 00:00:00 2001
From: Oleg Tkachenko <oleg@elsci.io>
Date: Fri, 16 Jan 2026 12:54:14 +0100
Subject: [PATCH] Clamp truncation_block_length to RELSEG_SIZE when sending
incremental files
- Fix bug where truncation_block_length could exceed RELSEG_SIZE, leading to restore failures.
- Add TAP test to verify that a relation truncated after the last base backup can be safely included in an incremental backup.
While fixing the issue with the truncation of the main fork, we discovered another bug in the incremental backup. When WAL summaries are processed, the incremental backup code incorrectly assumes that truncation of the main fork also applies to the VM fork. As a result, after restore, the VM fork file can end up with an incorrect size.
At the time of writing, it was decided to postpone fixing the VM fork issue and proceed only with the fix for the main fork truncation.
This patch follows guidance from Robert Haas.
---
src/backend/backup/basebackup_incremental.c | 14 ++
src/bin/pg_combinebackup/meson.build | 1 +
...011_incremental_backup_truncation_block.pl | 131 ++++++++++++++++++
3 files changed, 146 insertions(+)
create mode 100644 src/bin/pg_combinebackup/t/011_incremental_backup_truncation_block.pl
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index 891af7cad8e..77dce24ad38 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -850,8 +850,22 @@ GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,
{
unsigned relative_limit = limit_block - segno * RELSEG_SIZE;
+ /*
+ * We can't set a truncation_block_length in excess of the limit block
+ * number (relativized to the current segment). To do so would be to
+ * treat blocks from older backups as valid current contents even if
+ * they were subsequently truncated away.
+ */
if (*truncation_block_length < relative_limit)
*truncation_block_length = relative_limit;
+
+ /*
+ * We also can't set a truncation_block_length in excess of the
+ * segment size, since the reconstructed file can't be larger than
+ * that.
+ */
+ if (*truncation_block_length > RELSEG_SIZE)
+ *truncation_block_length = RELSEG_SIZE;
}
/* Send it incrementally. */
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 44780927cd6..613f73a756b 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -38,6 +38,7 @@ tests += {
't/008_promote.pl',
't/009_no_full_file.pl',
't/010_hardlink.pl',
+ 't/011_incremental_backup_truncation_block.pl',
],
}
}
diff --git a/src/bin/pg_combinebackup/t/011_incremental_backup_truncation_block.pl b/src/bin/pg_combinebackup/t/011_incremental_backup_truncation_block.pl
new file mode 100644
index 00000000000..6c9e39e84e5
--- /dev/null
+++ b/src/bin/pg_combinebackup/t/011_incremental_backup_truncation_block.pl
@@ -0,0 +1,131 @@
+# Test case for a scenario where truncation_block is miscalculated
+# during an incremental backup
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Initialize primary node
+my $primary = PostgreSQL::Test::Cluster->new('primary');
+$primary->init(has_archiving => 1, allows_streaming => 1);
+# Enable WAL summarization to support incremental backup
+$primary->append_conf('postgresql.conf', 'summarize_wal = on');
+$primary->start;
+
+# Backup locations
+my $backup_path = $primary->backup_dir;
+my $full_backup = "$backup_path/full";
+
+# While fixing the issue with the truncation of the main fork,
+# we discovered another bug in the incremental backup.
+# When WAL summaries are processed, the incremental backup code
+# incorrectly assumes that truncation of the main fork also applies to
+# the VM fork. As a result, after restore, the VM fork file can end up
+# with an incorrect size.
+#
+# At the time of writing, it was decided to postpone fixing the VM fork
+# issue and proceed only with the fix for the main fork truncation.
+#
+# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+# Some test platforms are constrained by disk space, so creating a
+# 1GB VM fork file is not possible there.
+# Other platforms configure the segment size to six blocks,
+# which means that only on those platforms this test actually tests the
+# problematic code path.
+# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+#
+# Therefore, the test is written to produce a relation that spans more
+# than one segment but remains small on most systems. We first create
+# enough pages to exceed a single segment, then remove rows to make
+# truncation possible.
+my $segment_size_blocks = 6;
+my $block_size = $primary->safe_psql('postgres',
+ "SELECT current_setting('block_size')::int;");
+
+# Two blocks will be located in the second segment, and one block
+# will still stay there after truncation.
+my $target_rows = int($segment_size_blocks + 2);
+my $rows_after_truncation = int($target_rows - 1);
+
+# Create a test table.
+# STORAGE PLAIN prevents compression and TOASTing of repetitive data,
+# ensuring predictable row sizes.
+$primary->safe_psql(
+ 'postgres', q{
+ CREATE TABLE t (
+ id int,
+ data text STORAGE PLAIN
+ );
+});
+
+# The tuple size should be enough to prevent two tuples
+# from being on the same page.
+# Since the template string has a length of 32 bytes,
+# it's enough to repeat it (block_size / (2*32)) times.
+$primary->safe_psql(
+ 'postgres',
+ "INSERT INTO t
+ SELECT i,
+ repeat('0123456789ABCDEF0123456789ABCDEF', ($block_size / (2*32)))
+ FROM generate_series(1, $target_rows) i;"
+);
+
+# This step is required because at this moment,
+# tuples do not have hint bits set.
+# Later (for example, soon after the base backup is created), a background
+# process may set hint bits on many tuples and change many heap pages.
+# Because of this, the WAL summary may show that too many pages were changed
+# and create a full file copy instead of an incremental one, which makes the
+# issue non-reproducible.
+$primary->safe_psql('postgres', 'VACUUM t;');
+
+# Verify that relation spans more than one physical storage segment
+my $t_blocks = $primary->safe_psql('postgres',
+ "SELECT pg_relation_size('t') / current_setting('block_size')::int;");
+cmp_ok($t_blocks, '>', $segment_size_blocks,
+ 'Relation spans more than one physical segment.');
+
+# Take a full base backup
+$primary->backup('full');
+
+# Delete rows at the logical end of the table.
+# This creates removable empty pages at the tail
+$primary->safe_psql('postgres',
+ "DELETE FROM t WHERE id > ($rows_after_truncation);");
+
+# Although TRUNCATE is enabled by default,
+# here it emphasizes the expected behavior of the operation.
+$primary->safe_psql('postgres', 'VACUUM (TRUNCATE) t;');
+
+# Verify that after VACUUM relation is truncated but still spans more than one
+# physical storage segment.
+$t_blocks = $primary->safe_psql('postgres',
+ "SELECT pg_relation_size('t') / current_setting('block_size')::int;");
+is($t_blocks, $rows_after_truncation, 'Relation has expected size.');
+cmp_ok($t_blocks, '>', $segment_size_blocks,
+ 'Relation spans more than one physical segment.');
+
+# Take an incremental backup based on the full backup manifest
+$primary->backup('incr',
+ backup_options => [ '--incremental', "$full_backup/backup_manifest" ]);
+
+# Combine full and incremental backups.
+# This step must correctly handle truncated relation segments.
+# Before the fix, this failed because the INCREMENTAL file header
+# contained an incorrect truncation_block value.
+my $restored = PostgreSQL::Test::Cluster->new('node2');
+$restored->init_from_backup($primary, 'incr', combine_with_prior => ['full']);
+$restored->start();
+
+# Check that the restored table contains the correct number of rows
+my $restored_count =
+ $restored->safe_psql('postgres', "SELECT count(*) FROM t;");
+is($restored_count, $rows_after_truncation,
+ 'Restored backup has correct row count');
+
+$primary->stop;
+$restored->stop;
+
+done_testing();
--
2.52.0
=
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-07 14:49 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-14 15:38 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-16 12:10 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
@ 2026-01-19 17:42 ` Robert Haas <robertmhaas@gmail.com>
2026-01-25 12:00 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Alexander Lakhin <exclusion@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Robert Haas @ 2026-01-19 17:42 UTC (permalink / raw)
To: Oleg Tkachenko <oatkachenko@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
On Fri, Jan 16, 2026 at 7:10 AM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
> As you suggested, I've updated the test so that it produces small files consistently across platforms.
> On one specific platform (with segment size = 6 blocks), it still exercises the relevant code path.
> And I've also applied the style changes you mentioned.
Thanks. I have committed the fix and back-patched to v17. I made some
style changes to your test, especially rewriting comments, but the
substance of it is unchanged.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-07 14:49 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-14 15:38 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-16 12:10 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-19 17:42 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
@ 2026-01-25 12:00 ` Alexander Lakhin <exclusion@gmail.com>
2026-01-26 18:14 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Alexander Lakhin @ 2026-01-25 12:00 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; Andrew Dunstan <andrew@dunslane.net>; +Cc: Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Oleg Tkachenko <oatkachenko@gmail.com>; Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
Hello Robert,
19.01.2026 19:42, Robert Haas wrote:
> On Fri, Jan 16, 2026 at 7:10 AM Oleg Tkachenko <oatkachenko@gmail.com> wrote:
>> As you suggested, I've updated the test so that it produces small files consistently across platforms.
>> On one specific platform (with segment size = 6 blocks), it still exercises the relevant code path.
>> And I've also applied the style changes you mentioned.
> Thanks. I have committed the fix and back-patched to v17. I made some
> style changes to your test, especially rewriting comments, but the
> substance of it is unchanged.
As Windows animal fairywren shows at [1], the test
011_incremental_backup_truncation_block, added by ecd275718, hit Windows
limitation to the path length:
126/281 postgresql:pg_combinebackup / pg_combinebackup/011_incremental_backup_truncation_block ERROR 96.71s
(exit status 255 or signal 127 SIGinvalid)
log/011_incremental_backup_truncation_block_primary.log contains:
...
2026-01-20 02:11:14.760 UTC [7480:2] [unknown] LOG: connection authenticated: user="pgrunner" method=trust
(C:/tools/xmsys64/home/pgrunner/bf/root/REL_18_STABLE/pgsql.build/testrun/pg_combinebackup/011_incremental_backup_truncation_block/data/t_011_incremental_backup_truncation_block_primary_data/pgdata/pg_hba.conf:117)
...
2026-01-20 02:11:23.506 UTC [1252:3] LOG: checkpoint starting: immediate force wait
2026-01-20 02:11:23.580 UTC [1252:4] LOG: checkpoint complete: wrote 5 buffers (3.9%), wrote 1 SLRU buffers; 0 WAL
file(s) added, 0 removed, 0 recycled; write=0.001 s, sync=0.001 s, total=0.074 s; sync files=0, longest=0.000 s,
average=0.000 s; distance=32768 kB, estimate=32768 kB; lsn=0/4000080, redo lsn=0/4000028
2026-01-20 02:11:23.974 UTC [8968:2] ERROR: could not rename file "pg_wal/summaries/temp.summary" to
"pg_wal/summaries/00000001000000000100002800000000010CAA50.summary": No such file or directory
1 file(s) copied.
2026-01-20 02:11:33.645 UTC [7968:14] 011_incremental_backup_truncation_block.pl WARNING: still waiting for WAL
summarization through 0/4000028 after 10 seconds
2026-01-20 02:11:33.645 UTC [7968:15] 011_incremental_backup_truncation_block.pl DETAIL: Summarization has reached
0/1000000 on disk and 0/10CAA50 in memory.
That is, the target filename for the rename operation is:
C:/tools/xmsys64/home/pgrunner/bf/root/REL_18_STABLE/pgsql.build/testrun/pg_combinebackup/011_incremental_backup_truncation_block/data/t_011_incremental_backup_truncation_block_primary_data/pgdata/pg_wal/summaries/00000001000000000100002800000000010CAA50.summary
I can reproduce this locally with the source tree located at
/c/src/postgresql12345678901234567890123456789012345678901.
Not reproduced with HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled set to 1.
Note that the test passes at the master branch ([2]) because "master" is
shorter than "REL_18_STABLE" by 7 chars.
See also [3].
[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2026-01-20%2001%3A26%3A02
[2] https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=fairywren&dt=2026-01-24%2010%3A03%...
[3] https://www.postgresql.org/message-id/666ac55b-3400-fb2c-2cea-0281bf36a53c%40dunslane.net
Best regards,
Alexander
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-07 14:49 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-14 15:38 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-16 12:10 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-19 17:42 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2026-01-25 12:00 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Alexander Lakhin <exclusion@gmail.com>
@ 2026-01-26 18:14 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 19+ messages in thread
From: Robert Haas @ 2026-01-26 18:14 UTC (permalink / raw)
To: Alexander Lakhin <exclusion@gmail.com>; +Cc: Andrew Dunstan <andrew@dunslane.net>; Amul Sul <sulamul@gmail.com>; pgsql-hackers@lists.postgresql.org, Oleg Tkachenko <oatkachenko@gmail.com>; Stanislav Bashkyrtsev <stanislav.bashkyrtsev@elsci.io>
On Sun, Jan 25, 2026 at 7:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
> As Windows animal fairywren shows at [1], the test
> 011_incremental_backup_truncation_block, added by ecd275718, hit Windows
> limitation to the path length:
> 126/281 postgresql:pg_combinebackup / pg_combinebackup/011_incremental_backup_truncation_block ERROR 96.71s
> (exit status 255 or signal 127 SIGinvalid)
Thanks for the report. I have pushed a commit to rename this test case
to 011_incremental_backup_truncation_block to 011_ib_truncation, which
I hope will be enough to fix this. I noticed when committing
originally that this test case's name was a lot longer than anything
else in the same directory, but I figured it didn't matter enough to
bother changing it. Oops.
I also wonder if there's some way we could change some of our pathname
construction logic to mitigate this. Notice that in this pathname:
C:/tools/xmsys64/home/pgrunner/bf/root/REL_18_STABLE/pgsql.build/testrun/pg_combinebackup/011_incremental_backup_truncation_block/data/t_011_incremental_backup_truncation_block_primary_data/pgdata/pg_wal/summaries/00000001000000000100002800000000010CAA50.summary
...the full name of the test case appears twice, once as a
subdirectory of pg_combinebackup, indicating which pg_combinebackup
test is running, and then again as part of the name of the data
directory. But why does the directory need to be named
t_011_incremental_backup_truncation_block_primary_data instead of, you
know, primary_data? I would sort of like to hope that in 2026, we
wouldn't be subject to a 260-character pathname limit. But if we are,
repeating the same strings multiple times in that pathname doesn't
seem like the way to go.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
@ 2025-12-16 08:05 ` Chao Li <li.evan.chao@gmail.com>
2025-12-16 18:43 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 19+ messages in thread
From: Chao Li @ 2025-12-16 08:05 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; Oleg Tkachenko <oatkachenko@gmail.com>; pgsql-hackers@lists.postgresql.org
> On Dec 16, 2025, at 00:35, Robert Haas <robertmhaas@gmail.com> wrote:
>
> [ sorry for not noticing this thread sooner; thanks to Andres for
> pointing me to it ]
>
> On Mon, Dec 15, 2025 at 9:01 AM Amul Sul <sulamul@gmail.com> wrote:
>> Thanks for the reproducer; I can see the reported issue, but I am not
>> quite sure the proposed fix is correct and might break other cases (I
>> haven't tried constructed that case yet) but there is a comment
>> detailing that case just before the point where you are planning to do
>> the changes:
>>
>> /*
>> * The truncation block length is the minimum length of the reconstructed
>> * file. Any block numbers below this threshold that are not present in
>> * the backup need to be fetched from the prior backup. At or above this
>> * threshold, blocks should only be included in the result if they are
>> * present in the backup. (This may require inserting zero blocks if the
>> * blocks included in the backup are non-consecutive.)
>> */
>>
>> IIUC, we might need the original assignment logic as it is. But we
>> need to ensure that truncation_block_length is not set to a value that
>> exceeds RELSEG_SIZE.
>
> I think you're right. By way of example, let's say that the current
> length of the file is 200 blocks, but the limit block is 100 blocks
> into the current segment. That means that the only blocks that we can
> get from any previous backup are blocks 0-99. Blocks 100-199 of the
> current segment are either mentioned in the WAL summaries we're using
> for this backup, or they're all zeroes. We can't set the
> truncation_block_length to a value greater than 100, or we'll go
> looking for the contents of any zero-filled blocks in previous
> backups, will will either fail or produce the wrong answer. But Oleg
> is correct that we also shouldn't set it to a value greater than
> RELSEG_SIZE. So my guess is that the correct fix might be something
> like the attached (untested, for discussion).
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
> <v1-0001-Don-t-set-the-truncation_block_length-rather-than.patch>
The change looks good to me. Only nitpick is:
```
Subject: [PATCH v1] Don't set the truncation_block_length rather than
RELSEG_SIZE.
```
I guess you meant to say “larger (or greater) than” instead of “rather than”.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Robert Haas <robertmhaas@gmail.com>
2025-12-16 08:05 ` Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Chao Li <li.evan.chao@gmail.com>
@ 2025-12-16 18:43 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 19+ messages in thread
From: Robert Haas @ 2025-12-16 18:43 UTC (permalink / raw)
To: Chao Li <li.evan.chao@gmail.com>; +Cc: Amul Sul <sulamul@gmail.com>; Oleg Tkachenko <oatkachenko@gmail.com>; pgsql-hackers@lists.postgresql.org
On Tue, Dec 16, 2025 at 3:06 AM Chao Li <li.evan.chao@gmail.com> wrote:
> I guess you meant to say “larger (or greater) than” instead of “rather than”.
Yes, thanks.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 19+ messages in thread
end of thread, other threads:[~2026-01-26 18:14 UTC | newest]
Thread overview: 19+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-11-14 19:43 [BUG] pg_basebackup produces wrong incremental files after relation truncation in segmented tables Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 11:09 ` Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 14:01 ` Amul Sul <sulamul@gmail.com>
2025-12-15 16:35 ` Robert Haas <robertmhaas@gmail.com>
2025-12-15 18:46 ` Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-15 20:13 ` Robert Haas <robertmhaas@gmail.com>
2025-12-16 21:55 ` Oleg Tkachenko <oatkachenko@gmail.com>
2025-12-18 06:05 ` Amul Sul <sulamul@gmail.com>
2025-12-18 14:26 ` Robert Haas <robertmhaas@gmail.com>
2025-12-18 17:23 ` Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-05 16:05 ` Robert Haas <robertmhaas@gmail.com>
2026-01-07 14:49 ` Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-14 15:38 ` Robert Haas <robertmhaas@gmail.com>
2026-01-16 12:10 ` Oleg Tkachenko <oatkachenko@gmail.com>
2026-01-19 17:42 ` Robert Haas <robertmhaas@gmail.com>
2026-01-25 12:00 ` Alexander Lakhin <exclusion@gmail.com>
2026-01-26 18:14 ` Robert Haas <robertmhaas@gmail.com>
2025-12-16 08:05 ` Chao Li <li.evan.chao@gmail.com>
2025-12-16 18:43 ` Robert Haas <robertmhaas@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox