pg.ddx.io  pgbackrest/pgbackrest GitHub issues and pull requests  
help / color / mirror / Atom feed
[pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
20+ messages / 4 participants
[nested] [flat]

* [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
@ 2026-11-07 14:29 "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
  0 siblings, 0 replies; 20+ messages in thread

From: melancholictheory (@melancholictheory) @ 2026-11-07 14:29 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

## Summary

- Limit the memory context free list to allocations matching the configured I/O buffer size (buffer-size).
- Keep the eight-block and 64 MiB limits per process. Return blocks of other sizes to the allocator.
- Cover reuse, size mismatches, both limits, and memory-context cleanup in the unit tests.

Fixes #2911. This is the single-bucket variant discussed in the review.

## Testing

- Passed common/mem-context and all six storage unit-test modules with full coverage.
- Generated the HTML documentation with example execution disabled.
- Repeated the block-incremental differential backup workload from #2816 against local MinIO, using 10,000 small PostgreSQL 18 tables and six runs per build. Compared with the revert, median elapsed time fell from 3.22 s to 2.88 s on ARM64 and from 3.31 s to 2.95 s on x86-64. On the x86-64 host the single bucket was faster in all six rounds, with 33.4 MiB peak anonymous RSS versus 51.3 MiB for the original free list. [ARM64 results](https://github.com/pgbackrest/pgbackrest/pull/2912#issuecomment-6739740033) · [x86-64 results](https://github.com/pgbackrest/pgbackrest/pull/2912#issuecomment-6740094300)


^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Reduce free-list retention during chunked uploads
@ 2026-11-09 10:21 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-09 10:21 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

Thank you for the patch! On further reflection I think the best thing to do is revert the free list for the Nov patch release (#2914). Patch releases are not a good time for experimentation and that's what we are doing here.

If we want to bring back the free list in this PR it might be best to limit it to *only* buffers of `buffer-size`. Thoughts?

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Reduce free-list retention during chunked uploads
@ 2026-11-09 11:15 ` "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: melancholictheory (@melancholictheory) @ 2026-11-09 11:15 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

Thanks, David. I agree with reverting for the November patch release. A single bucket keyed to `buffer-size` makes sense if we bring the cache back.

I compared the #2914 revert (`f8b4d4a1d`) with a prototype that caches only allocations of exactly `buffer-size`, with up to eight blocks per process. Both used the same upload code, local MinIO on Linux arm64, `buffer-size=1MiB`, 5 MiB S3 chunks, zstd level 1, and `process-max=16`.

| Workload | Version | Peak summed RssAnon (MiB), per run | Elapsed (s), per run |
| --- | --- | --- | --- |
| Async archive-push, same 231 PostgreSQL 18 WAL segments | Revert | 322.5 / 322.5 / 321.3 | 27.74 / 16.74 / 14.06 |
| | One bucket | 349.2 / 654.8 / 341.2 | 23.45 / 146.69 / 23.19 |
| Full backup, 785 MiB, archive check disabled for this measurement | Revert | 80.9 / 82.0 | 12.06 / 7.72 |
| | One bucket | 83.7 / 84.2 | 7.80 / 7.96 |

I retrieved all 231 segments from both variants in the first pair and verified their SHA-256 hashes against the originals. The slow archive-push run with the one-bucket prototype did not repeat in the third pair, so I would treat it as an unexplained outlier. On backup the one-bucket variant recorded roughly 17–23% fewer minor faults, with elapsed times near the faster revert run.

I'd ship the revert and revisit the single-size cache if a workload shows a repeatable elapsed-time improvement from the reduction in page faults.


^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Reduce free-list retention during chunked uploads
@ 2026-11-09 11:40 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-09 11:40 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

> revisit the single-size cache if a workload shows a repeatable elapsed-time improvement from the reduction in page faults.

I'm not sure that I'm seeing that in the numbers above. There is some expected jitter but no evidence that caching the I/O buffers gives a measurable improvement in performance. If possible, this is a question I'd like to resolve this week. If we decide to keep I/O buffer caching then I'd like to apply that on main without the revert in between. In that case the revert can go only in the patch branch.

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-09 12:11 ` "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: melancholictheory (@melancholictheory) @ 2026-11-09 12:11 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

I tried the workload described in #2816 on Linux ARM64 with local MinIO. So far that's the only platform I've tested for this workload.

I created 10,000 small PostgreSQL 18 tables, took one full backup with block incremental enabled, and updated every other table. Each build then ran six differential backups against the same full backup, with the build order rotated between rounds. The runs used process-max=1, buffer-size=1MiB, and zstd level 1.

| Build | Median elapsed time | Median minor faults | Median peak summed RssAnon |
| --- | ---: | ---: | ---: |
| #2914 revert | 3.22 s | 302k | 28.3 MiB |
| Original free list | 2.96 s | 24k | 50.8 MiB |
| Single bucket for buffer-size | 2.88 s | 22k | 34.6 MiB |

The single bucket was faster than the revert in five of six rounds. The median difference was about 11%, or 6% when excluding the first round, where the revert took 8.38 s. That gives us a modest elapsed-time improvement on this ARM64 workload alongside a much larger reduction in minor faults. The x86-64/KPTI case is still untested.


^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-09 12:30 ` "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: melancholictheory (@melancholictheory) @ 2026-11-09 12:30 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

I repeated the #2816 workload on native Linux x86-64 with local MinIO and glibc 2.41. The Intel Xeon E-2388G host reports “Not affected” for Meltdown and does not have PTI active.

The setup was PostgreSQL 18 with 10,000 small tables, a shared full backup with block incremental enabled, and an update to every other table. I ran six differential backups per build, rotating the order, with process-max=1, buffer-size=1MiB, and zstd level 1.

| Build | Median elapsed time | Median minor faults | Median peak summed RssAnon |
| --- | ---: | ---: | ---: |
| #2914 revert | 3.31 s | 293k | 28.3 MiB |
| Original free list | 2.58 s | 24k | 51.3 MiB |
| Single bucket for buffer-size | 2.95 s | 24k | 33.4 MiB |

The single bucket was faster than the revert in all six rounds, with an 11% lower median elapsed time. It used about 18 MiB less peak anonymous RSS than the original free list. This confirms a repeatable elapsed-time benefit for the single bucket on this x86-64 workload.


^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-09 12:31 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-09 12:31 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

Well that looks pretty good, then. Can you update this PR to the "single bucket for buffer-size" that you tested?

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-09 12:35 ` "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: melancholictheory (@melancholictheory) @ 2026-11-09 12:35 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@melancholictheory renamed this.

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912 (approved): Fix excessive memory use by the memory context free list.
@ 2026-11-09 13:59 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-09 13:59 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

I made a few updates to comments but otherwise looks good to me.

Now the question is, should we ship this partial revert in the Nov patch release or the full revert in #2914? Caution would argue for the full revert since that restores to pre-2.60 code which has been in use for a long time.

I'm leaning towards the partial revert since the issue was excessive memory use and this should definitely help with that. If the problem was a segfault or some other hard error I'd feel differently, for sure.

@pgstef @hunleyd Thoughts?

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-09 14:48 ` "pgstef (@pgstef)" <noreply+pgstef@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: pgstef (@pgstef) @ 2026-11-09 14:48 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

> I'm leaning towards the partial revert since the issue was excessive memory use and this should definitely help with that. If the problem was a segfault or some other hard error I'd feel differently, for sure.

I didn't get a chance to comment earlier, but the single-bucket rewrite looks good to me.

Since we're changing the initial feature, I would have preferred a full revert for the patch release and this improvement in 2.61. However, since the new approach is fixing the original issue, I'm fine with skipping the revert and going with this patch instead.



^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-09 15:01 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-09 15:01 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

> However, since the new approach is fixing the original issue, I'm fine with skipping the revert and going with this patch instead.

The initial PR was definitely too much to put in a patch release, but I think this cut-down version is OK.

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-10 11:00 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 11:00 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele set milestone to "2.60.1".

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912 (approved): Fix excessive memory use by the memory context free list.
@ 2026-11-10 14:18 ` "hunleyd (@hunleyd)" <noreply+hunleyd@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: hunleyd (@hunleyd) @ 2026-11-10 14:18 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>



^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size
@ 2026-11-10 14:22 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 14:22 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@hunleyd So what do you think about putting this in the 2.60.1 patch release vs the full revert in #2914?

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
@ 2026-11-13 15:16 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-13 15:16 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele renamed this.

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
@ 2026-11-13 15:19 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-13 15:19 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele assigned @dwsteele.

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
@ 2026-11-13 15:19 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-13 15:19 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele applied label "bug".

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
@ 2026-11-13 15:19 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-13 15:19 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele applied label "module [core]".

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
@ 2026-11-13 16:13 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-13 16:13 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele merged this (commit ac13a75b8300).

^ permalink  raw  [nested|flat] 20+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list.
@ 2026-11-13 16:13 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  18 siblings, 0 replies; 20+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-13 16:13 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele closed this.

^ permalink  raw  [nested|flat] 20+ messages in thread


end of thread, other threads:[~2026-11-13 16:13 UTC | newest]

Thread overview: 20+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-07 14:29 [pgbackrest/pgbackrest] PR #2912: Fix excessive memory use by the memory context free list. "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
2026-11-09 10:21 ` Re: [pgbackrest/pgbackrest] PR #2912: Reduce free-list retention during chunked uploads "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-09 11:15 ` Re: [pgbackrest/pgbackrest] PR #2912: Reduce free-list retention during chunked uploads "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
2026-11-09 11:40 ` Re: [pgbackrest/pgbackrest] PR #2912: Reduce free-list retention during chunked uploads "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-09 12:11 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
2026-11-09 12:30 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
2026-11-09 12:31 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-09 12:35 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "melancholictheory (@melancholictheory)" <noreply+melancholictheory@github.com>
2026-11-09 13:59 ` Re: [pgbackrest/pgbackrest] PR #2912 (approved): Fix excessive memory use by the memory context free list. "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-09 14:48 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "pgstef (@pgstef)" <noreply+pgstef@github.com>
2026-11-09 15:01 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 11:00 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 14:18 ` Re: [pgbackrest/pgbackrest] PR #2912 (approved): Fix excessive memory use by the memory context free list. "hunleyd (@hunleyd)" <noreply+hunleyd@github.com>
2026-11-10 14:22 ` Re: [pgbackrest/pgbackrest] PR #2912: Limit the free list to the configured I/O buffer size "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-13 15:16 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-13 15:19 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-13 15:19 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-13 15:19 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-13 16:13 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-13 16:13 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox