pg.ddx.io  pgbackrest/pgbackrest GitHub issues and pull requests  
help / color / mirror / Atom feed
From: dwsteele (@dwsteele) <noreply+dwsteele@github.com>
To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>
Subject: [pgbackrest/pgbackrest] PR #2914: Revert "Reuse large freed allocations from a memory context free list".
Date: Mon, 09 Nov 2026 10:15:20 +0000
Message-ID: <gh-pgbackrest-pgbackrest-2914@github.com> (raw)

702d664f retained large freed allocations on a free list in the memory context manager to avoid page faults when they were allocated again.

The free list retained much more memory than intended. A buffer that grows with memResize() is never taken from the list but is added to it when freed, so the object store chunk buffer filled its bucket. Exact-size copies that do not repeat, e.g. the last part of a multi-part upload, were also retained until evicted. Each local process could retain up to 64MiB and in testing a 16 process archive-push peaked at 1212MiB compared to 400MiB without the free list. When the list was full the I/O buffers it was intended for could not be retained.

The time saved was small even in benchmarks designed to stress allocation and was not measurable in archive-push testing. It does not seem to be worth the memory or the complexity.

The Valgrind client requests in valgrind.h remain since d5f74996 uses them for Buffer, String, and List. The VALGRIND_MAKE_MEM_NOACCESS() fallback is removed since it is no longer used.

inline patch 1: meson.build, src/common/memContext.c, src/common/memContext.h and 3 more (+9/-288 lines) download inline-1.patchdiff --git a/meson.build b/meson.build index e6e99e77d2..f14d9242d6 100644 --- a/meson.build +++ b/meson.build @@ -271,8 +271,7 @@ endif if get_option('debug') configuration.set('DEBUG', true, description: 'Enable debug code') - # Enable Valgrind memcheck client requests when the header is available so allocators that retain freed memory for reuse keep - # use-after-free and uninitialized-read detection accurate under Valgrind. The requests are no-ops when not run under Valgrind. + # Enable Valgrind memcheck client requests when the header is available. The requests are no-ops when not run under Valgrind. if cc.has_header('valgrind/memcheck.h') add_project_arguments('-DWITH_VALGRIND', language: 'c') endif diff --git a/src/common/memContext.c b/src/common/memContext.c index f39810989a..deea87e607 100644 --- a/src/common/memContext.c +++ b/src/common/memContext.c @@ -9,7 +9,6 @@ Memory Context Manager #include "common/debug.h" #include "common/macro.h" #include "common/memContext.h" -#include "common/valgrind.h" /*********************************************************************************************************************************** Contains information about a memory allocation. This header is placed at the beginning of every memory allocation returned to the @@ -497,185 +496,6 @@ memFreeInternal(void *const buffer) FUNCTION_TEST_RETURN_VOID(); } -/*********************************************************************************************************************************** -Free list for recycling large allocations - -Commands that process many files, e.g. backup, churn large uniformly-sized allocations (buffers, etc.) through short-lived -contexts. Returning these to the allocator lets it hand the pages back to the kernel, which then faults them back in for the next -file. Minor page faults are cheap on some hosts but expensive on others, so this churn can dominate run time and vary from run to -run with heap layout. Retaining a bounded number of large freed blocks here keeps the pages resident and makes reuse -deterministic, independent of the C library and heap layout. - -Only allocations at least MEM_FREE_LIST_THRESHOLD are recycled since smaller allocations are handled well by the allocator's own -arena and do not cause churn. The buckets are keyed by exact total size, which is a perfect fit for the uniformly-sized allocations -that churn. Retained memory is limited by MEM_FREE_LIST_BYTE_MAX so the cost is bounded regardless of buffer size. -***********************************************************************************************************************************/ -#define MEM_FREE_LIST_THRESHOLD ((size_t)(64 * 1024)) -#define MEM_FREE_LIST_SIZE_MAX 8 -#define MEM_FREE_LIST_BLOCK_MAX 8 -#define MEM_FREE_LIST_BYTE_MAX ((size_t)(64 * 1024 * 1024)) -#define MEM_FREE_LIST_REUSE_MAX 8 - -typedef struct MemFreeBucket -{ - unsigned int size; // Total block size (including header) held by this bucket - unsigned int blockListSize; // Blocks currently retained - unsigned int reuse; // Times reused, decremented on each eviction (see below) - void *block[MEM_FREE_LIST_BLOCK_MAX]; // Retained block (header) pointers -} MemFreeBucket; - -// The first bucket is reserved (bucketListSize starts at 1) so memFreeReuse() never assigns it so memContextFreeListReserve() can -// claim it at any time. It stays unused (size 0) until claimed. -static struct MemFreeList -{ - size_t size; // Total size currently retained - unsigned int bucketListSize; // Distinct sizes currently tracked (bucket 0 always reserved) - MemFreeBucket bucket[MEM_FREE_LIST_SIZE_MAX]; -} memFreeList = {.bucketListSize = 1}; - -/*********************************************************************************************************************************** -Allocate, reusing a retained block of the same total size when one is available -***********************************************************************************************************************************/ -static void * -memAllocReuse(const size_t size) -{ - FUNCTION_TEST_BEGIN(); - FUNCTION_TEST_PARAM(SIZE, size); - FUNCTION_TEST_END(); - - // Reuse a retained block of the same size when large enough to be worth recycling - if (size >= MEM_FREE_LIST_THRESHOLD) - { - for (unsigned int bucketIdx = 0; bucketIdx < memFreeList.bucketListSize; bucketIdx++) - { - MemFreeBucket *const bucket = &memFreeList.bucket[bucketIdx]; - - if (bucket->size == size && bucket->blockListSize > 0) - { - void *const result = bucket->block[--bucket->blockListSize]; - memFreeList.size -= size; - - // Count the reuse up to a limit so a bucket that stops being reused cannot resist eviction indefinitely - if (bucket->reuse < MEM_FREE_LIST_REUSE_MAX) - bucket->reuse++; - - // Restore uninitialized-read and bounds detection for the recycled block (matches malloc semantics) - VALGRIND_MAKE_MEM_UNDEFINED(result, size); - - FUNCTION_TEST_RETURN_P(VOID, result); - } - } - } - - FUNCTION_TEST_RETURN_P(VOID, memAllocInternal(size)); -} - -/*********************************************************************************************************************************** -Free, retaining large blocks for reuse instead of returning them to the allocator -***********************************************************************************************************************************/ -static void -memFreeReuse(MemContextAlloc *const alloc) -{ - FUNCTION_TEST_BEGIN(); - FUNCTION_TEST_PARAM_P(VOID, alloc); - FUNCTION_TEST_END(); - - ASSERT(alloc != NULL); - - // Read the size before the block is poisoned for Valgrind - const size_t size = alloc->size; - - // Retain the block for reuse when large enough and the retained memory limit will not be exceeded - if (size >= MEM_FREE_LIST_THRESHOLD && memFreeList.size + size <= MEM_FREE_LIST_BYTE_MAX) - { - // Find the bucket for this exact size - MemFreeBucket *bucket = NULL; - - for (unsigned int bucketIdx = 0; bucketIdx < memFreeList.bucketListSize; bucketIdx++) - { - if (memFreeList.bucket[bucketIdx].size == size) - { - bucket = &memFreeList.bucket[bucketIdx]; - break; - } - } - - // Create a bucket for this size if there is none and there is room for one. The reuse starts at one so a new bucket is not - // evicted before it has had a chance to be reused, which would leave two sizes trading the same slot and never settling. - if (bucket == NULL && memFreeList.bucketListSize < MEM_FREE_LIST_SIZE_MAX) - { - bucket = &memFreeList.bucket[memFreeList.bucketListSize++]; - *bucket = (MemFreeBucket){.size = (unsigned int)size, .reuse = 1}; - } - // Otherwise evict the least reused bucket to claim its slot, ignoring the reserved bucket 0. The reuse of every remaining - // bucket is then decremented so a bucket that was reused a lot but is no longer being reused erodes over time and is - // eventually evicted. Specialized one-off sizes never get reused so they erode immediately and the buckets settle on the - // sizes that recur. - else if (bucket == NULL) - { - unsigned int evictIdx = 1; - - for (unsigned int bucketIdx = 2; bucketIdx < MEM_FREE_LIST_SIZE_MAX; bucketIdx++) - { - if (memFreeList.bucket[bucketIdx].reuse < memFreeList.bucket[evictIdx].reuse) - evictIdx = bucketIdx; - } - - for (unsigned int bucketIdx = 1; bucketIdx < MEM_FREE_LIST_SIZE_MAX; bucketIdx++) - { - if (memFreeList.bucket[bucketIdx].reuse > 0) - memFreeList.bucket[bucketIdx].reuse--; - } - - bucket = &memFreeList.bucket[evictIdx]; - - // Return the evicted bucket's retained blocks to the allocator - while (bucket->blockListSize > 0) - { - memFreeInternal(bucket->block[--bucket->blockListSize]); - memFreeList.size -= bucket->size; - } - - *bucket = (MemFreeBucket){.size = (unsigned int)size, .reuse = 1}; - } - - // Retain the block if there is room in the bucket - if (bucket->blockListSize < MEM_FREE_LIST_BLOCK_MAX) - { - bucket->block[bucket->blockListSize++] = alloc; - memFreeList.size += size; - - // Poison so use-after-free is still detected while the block is retained - VALGRIND_MAKE_MEM_NOACCESS(alloc, size); - - FUNCTION_TEST_RETURN_VOID(); - } - } - - // Otherwise return the block to the allocator - memFreeInternal(alloc); - - FUNCTION_TEST_RETURN_VOID(); -} - -/**********************************************************************************************************************************/ -FN_EXTERN void -memContextFreeListReserve(const size_t size) -{ - FUNCTION_TEST_BEGIN(); - FUNCTION_TEST_PARAM(SIZE, size); - FUNCTION_TEST_END(); - - // Reserve the first bucket for allocations of this size (plus the allocation header) so the size that churns most, e.g. - // buffer size, always has a bucket and is checked first. The first bucket is permanently reserved and never assigned by - // memFreeReuse(), so this can be called at any time, but the bucket must only be reserved once. - ASSERT(memFreeList.bucket[0].size == 0); - - memFreeList.bucket[0].size = (unsigned int)(sizeof(MemContextAlloc) + size); - - FUNCTION_TEST_RETURN_VOID(); -} - /*********************************************************************************************************************************** Find space for a new mem context ***********************************************************************************************************************************/ @@ -914,7 +734,7 @@ memContextAllocNew(const size_t size) FUNCTION_TEST_END(); // Allocate memory - MemContextAlloc *const result = memAllocReuse(sizeof(MemContextAlloc) + size); + MemContextAlloc *const result = memAllocInternal(sizeof(MemContextAlloc) + size); // Find space for the new allocation MemContext *const contextCurrent = memContextStack[memContextCurrentStackIdx].memContext; @@ -1094,8 +914,8 @@ memFree(void *const buffer) contextAlloc->list[alloc->allocIdx] = NULL; } - // Free the allocation, retaining it for reuse if large enough - memFreeReuse(alloc); + // Free the allocation + memFreeInternal(alloc); FUNCTION_TEST_RETURN_VOID(); } @@ -1519,7 +1339,7 @@ memContextFreeRecurse(MemContext *const this) MemContextAllocOne *const contextAlloc = memContextAllocOne(this); if (contextAlloc->alloc != NULL) - memFreeReuse(contextAlloc->alloc); + memFreeInternal(contextAlloc->alloc); } else { @@ -1529,7 +1349,7 @@ memContextFreeRecurse(MemContext *const this) for (unsigned int allocIdx = 0; allocIdx < contextAlloc->listSize; allocIdx++) if (contextAlloc->list[allocIdx] != NULL) - memFreeReuse(contextAlloc->list[allocIdx]); + memFreeInternal(contextAlloc->list[allocIdx]); memFreeInternal(contextAlloc->list); } diff --git a/src/common/memContext.h b/src/common/memContext.h index 4f2dc5f725..6cf0457d60 100644 --- a/src/common/memContext.h +++ b/src/common/memContext.h @@ -84,10 +84,6 @@ FN_EXTERN void *memResize(void *buffer, size_t size); // Free memory allocation FN_EXTERN void memFree(void *buffer); -// Reserve the first free list bucket for allocations of the given size so the size that churns most, e.g. buffer size, always -// has a bucket and is checked first. Must be called at startup before allocations of this size begin. -FN_EXTERN void memContextFreeListReserve(size_t size); - /*********************************************************************************************************************************** Ensure that the prior memory context is restored after the block executes (even on error) diff --git a/src/common/valgrind.h b/src/common/valgrind.h index 2082c613ad..221e716e79 100644 --- a/src/common/valgrind.h +++ b/src/common/valgrind.h @@ -1,10 +1,9 @@ /*********************************************************************************************************************************** Valgrind Memcheck Client Requests -Memcheck tracks the state of memory using the requests made to the allocator, which is not enough when memory is retained by the -allocator for reuse or when part of an allocation is invalidated without being freed, e.g. when a buffer shrinks. In these cases -memory that should be inaccessible or undefined still looks valid to memcheck and errors go undetected. The client requests below -update the state directly so detection remains accurate. +Memcheck tracks the state of memory using the requests made to the allocator, which is not enough when part of an allocation is +invalidated without being freed, e.g. when a buffer shrinks. In this case memory that should be undefined still looks valid to +memcheck and errors go undetected. The client requests below update the state directly so detection remains accurate. The requests are compiled in for debug builds when the Valgrind header is available, which includes the unit tests, and are no-ops when not run under Valgrind. When the header is not available they compile to nothing. @@ -15,7 +14,6 @@ when not run under Valgrind. When the header is not available they compile to no #ifdef WITH_VALGRIND #include <valgrind/memcheck.h> #else -#define VALGRIND_MAKE_MEM_NOACCESS(addr, size) #define VALGRIND_MAKE_MEM_UNDEFINED(addr, size) #endif diff --git a/src/main.c b/src/main.c index a8e438b150..cc339d1467 100644 --- a/src/main.c +++ b/src/main.c @@ -36,7 +36,6 @@ Main #include "common/debug.h" #include "common/io/fdRead.h" #include "common/io/fdWrite.h" -#include "common/io/io.h" #include "common/stat.h" #include "config/config.h" #include "config/load.h" @@ -97,9 +96,6 @@ main(int argListSize, const char *argList[]) cfgLoad((unsigned int)argListSize, argList); const ConfigCommandRole commandRole = cfgCommandRole(); - // Reserve the first free list bucket for the I/O buffer size, the allocation that churns most while processing files - memContextFreeListReserve(ioBufferSize()); - // Main/async commands // ------------------------------------------------------------------------------------------------------------------------- if (commandRole == cfgCmdRoleMain || commandRole == cfgCmdRoleAsync) diff --git a/test/src/module/common/memContextTest.c b/test/src/module/common/memContextTest.c index 899c78c6ce..7f5e559a3b 100644 --- a/test/src/module/common/memContextTest.c +++ b/test/src/module/common/memContextTest.c @@ -319,94 +319,6 @@ testRun(void) TEST_RESULT_VOID(memContextSwitch(memContextTop()), "switch to top"); TEST_RESULT_VOID(memContextFree(memContext), "context free"); - - // ------------------------------------------------------------------------------------------------------------------------- - TEST_TITLE("large allocations are recycled through the free list"); - - // Bucket 0 is always reserved; reserve it for a fixed size, as main() does for ioBufferSize() - TEST_RESULT_UINT(memFreeList.bucket[0].size, 0, "first bucket initially unreserved"); - TEST_RESULT_VOID(memContextFreeListReserve(64 * 1024), "reserve first bucket"); - TEST_RESULT_UINT(memFreeList.bucket[0].size, sizeof(MemContextAlloc) + 64 * 1024, "reserved bucket size"); - - TEST_ASSIGN(memContext, memContextNewP("test-free-list", .allocQty = MEM_CONTEXT_QTY_MAX), "new"); - memContextKeep(); - memContextSwitch(memContext); - - // The reserved-size allocation misses the empty reserved bucket, then freeing it retains the block in that bucket - uint8_t *const large = memNew(64 * 1024); - TEST_RESULT_VOID(memFree(large), "free retains large allocation in reserved bucket"); - TEST_RESULT_UINT(memFreeList.bucketListSize, 1, "still one bucket"); - - // Allocating the same size reuses the retained block - TEST_RESULT_PTR(memNew(64 * 1024), large, "reuse retained block of same size"); - - // The same size again finds the now-empty bucket and allocates fresh - TEST_RESULT_BOOL(memNew(64 * 1024) == large, false, "empty bucket allocates fresh"); - - // Each reuse is counted up to a limit so a bucket that stops being reused cannot resist eviction indefinitely - for (unsigned int reuseIdx = 0; reuseIdx <= MEM_FREE_LIST_REUSE_MAX; reuseIdx++) - memFree(memNew(64 * 1024)); - - TEST_RESULT_UINT(memFreeList.bucket[0].reuse, MEM_FREE_LIST_REUSE_MAX, "reuse limited"); - - // A different large size does not match the existing bucket and creates a second bucket when freed - uint8_t *const largeOther = memNew(128 * 1024); - TEST_RESULT_BOOL(largeOther == large, false, "different size allocated fresh"); - TEST_RESULT_VOID(memFree(largeOther), "free different size"); - TEST_RESULT_UINT(memFreeList.bucketListSize, 2, "second bucket created"); - - // Freeing more same-size blocks than the per-bucket limit returns the extra to the allocator - void *fill[MEM_FREE_LIST_BLOCK_MAX + 1]; - - for (unsigned int fillIdx = 0; fillIdx <= MEM_FREE_LIST_BLOCK_MAX; fillIdx++) - fill[fillIdx] = memNew(96 * 1024); - - for (unsigned int fillIdx = 0; fillIdx <= MEM_FREE_LIST_BLOCK_MAX; fillIdx++) - memFree(fill[fillIdx]); - - TEST_RESULT_UINT(memFreeList.bucket[2].blockListSize, MEM_FREE_LIST_BLOCK_MAX, "bucket holds at most the per-bucket limit"); - - // Freeing blocks past the retained-memory limit returns them to the allocator - void *big[5]; - - for (unsigned int bigIdx = 0; bigIdx < 5; bigIdx++) - big[bigIdx] = memNew(16 * 1024 * 1024); - - for (unsigned int bigIdx = 0; bigIdx < 5; bigIdx++) - memFree(big[bigIdx]); - - TEST_RESULT_BOOL(memFreeList.size <= MEM_FREE_LIST_BYTE_MAX, true, "retained memory within limit"); - - // Fill all the bucket slots so a new size must evict a bucket - for (unsigned int sizeIdx = 0; memFreeList.bucketListSize < MEM_FREE_LIST_SIZE_MAX; sizeIdx++) - memFree(memNew((100 + sizeIdx) * 1024)); - - TEST_RESULT_UINT(memFreeList.bucketListSize, MEM_FREE_LIST_SIZE_MAX, "all bucket slots used"); - - // No bucket has been reused so all reuses are equal and the first is evicted. The evicted bucket still holds a retained - // block, which is returned to the allocator. The reuse of every remaining bucket is decremented. - TEST_RESULT_BOOL(memFreeList.bucket[1].blockListSize > 0, true, "bucket to evict has a retained block"); - - TEST_RESULT_VOID(memFree(memNew(1024 * 1024)), "new size evicts a bucket"); - TEST_RESULT_UINT(memFreeList.bucket[1].size, sizeof(MemContextAlloc) + 1024 * 1024, "evicted slot reused for new size"); - TEST_RESULT_UINT(memFreeList.bucketListSize, MEM_FREE_LIST_SIZE_MAX, "bucket count unchanged"); - TEST_RESULT_UINT(memFreeList.bucket[2].reuse, 0, "reuse of remaining bucket decremented"); - - // The bucket that was just cached has a higher count than the eroded buckets so the next new size evicts a different - // bucket rather than stealing the slot back, which would leave the two sizes trading the same slot and never settling - TEST_RESULT_VOID(memFree(memNew(2 * 1024 * 1024)), "another new size evicts a different bucket"); - TEST_RESULT_UINT(memFreeList.bucket[1].size, sizeof(MemContextAlloc) + 1024 * 1024, "just cached bucket survives"); - TEST_RESULT_UINT( - memFreeList.bucket[2].size, sizeof(MemContextAlloc) + 2 * 1024 * 1024, "next slot reused for new size"); - - // The reserved size still recycles even when all bucket slots are used - uint8_t *const reserved = memNew(64 * 1024); - memFree(reserved); - TEST_RESULT_PTR(memNew(64 * 1024), reserved, "reserved size recycled even when all buckets full"); - - // Freeing the context recycles its remaining live large allocations through the free list - TEST_RESULT_VOID(memContextSwitch(memContextTop()), "switch to top"); - TEST_RESULT_VOID(memContextFree(memContext), "context free recycles large allocations"); } // *****************************************************************************************************************************

view thread (13+ messages)  latest in thread

Message-ID: <gh-pgbackrest-pgbackrest-2914@github.com>
Permalink:  ../gh-pgbackrest-pgbackrest-2914@github.com/

 ·  · 


This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox