public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v5 1/2] Move a few ResourceOwnerEnlarge() calls for safety and clarity. 6+ messages / 3 participants [nested] [flat]
* [PATCH v5 1/2] Move a few ResourceOwnerEnlarge() calls for safety and clarity. @ 2021-01-13 10:21 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2021-01-13 10:21 UTC (permalink / raw) These are functions where quite a lot of things happen between the ResourceOwnerEnlarge and ResourceOwnerRemember calls. It's important that there are no unrelated ResourceOwnerRemember() calls in the code inbetween, otherwise the entry reserved by the ResourceOwnerEnlarge() call might be used up by the intervening ResourceOwnerRemember() and not be available at the intended ResourceOwnerRemember() call anymore. The longer the code path between them is, the harder it is to verify that. In bufmgr.c, there is a function similar to ResourceOwnerEnlarge(), to ensure that the private refcount array has enough space. The ReservePrivateRefCountEntry() calls, analogous to ResourceOwnerEnlarge(), were made at different places than the ResourceOwnerEnlarge() calls. Move the ResourceOwnerEnlarge() calls together with the ReservePrivateRefCountEntry() calls for consistency. --- src/backend/storage/buffer/bufmgr.c | 39 +++++++++++---------------- src/backend/storage/buffer/localbuf.c | 3 +++ src/backend/utils/cache/catcache.c | 13 ++++++--- 3 files changed, 28 insertions(+), 27 deletions(-) diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index 561c212092f..cde869e7d64 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -738,9 +738,6 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, *hit = false; - /* Make sure we will have room to remember the buffer pin */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); - isExtend = (blockNum == P_NEW); TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum, @@ -1093,9 +1090,11 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum, { /* * Ensure, while the spinlock's not yet held, that there's a free - * refcount entry. + * refcount entry and that the resoure owner has room to remember the + * pin. */ ReservePrivateRefCountEntry(); + ResourceOwnerEnlargeBuffers(CurrentResourceOwner); /* * Select a victim buffer. The buffer is returned with its header @@ -1595,8 +1594,6 @@ ReleaseAndReadBuffer(Buffer buffer, * taking the buffer header lock; instead update the state variable in loop of * CAS operations. Hopefully it's just a single CAS. * - * Note that ResourceOwnerEnlargeBuffers must have been done already. - * * Returns true if buffer is BM_VALID, else false. This provision allows * some callers to avoid an extra spinlock cycle. */ @@ -1607,6 +1604,8 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) bool result; PrivateRefCountEntry *ref; + ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + ref = GetPrivateRefCountEntry(b, true); if (ref == NULL) @@ -1687,7 +1686,8 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy) * The spinlock is released before return. * * As this function is called with the spinlock held, the caller has to - * previously call ReservePrivateRefCountEntry(). + * previously call ReservePrivateRefCountEntry() and + * ResourceOwnerEnlargeBuffers(CurrentResourceOwner); * * Currently, no callers of this function want to modify the buffer's * usage_count at all, so there's no need for a strategy parameter. @@ -1857,9 +1857,6 @@ BufferSync(int flags) int mask = BM_DIRTY; WritebackContext wb_context; - /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); - /* * Unless this is a shutdown checkpoint or we have been explicitly told, * we write only permanent, dirty buffers. But at shutdown or end of @@ -2334,9 +2331,6 @@ BgBufferSync(WritebackContext *wb_context) * requirements, or hit the bgwriter_lru_maxpages limit. */ - /* Make sure we can handle the pin inside SyncOneBuffer */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); - num_to_scan = bufs_to_lap; num_written = 0; reusable_buffers = reusable_buffers_est; @@ -2418,8 +2412,6 @@ BgBufferSync(WritebackContext *wb_context) * * (BUF_WRITTEN could be set in error if FlushBuffer finds the buffer clean * after locking it, but we don't care all that much.) - * - * Note: caller must have done ResourceOwnerEnlargeBuffers. */ static int SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) @@ -2429,7 +2421,9 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) uint32 buf_state; BufferTag tag; + /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); + ResourceOwnerEnlargeBuffers(CurrentResourceOwner); /* * Check whether buffer needs writing. @@ -3487,9 +3481,6 @@ FlushRelationBuffers(Relation rel) return; } - /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); - for (i = 0; i < NBuffers; i++) { uint32 buf_state; @@ -3503,7 +3494,9 @@ FlushRelationBuffers(Relation rel) if (!RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node)) continue; + /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); + ResourceOwnerEnlargeBuffers(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) && @@ -3560,9 +3553,6 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) if (use_bsearch) pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator); - /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); - for (i = 0; i < NBuffers; i++) { SMgrSortArray *srelent = NULL; @@ -3599,7 +3589,9 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels) if (srelent == NULL) continue; + /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); + ResourceOwnerEnlargeBuffers(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) && @@ -3639,9 +3631,6 @@ FlushDatabaseBuffers(Oid dbid) int i; BufferDesc *bufHdr; - /* Make sure we can handle the pin inside the loop */ - ResourceOwnerEnlargeBuffers(CurrentResourceOwner); - for (i = 0; i < NBuffers; i++) { uint32 buf_state; @@ -3655,7 +3644,9 @@ FlushDatabaseBuffers(Oid dbid) if (bufHdr->tag.rnode.dbNode != dbid) continue; + /* Make sure we can handle the pin */ ReservePrivateRefCountEntry(); + ResourceOwnerEnlargeBuffers(CurrentResourceOwner); buf_state = LockBufHdr(bufHdr); if (bufHdr->tag.rnode.dbNode == dbid && diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 04b3558ea33..f7c15ea8a44 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -123,6 +123,9 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, if (LocalBufHash == NULL) InitLocalBuffers(); + /* Make sure we will have room to remember the buffer pin */ + ResourceOwnerEnlargeBuffers(CurrentResourceOwner); + /* See if the desired buffer already exists */ hresult = (LocalBufferLookupEnt *) hash_search(LocalBufHash, (void *) &newTag, HASH_FIND, NULL); diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index fa2b49c676e..86fdfc454d2 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -1607,8 +1607,6 @@ SearchCatCacheList(CatCache *cache, * block to ensure we can undo those refcounts if we get an error before * we finish constructing the CatCList. */ - ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); - ctlist = NIL; PG_TRY(); @@ -1696,13 +1694,22 @@ SearchCatCacheList(CatCache *cache, table_close(relation, AccessShareLock); + /* Make sure the resource owner has room to remember this entry. */ + ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); + /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); nmembers = list_length(ctlist); cl = (CatCList *) palloc(offsetof(CatCList, members) + nmembers * sizeof(CatCTup *)); - /* Extract key values */ + /* + * Extract key values. + * + * XXX: If we run out of memory while copying the key values, we will + * leak any allocations we had already made in the CacheMemoryContext. + * That is unlikely enough that we just accept the risk. + */ CatCacheCopyKeys(cache->cc_tupdesc, nkeys, cache->cc_keyno, arguments, cl->keys); MemoryContextSwitchTo(oldcxt); -- 2.29.2 --------------1104342304EC6341F5D7C60C Content-Type: text/x-patch; charset=UTF-8; name="v5-0002-Make-resowners-more-easily-extensible.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="v5-0002-Make-resowners-more-easily-extensible.patch" ^ permalink raw reply [nested|flat] 6+ messages in thread
* Fix resource leak (src/backend/libpq/be-secure-common.c) @ 2024-04-02 18:13 Ranier Vilela <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Ranier Vilela @ 2024-04-02 18:13 UTC (permalink / raw) To: pgsql-hackers Hi, Per Coverity. Coverity reported a resource leak at the function run_ssl_passphrase_command. 7. alloc_fn: Storage is returned from allocation function wait_result_to_str.["show details"] 8. noescape: Assuming resource wait_result_to_str(pclose_rc) is not freed or pointed-to as ellipsis argument to errdetail_internal. CID 1533043: (#1 of 1): Resource leak (RESOURCE_LEAK) 9. leaked_storage: Failing to save or free storage allocated by wait_result_to_str(pclose_rc) leaks it. I think that Coverity is right. Fix by freeing the pointer, like pclose_check (src/common/exec.c) similar case. Patch attached. best regards, Attachments: [application/octet-stream] fix-resource-leak-be-secure-common.patch (697B, ../../CAEudQAqMeE0AHcOsOzZx51Z0eiFJAjhBPRFt+Bxi3ETXWen7ig@mail.gmail.com/3-fix-resource-leak-be-secure-common.patch) download | inline diff: diff --git a/src/backend/libpq/be-secure-common.c b/src/backend/libpq/be-secure-common.c index 0582606192..a67a5e8619 100644 --- a/src/backend/libpq/be-secure-common.c +++ b/src/backend/libpq/be-secure-common.c @@ -85,12 +85,16 @@ run_ssl_passphrase_command(const char *prompt, bool is_server_start, char *buf, } else if (pclose_rc != 0) { + const char * reason; + explicit_bzero(buf, size); + reason = wait_result_to_str(pclose_rc); ereport(loglevel, (errcode_for_file_access(), errmsg("command \"%s\" failed", command), - errdetail_internal("%s", wait_result_to_str(pclose_rc)))); + errdetail_internal("%s", reason))); + pfree(reason); goto error; } ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix resource leak (src/backend/libpq/be-secure-common.c) @ 2024-04-02 18:31 Daniel Gustafsson <[email protected]> parent: Ranier Vilela <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Daniel Gustafsson @ 2024-04-02 18:31 UTC (permalink / raw) To: Ranier Vilela <[email protected]>; +Cc: pgsql-hackers > On 2 Apr 2024, at 20:13, Ranier Vilela <[email protected]> wrote: > Fix by freeing the pointer, like pclose_check (src/common/exec.c) similar case. Off the cuff, seems reasonable when loglevel is LOG. -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix resource leak (src/backend/libpq/be-secure-common.c) @ 2024-04-10 18:31 Ranier Vilela <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Ranier Vilela @ 2024-04-10 18:31 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; +Cc: pgsql-hackers Em ter., 2 de abr. de 2024 às 15:31, Daniel Gustafsson <[email protected]> escreveu: > > On 2 Apr 2024, at 20:13, Ranier Vilela <[email protected]> wrote: > > > Fix by freeing the pointer, like pclose_check (src/common/exec.c) > similar case. > > Off the cuff, seems reasonable when loglevel is LOG. > Per Coverity. Another case of resource leak, when loglevel is LOG. In the function shell_archive_file (src/backend/archive/shell_archive.c) The pointer *xlogarchcmd* is not freed. best regards, Ranier Vilela Attachments: [application/octet-stream] fix-resource-leak-shell_archive.patch (525B, ../../CAEudQArcwFzYdGcRk8Tyf2ij0DR7mOoqSJejsa00+SkTjFNHSw@mail.gmail.com/3-fix-resource-leak-shell_archive.patch) download | inline diff: diff --git a/src/backend/archive/shell_archive.c b/src/backend/archive/shell_archive.c index 0925348bfe..506c5a30ad 100644 --- a/src/backend/archive/shell_archive.c +++ b/src/backend/archive/shell_archive.c @@ -125,9 +125,11 @@ shell_archive_file(ArchiveModuleState *state, const char *file, errdetail("The failed archive command was: %s", xlogarchcmd))); } + pfree(xlogarchcmd); return false; } + pfree(xlogarchcmd); elog(DEBUG1, "archived write-ahead log file \"%s\"", file); return true; ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix resource leak (src/backend/libpq/be-secure-common.c) @ 2024-04-10 18:33 Daniel Gustafsson <[email protected]> parent: Ranier Vilela <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Daniel Gustafsson @ 2024-04-10 18:33 UTC (permalink / raw) To: Ranier Vilela <[email protected]>; +Cc: pgsql-hackers > On 10 Apr 2024, at 20:31, Ranier Vilela <[email protected]> wrote: > > Em ter., 2 de abr. de 2024 às 15:31, Daniel Gustafsson <[email protected] <mailto:[email protected]>> escreveu: >> > On 2 Apr 2024, at 20:13, Ranier Vilela <[email protected] <mailto:[email protected]>> wrote: >> >> > Fix by freeing the pointer, like pclose_check (src/common/exec.c) similar case. >> >> Off the cuff, seems reasonable when loglevel is LOG. > > Per Coverity. > > Another case of resource leak, when loglevel is LOG. > In the function shell_archive_file (src/backend/archive/shell_archive.c) > The pointer *xlogarchcmd* is not freed. Thanks, I'll have a look. I've left this for post-freeze on purpose to not cause unnecessary rebasing. Will take a look over the next few days unless beaten to it. -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Fix resource leak (src/backend/libpq/be-secure-common.c) @ 2024-05-13 18:05 Ranier Vilela <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Ranier Vilela @ 2024-05-13 18:05 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; +Cc: pgsql-hackers Em qua., 10 de abr. de 2024 às 15:33, Daniel Gustafsson <[email protected]> escreveu: > On 10 Apr 2024, at 20:31, Ranier Vilela <[email protected]> wrote: > > Em ter., 2 de abr. de 2024 às 15:31, Daniel Gustafsson <[email protected]> > escreveu: > >> > On 2 Apr 2024, at 20:13, Ranier Vilela <[email protected]> wrote: >> >> > Fix by freeing the pointer, like pclose_check (src/common/exec.c) >> similar case. >> >> Off the cuff, seems reasonable when loglevel is LOG. >> > > Per Coverity. > > Another case of resource leak, when loglevel is LOG. > In the function shell_archive_file (src/backend/archive/shell_archive.c) > The pointer *xlogarchcmd* is not freed. > > > Thanks, I'll have a look. I've left this for post-freeze on purpose to not > cause unnecessary rebasing. Will take a look over the next few days unless > beaten to it. > Any chance we'll have these fixes in v17? best regards, Ranier Vilela ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2024-05-13 18:05 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-01-13 10:21 [PATCH v5 1/2] Move a few ResourceOwnerEnlarge() calls for safety and clarity. Heikki Linnakangas <[email protected]> 2024-04-02 18:13 Fix resource leak (src/backend/libpq/be-secure-common.c) Ranier Vilela <[email protected]> 2024-04-02 18:31 ` Re: Fix resource leak (src/backend/libpq/be-secure-common.c) Daniel Gustafsson <[email protected]> 2024-04-10 18:31 ` Re: Fix resource leak (src/backend/libpq/be-secure-common.c) Ranier Vilela <[email protected]> 2024-04-10 18:33 ` Re: Fix resource leak (src/backend/libpq/be-secure-common.c) Daniel Gustafsson <[email protected]> 2024-05-13 18:05 ` Re: Fix resource leak (src/backend/libpq/be-secure-common.c) Ranier Vilela <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox