public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v6 1/3] Move a few ResourceOwnerEnlarge() calls for safety and clarity.
6+ messages / 5 participants
[nested] [flat]
* [PATCH v6 1/3] Move a few ResourceOwnerEnlarge() calls for safety and clarity.
@ 2021-01-13 10:21 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 6+ messages in thread
From: Heikki Linnakangas @ 2021-01-13 10:21 UTC (permalink / raw)
These are functions where quite a lot of things happen between the
ResourceOwnerEnlarge and ResourceOwnerRemember calls. It's important that
there are no unrelated ResourceOwnerRemember() calls in the code
inbetween, otherwise the entry reserved by the ResourceOwnerEnlarge() call
might be used up by the intervening ResourceOwnerRemember() and not be
available at the intended ResourceOwnerRemember() call anymore. The longer
the code path between them is, the harder it is to verify that.
In bufmgr.c, there is a function similar to ResourceOwnerEnlarge(),
to ensure that the private refcount array has enough space. The
ReservePrivateRefCountEntry() calls, analogous to ResourceOwnerEnlarge(),
were made at different places than the ResourceOwnerEnlarge() calls.
Move the ResourceOwnerEnlarge() calls together with the
ReservePrivateRefCountEntry() calls for consistency.
---
src/backend/storage/buffer/bufmgr.c | 39 +++++++++++----------------
src/backend/storage/buffer/localbuf.c | 3 +++
src/backend/utils/cache/catcache.c | 13 ++++++---
3 files changed, 28 insertions(+), 27 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 561c212092f..cde869e7d64 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -738,9 +738,6 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*hit = false;
- /* Make sure we will have room to remember the buffer pin */
- ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
-
isExtend = (blockNum == P_NEW);
TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
@@ -1093,9 +1090,11 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* Ensure, while the spinlock's not yet held, that there's a free
- * refcount entry.
+ * refcount entry and that the resoure owner has room to remember the
+ * pin.
*/
ReservePrivateRefCountEntry();
+ ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
/*
* Select a victim buffer. The buffer is returned with its header
@@ -1595,8 +1594,6 @@ ReleaseAndReadBuffer(Buffer buffer,
* taking the buffer header lock; instead update the state variable in loop of
* CAS operations. Hopefully it's just a single CAS.
*
- * Note that ResourceOwnerEnlargeBuffers must have been done already.
- *
* Returns true if buffer is BM_VALID, else false. This provision allows
* some callers to avoid an extra spinlock cycle.
*/
@@ -1607,6 +1604,8 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
bool result;
PrivateRefCountEntry *ref;
+ ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
+
ref = GetPrivateRefCountEntry(b, true);
if (ref == NULL)
@@ -1687,7 +1686,8 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* The spinlock is released before return.
*
* As this function is called with the spinlock held, the caller has to
- * previously call ReservePrivateRefCountEntry().
+ * previously call ReservePrivateRefCountEntry() and
+ * ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
*
* Currently, no callers of this function want to modify the buffer's
* usage_count at all, so there's no need for a strategy parameter.
@@ -1857,9 +1857,6 @@ BufferSync(int flags)
int mask = BM_DIRTY;
WritebackContext wb_context;
- /* Make sure we can handle the pin inside SyncOneBuffer */
- ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
-
/*
* Unless this is a shutdown checkpoint or we have been explicitly told,
* we write only permanent, dirty buffers. But at shutdown or end of
@@ -2334,9 +2331,6 @@ BgBufferSync(WritebackContext *wb_context)
* requirements, or hit the bgwriter_lru_maxpages limit.
*/
- /* Make sure we can handle the pin inside SyncOneBuffer */
- ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
-
num_to_scan = bufs_to_lap;
num_written = 0;
reusable_buffers = reusable_buffers_est;
@@ -2418,8 +2412,6 @@ BgBufferSync(WritebackContext *wb_context)
*
* (BUF_WRITTEN could be set in error if FlushBuffer finds the buffer clean
* after locking it, but we don't care all that much.)
- *
- * Note: caller must have done ResourceOwnerEnlargeBuffers.
*/
static int
SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context)
@@ -2429,7 +2421,9 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context)
uint32 buf_state;
BufferTag tag;
+ /* Make sure we can handle the pin */
ReservePrivateRefCountEntry();
+ ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
/*
* Check whether buffer needs writing.
@@ -3487,9 +3481,6 @@ FlushRelationBuffers(Relation rel)
return;
}
- /* Make sure we can handle the pin inside the loop */
- ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
-
for (i = 0; i < NBuffers; i++)
{
uint32 buf_state;
@@ -3503,7 +3494,9 @@ FlushRelationBuffers(Relation rel)
if (!RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node))
continue;
+ /* Make sure we can handle the pin */
ReservePrivateRefCountEntry();
+ ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
buf_state = LockBufHdr(bufHdr);
if (RelFileNodeEquals(bufHdr->tag.rnode, rel->rd_node) &&
@@ -3560,9 +3553,6 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels)
if (use_bsearch)
pg_qsort(srels, nrels, sizeof(SMgrSortArray), rnode_comparator);
- /* Make sure we can handle the pin inside the loop */
- ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
-
for (i = 0; i < NBuffers; i++)
{
SMgrSortArray *srelent = NULL;
@@ -3599,7 +3589,9 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels)
if (srelent == NULL)
continue;
+ /* Make sure we can handle the pin */
ReservePrivateRefCountEntry();
+ ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
buf_state = LockBufHdr(bufHdr);
if (RelFileNodeEquals(bufHdr->tag.rnode, srelent->rnode) &&
@@ -3639,9 +3631,6 @@ FlushDatabaseBuffers(Oid dbid)
int i;
BufferDesc *bufHdr;
- /* Make sure we can handle the pin inside the loop */
- ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
-
for (i = 0; i < NBuffers; i++)
{
uint32 buf_state;
@@ -3655,7 +3644,9 @@ FlushDatabaseBuffers(Oid dbid)
if (bufHdr->tag.rnode.dbNode != dbid)
continue;
+ /* Make sure we can handle the pin */
ReservePrivateRefCountEntry();
+ ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
buf_state = LockBufHdr(bufHdr);
if (bufHdr->tag.rnode.dbNode == dbid &&
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 04b3558ea33..f7c15ea8a44 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -123,6 +123,9 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
if (LocalBufHash == NULL)
InitLocalBuffers();
+ /* Make sure we will have room to remember the buffer pin */
+ ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
+
/* See if the desired buffer already exists */
hresult = (LocalBufferLookupEnt *)
hash_search(LocalBufHash, (void *) &newTag, HASH_FIND, NULL);
diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c
index fa2b49c676e..86fdfc454d2 100644
--- a/src/backend/utils/cache/catcache.c
+++ b/src/backend/utils/cache/catcache.c
@@ -1607,8 +1607,6 @@ SearchCatCacheList(CatCache *cache,
* block to ensure we can undo those refcounts if we get an error before
* we finish constructing the CatCList.
*/
- ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner);
-
ctlist = NIL;
PG_TRY();
@@ -1696,13 +1694,22 @@ SearchCatCacheList(CatCache *cache,
table_close(relation, AccessShareLock);
+ /* Make sure the resource owner has room to remember this entry. */
+ ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner);
+
/* Now we can build the CatCList entry. */
oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
nmembers = list_length(ctlist);
cl = (CatCList *)
palloc(offsetof(CatCList, members) + nmembers * sizeof(CatCTup *));
- /* Extract key values */
+ /*
+ * Extract key values.
+ *
+ * XXX: If we run out of memory while copying the key values, we will
+ * leak any allocations we had already made in the CacheMemoryContext.
+ * That is unlikely enough that we just accept the risk.
+ */
CatCacheCopyKeys(cache->cc_tupdesc, nkeys, cache->cc_keyno,
arguments, cl->keys);
MemoryContextSwitchTo(oldcxt);
--
2.29.2
--------------F30A5443A5D524A5CBC51AE7
Content-Type: text/x-patch; charset=UTF-8;
name="v6-0002-Make-resowners-more-easily-extensible.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="v6-0002-Make-resowners-more-easily-extensible.patch"
^ permalink raw reply [nested|flat] 6+ messages in thread
* Add missing includes
@ 2023-05-22 15:10 Tristan Partin <[email protected]>
2023-05-22 15:16 ` Re: Add missing includes Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Tristan Partin @ 2023-05-22 15:10 UTC (permalink / raw)
To: pgsql-hackers
Hi,
Some files were missing information from the c.h header.
--
Tristan Partin
Neon (https://neon.tech)
Attachments:
[text/x-patch] v1-0001-Add-some-missing-includes.patch (1.0K, ../../CSSWHSSN4B6S.14UNR4TB14WOU@c3po/2-v1-0001-Add-some-missing-includes.patch)
download | inline diff:
From 3c0c32077bb9626d4ebad1452b9fc9937c99518d Mon Sep 17 00:00:00 2001
From: Tristan Partin <[email protected]>
Date: Mon, 22 May 2023 10:07:45 -0500
Subject: [PATCH postgres v1] Add some missing includes
Both files were missing defines/types from c.h. Issue found through
usage of clangd.
---
src/bin/psql/settings.h | 1 +
src/include/fe_utils/cancel.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 73d4b393bc..f429ca6ad9 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -8,6 +8,7 @@
#ifndef SETTINGS_H
#define SETTINGS_H
+#include "c.h"
#include "fe_utils/print.h"
#include "variables.h"
diff --git a/src/include/fe_utils/cancel.h b/src/include/fe_utils/cancel.h
index 6e8b9ddfc4..733aaf7751 100644
--- a/src/include/fe_utils/cancel.h
+++ b/src/include/fe_utils/cancel.h
@@ -16,6 +16,7 @@
#include <signal.h>
+#include "c.h"
#include "libpq-fe.h"
extern PGDLLIMPORT volatile sig_atomic_t CancelRequested;
--
--
Tristan Partin
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Add missing includes
2023-05-22 15:10 Add missing includes Tristan Partin <[email protected]>
@ 2023-05-22 15:16 ` Alvaro Herrera <[email protected]>
2023-05-22 15:27 ` Re: Add missing includes Tristan Partin <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Alvaro Herrera @ 2023-05-22 15:16 UTC (permalink / raw)
To: Tristan Partin <[email protected]>; +Cc: pgsql-hackers
Hi,
On 2023-May-22, Tristan Partin wrote:
> Some files were missing information from the c.h header.
Actually, these omissions are intentional, and we have bespoke handling
for this in our own header-checking scripts (src/tools/pginclude). I
imagine this is documented somewhere, but ATM I can't remember where.
(And if not, maybe that's something we should do.)
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Add missing includes
2023-05-22 15:10 Add missing includes Tristan Partin <[email protected]>
2023-05-22 15:16 ` Re: Add missing includes Alvaro Herrera <[email protected]>
@ 2023-05-22 15:27 ` Tristan Partin <[email protected]>
2023-05-22 15:34 ` Re: Add missing includes Tom Lane <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Tristan Partin @ 2023-05-22 15:27 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: pgsql-hackers
On Mon May 22, 2023 at 10:16 AM CDT, Alvaro Herrera wrote:
> > Some files were missing information from the c.h header.
>
> Actually, these omissions are intentional, and we have bespoke handling
> for this in our own header-checking scripts (src/tools/pginclude). I
> imagine this is documented somewhere, but ATM I can't remember where.
> (And if not, maybe that's something we should do.)
Interesting. Thanks for the information. Thought I had seen in a
different email thread that header files were supposed to include all
that they needed to. Anyways, ignore the patch :).
--
Tristan Partin
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Add missing includes
2023-05-22 15:10 Add missing includes Tristan Partin <[email protected]>
2023-05-22 15:16 ` Re: Add missing includes Alvaro Herrera <[email protected]>
2023-05-22 15:27 ` Re: Add missing includes Tristan Partin <[email protected]>
@ 2023-05-22 15:34 ` Tom Lane <[email protected]>
2023-05-22 22:55 ` Re: Add missing includes Michael Paquier <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Tom Lane @ 2023-05-22 15:34 UTC (permalink / raw)
To: Tristan Partin <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; pgsql-hackers
"Tristan Partin" <[email protected]> writes:
> Interesting. Thanks for the information. Thought I had seen in a
> different email thread that header files were supposed to include all
> that they needed to. Anyways, ignore the patch :).
There is such a policy, but not with respect to those particular
headers. You might find src/tools/pginclude/headerscheck and
src/tools/pginclude/cpluspluscheck to be interesting reading,
as those are the tests we run to verify this policy.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Add missing includes
2023-05-22 15:10 Add missing includes Tristan Partin <[email protected]>
2023-05-22 15:16 ` Re: Add missing includes Alvaro Herrera <[email protected]>
2023-05-22 15:27 ` Re: Add missing includes Tristan Partin <[email protected]>
2023-05-22 15:34 ` Re: Add missing includes Tom Lane <[email protected]>
@ 2023-05-22 22:55 ` Michael Paquier <[email protected]>
0 siblings, 0 replies; 6+ messages in thread
From: Michael Paquier @ 2023-05-22 22:55 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Tristan Partin <[email protected]>; Alvaro Herrera <[email protected]>; pgsql-hackers
On Mon, May 22, 2023 at 11:34:14AM -0400, Tom Lane wrote:
> There is such a policy, but not with respect to those particular
> headers. You might find src/tools/pginclude/headerscheck and
> src/tools/pginclude/cpluspluscheck to be interesting reading,
> as those are the tests we run to verify this policy.
The standalone compile policy is documented in
src/tools/pginclude/README, AFAIK, so close enough :p
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2023-05-22 22:55 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-01-13 10:21 [PATCH v6 1/3] Move a few ResourceOwnerEnlarge() calls for safety and clarity. Heikki Linnakangas <[email protected]>
2023-05-22 15:10 Add missing includes Tristan Partin <[email protected]>
2023-05-22 15:16 ` Re: Add missing includes Alvaro Herrera <[email protected]>
2023-05-22 15:27 ` Re: Add missing includes Tristan Partin <[email protected]>
2023-05-22 15:34 ` Re: Add missing includes Tom Lane <[email protected]>
2023-05-22 22:55 ` Re: Add missing includes Michael Paquier <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox