Subject: Buffer Cache Problem

From: jacktby jacktby <[email-masked]>
Date: Tue, 7 Nov 2023 21:28:12 +0800

Hi, postgres hackers, I’m studying postgres buffer cache part. So I open this thread to communicate some buffer cache codes design and try to improve some tricky codes.

For Buffer Cache, we know it’s a buffer array, every bucket of this array is consist of a data page and its header which is used to describe the state of the buffer. 

This is the origin code of buffer header:
typedef struct BufferDesc
{
	BufferTag	tag;			/* ID of page contained in buffer */
	int			buf_id;			/* buffer's index number (from 0) */

	/* state of the tag, containing flags, refcount and usagecount */
	pg_atomic_uint32 state;

	int			wait_backend_pgprocno;	/* backend of pin-count waiter */
	int			freeNext;		/* link in freelist chain */
	LWLock		content_lock;	/* to lock access to buffer contents */
} BufferDesc;

For field wait_backend_pgprocno, the comment is "backend of pin-count waiter”, I have problems below:
1. it means which processId is waiting this buffer, right? 
2. and if wait_backend_pgprocno is valid, so it says this buffer is in use by one process, right?
3. if one buffer is wait by another process, it means all buffers are out of use, right? So let’s try this: we have 5 buffers with ids (1,2,3,4,5), and they  are all in use, now another process  with processId 8017 is coming, and it choose buffer id 1, so  buffer1’s wait_backend_pgprocno is 8017, but later
buffer4 is released, can process 8017 change to get buffer4? how?
4. wait_backend_pgprocno is a “integer” type, not an array, why can one buffer be wait by only one process?

Hope your reply, thanks!! I’m willing to do contributions after I study buffer cache implementations.

---

From: Matthias van de Meent <[email-masked]>
Date: Tue, 7 Nov 2023 16:45:19 +0100

> For field wait_backend_pgprocno, the comment is "backend of pin-count waiter”, I have problems below:

Did you read the README at src/backend/storage/buffer/README, as well
as the comments and documentation in and around the buffer-locking
functions?

> buffer4 is released, can process 8017 change to get buffer4? how?

I believe these questions are generally answered by the README and the
comments in bufmgr.c/buf_internal.h for the functions that try to lock
buffers.

> 4. wait_backend_pgprocno is a “integer” type, not an array, why can one buffer be wait by only one process?

Yes, that is correct. It seems like PostgreSQL has yet to find a
workload requires more than one backend to wait for super exclusive
access to a buffer at the same time.
VACUUM seems to be the only workload that currently can wait and sleep
for this exclusive buffer access, and that is already limited to one
process per relation, so there are no explicit concurrent
super-exclusive waits in the system right now.

Kind regards,

Matthias van de Meent
Neon (https://neon.tech)