Feedback-ID: id4a34324:Fastmail
Date: Wed, 14 Jan 2026 11:30:43 -0500
From: Andres Freund <andres@anarazel.de>
To: Chao Li <li.evan.chao@gmail.com>
Cc: Kirill Reshke <reshkekirill@gmail.com>, 
	Heikki Linnakangas <hlinnaka@iki.fi>, Melanie Plageman <melanieplageman@gmail.com>, 
	Matthias van de Meent <boekewurm+postgres@gmail.com>, pgsql-hackers@postgresql.org, 
	Thomas Munro <thomas.munro@gmail.com>, Noah Misch <noah@leadboat.com>, 
	Robert Haas <robertmhaas@gmail.com>, Michael Paquier <michael.paquier@gmail.com>
Subject: Re: Buffer locking is special (hints, checksums, AIO writes)
Message-ID: <vrnib3bf726ht72jlccu252qm65lzexutydtagytstsjszpiya@ece6xi7o4nfx>
References: <q3ybyaiownp4ivt6xfy55s5llvtq6vqsd4zbt5kn6hacxdcpp4@2ngd4h4bev5d>
 <dsbacri4xldlobl3jbqqlksefsyrwj7lzljbohxsbqjus3z4o7@otb6rge4w7az>
 <1108f18d-cf7c-4f17-b29c-a119fe42f7e5@iki.fi>
 <5dwlfu2jyzkyf3nrlzxxblxctb6xio5es73ptgsahjnmfu5miu@772rc764hfhi>
 <ossv2eistssmubfsir6xjll76tynvxv5lup4zkrfzjkud7fycw@rf5vii6l6cha>
 <4csodkvvfbfloxxjlkgsnl2lgfv2mtzdl7phqzd4jxjadxm4o5@usw7feyb5bzf>
 <CALdSSPgyc7VuMLUZ8J7v2G-PebBa_vEi+mp-cLkcOwNycB56Hw@mail.gmail.com>
 <kjdgvws34gpnz4y6xu5aaeul5mspgy3ahvyiw4lndb3ecsacdb@b2ccyowotpqh>
 <jtg5cu4n6h5lib3kzx66ju4yhh6kmviaud7oq6dtut6c4q4rdi@xwsfoagt3c2b>
 <9C2494E1-F446-42DF-B070-7E231B8E6221@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <9C2494E1-F446-42DF-B070-7E231B8E6221@gmail.com>
Archived-At: <https://www.postgresql.org/message-id/vrnib3bf726ht72jlccu252qm65lzexutydtagytstsjszpiya%40ece6xi7o4nfx>
Precedence: bulk

Hi,

On 2026-01-14 11:41:19 +0800, Chao Li wrote:
> Basically, code changes in 0003 is straightforward, just a couple of small comments:
> 
> 1
> ```
> - * refcounts in buf_internals.h.  This limitation could be lifted by using a
> - * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
> - * currently realistic configurations. Even if that limitation were removed,
> - * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
> - * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
> - * 4*MaxBackends without any overflow check.  We check that the configured
> - * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
> + * refcounts in buf_internals.h.  This limitation could be lifted, but it's
> ```
> 
> Before this patch, there was room for lifting the limitation. With this
> patch, state is 64bit already, but the significant 32bit will be used for
> buffer locking as stated in buf_internals.h, in other words, there is no
> room for lifting the limitation now. If that’s true, then I think we can
> remove the statements about lifting limitation.

I'm not following - there's plenty space for more bits if we need that:

 * State of the buffer itself (in order):
 * - 18 bits refcount
 * - 4 bits usage count
 * - 12 bits of flags
 * - 18 bits share-lock count
 * - 1 bit share-exclusive locked
 * - 1 bit exclusive locked

That's 54 bits in total. Which part is in the lower and which in the upper
32bit isn't relevant for anything afaict?


> 2. By searching for “LockBufHdr”, I found one place missed to update in contrib/pg_prewarm/autoprewarm.c at line 706:
> ```
> 	for (num_blocks = 0, i = 0; i < NBuffers; i++)
> 	{
> 		uint32		buf_state; <=== line 706, should change to uint64
> 
> 		CHECK_FOR_INTERRUPTS();
> 
> 		bufHdr = GetBufferDescriptor(i);
> 
> 		/* Lock each buffer header before inspecting. */
> 		buf_state = LockBufHdr(bufHdr);
> ```

Good catch!  I didn't find any other similar omissions...


> I will continue reviewing 0004 tomorrow.

Cool.

I'd like to push

  bufmgr: Change BufferDesc.state to be a 64-bit atomic
  bufmgr: Implement buffer content locks independently of lwlocks

pretty soon, so that we then can concentrate on

  Require share-exclusive lock to set hint bits and to flush

and then subsequently on

  WIP: bufmgr: Don't copy pages while writing out

as there are other patches that have this work as a dependency...

Greetings,

Andres Freund