public inbox for [email protected]
help / color / mirror / Atom feed[bug report] The backend process cannot reuse VfdCache cache entries
2+ messages / 2 participants
[nested] [flat]
* [bug report] The backend process cannot reuse VfdCache cache entries
@ 2026-05-13 13:06 =?utf-8?B?ZGVuZ2thaQ==?= <[email protected]>
2026-05-13 13:41 ` Re: [bug report] The backend process cannot reuse VfdCache cache entries Heikki Linnakangas <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: =?utf-8?B?ZGVuZ2thaQ==?= @ 2026-05-13 13:06 UTC (permalink / raw)
To: =?utf-8?B?cGdzcWwtYnVncw==?= <[email protected]>
Hi All，
Recently, in a production environment, we encountered the following SQL error:
`ERROR: dn_6001_6002: dn_6011_6012: invalid memory alloc request size 1174405120 in fd.cpp:1094`
Based on this error message, the database was trying to allocate a VfdCache of size 1174405120 bytes, which exceeds the maximum allowable memory space `maxCacheSize` for a single allocation.
My analysis:
- The error occurred at the moment VfdCache was being expanded. According to the VfdCache expansion strategy, we can deduce that the system had approximately 9.4 million files open at that moment (or 9.4 million cache entries allocated). From the error size and :
newCacheSize = 1174405120 = 10485760 * 112
- At the time of the error, the system needed to expand VfdCache to 10,485,760 VFD entries, which must be the result of expansion strategy 2.
- The original VfdCache size `SizeVfdCache` would be: `10485760 - 1048576 = 9437184`, meaning there were already over 9.4 million VFD entries present before expansion.
## Why are there so many VFD entries?
**Root cause:**
The free-list pointer of VfdCache does not reuse old slots. Whenever a session opens a new file, it occupies a new slot. `VfdCache[0].nextFree` never goes backwards. It keeps moving forward. After opening a certain number of files, it repeatedly triggers VfdCache expansion, and when the size exceeds the threshold, an error is reported.
## GDB debugging information
```sql
# Prepare three tables
postgres=# select oid, relfilenode from pg_class where relname in ('t1','t2','t3');
oid | relfilenode
-------+-------------
16384 | 16384
24576 | 24576
24579 | 24579
(3 rows)
# Execute the following three SQL statements sequentially in the same session
postgres=# select * from t1;
postgres=# select * from t2;
postgres=# select * from t3;
# After first sql executed , VFD cache information at transaction commit
(gdb) p VfdCache[0]
(gdb) p VfdCache[0]
$34 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 4, lruMoreRecently = 3, lruLessRecently = 3, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[1]
$35 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 2, lruMoreRecently = 2, lruLessRecently = 0, fileSize = 0, fileName = 0x31d9ec0 "base/5/2601", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[2]
$36 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 3, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x31da120 "base/5/16384", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[3]
$37 = {fd = 7, fdstate = 0, resowner = 0x0, nextFree = 4, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x31da140 "base/5/24576", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[4]
$38 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 5, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[5]
$39 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 6, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[6]
$40 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 7, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[7]
$41 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 8, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
# After second sql executed , VFD cache information at transaction commit
(gdb) p VfdCache[0]
$34 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 4, lruMoreRecently = 3, lruLessRecently = 3, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[1]
$35 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 2, lruMoreRecently = 2, lruLessRecently = 0, fileSize = 0, fileName = 0x31d9ec0 "base/5/2601", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[2]
$36 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 3, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x31da120 "base/5/16384", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[3]
$37 = {fd = 7, fdstate = 0, resowner = 0x0, nextFree = 4, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x31da140 "base/5/24576", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[4]
$38 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 5, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[5]
$39 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 6, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[6]
$40 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 7, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[7]
$41 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 8, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
# After third sql executed , VFD cache information at transaction commit
(gdb) p VfdCache[0]
$67 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 7, lruMoreRecently = 4, lruLessRecently = 5, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[1]
$68 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 2, lruMoreRecently = 2, lruLessRecently = 0, fileSize = 0, fileName = 0x31d9ec0 "base/5/2601", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[2]
$69 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 3, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x31da120 "base/5/16384", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[3]
$70 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 4, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x31da140 "base/5/24576", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[4]
$71 = {fd = 7, fdstate = 0, resowner = 0x0, nextFree = 5, lruMoreRecently = 6, lruLessRecently = 0, fileSize = 0, fileName = 0x31da160 "base/5/1249", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[5]
$72 = {fd = 9, fdstate = 0, resowner = 0x0, nextFree = 6, lruMoreRecently = 0, lruLessRecently = 6, fileSize = 0, fileName = 0x31da180 "base/5/24579", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[6]
$73 = {fd = 14, fdstate = 0, resowner = 0x0, nextFree = 7, lruMoreRecently = 5, lruLessRecently = 4, fileSize = 0, fileName = 0x31dc5e0 "base/5/2619", fileFlags = 524290, fileMode = 384}
(gdb) p VfdCache[7]
$74 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 8, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[8]
$75 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 9, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[9]
$76 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 10, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
(gdb) p VfdCache[10]
$77 = {fd = -1, fdstate = 0, resowner = 0x0, nextFree = 11, lruMoreRecently = 0, lruLessRecently = 0, fileSize = 0, fileName = 0x0, fileFlags = 0, fileMode = 0}
## Analysis
1. When executing the first SQL statement `select * from t1;`, the physical file for table t1 (`filenode = 16384`) is stored in slot `VfdCache[2]`. At this point, the head pointer `VfdCache[0]` points to a free list where `VfdCache[0]->nextFree = 3`, meaning slots starting from 3 are free.
When executing the second SQL `select * from t2;`, the physical file for t2 (`filenode = 24576`) is stored in slot `VfdCache[3]`. Now `VfdCache[0]->nextFree = 4`, slots from 4 onward are free.
When executing the third SQL `select * from t3;`, the physical file for t3 (`filenode = 24579`) is stored in slot `VfdCache[5]` (this SQL opens 3 new files). The free list pointer becomes `VfdCache[0]->nextFree = 7`, slots from 7 onward are free.
2. During the third SQL, the file descriptors opened by previous SQL statements have been released: the physical file descriptor `VfdCache->fd` is closed and set to -1, but the VfdCache slots themselves are not released. When opening new files, the `nextFree` pointer always moves forward to new slots, and already-used slots are never reused by new files.
As a result, as long as the session connection remains open, VfdCache only grows. When the cumulative number of opened files becomes large enough, the VfdCache capacity exceeds the memory limit, causing SQL execution to fail.
3. Old VFD slots are bound to the SMGR and stored in the relation structure, which is cached in the relcache. If a subsequent SQL statement opens the same file again, it will reuse the cached VFD slot from the relcache.
4. VfdCache is at the backend process level. When the session disconnects, VfdCache is cleared. But if the connection stays open, the cache size keeps growing.
## Solutions
1. On the application side, use short-lived connections. Long-lived connections are more prone to this error.
2. Solve the problem from the database kernel side. I would like to ask the experts: is this a defect in VfdCache? If so, my idea is to make `VfdCache[0]->nextFree` point backwards when a SQL statement ends, so that the same connection can reuse old entries when opening new files.
That concludes my question. Looking forward to your reply.
BG
dengkai
[email protected]
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: [bug report] The backend process cannot reuse VfdCache cache entries
2026-05-13 13:06 [bug report] The backend process cannot reuse VfdCache cache entries =?utf-8?B?ZGVuZ2thaQ==?= <[email protected]>
@ 2026-05-13 13:41 ` Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Heikki Linnakangas @ 2026-05-13 13:41 UTC (permalink / raw)
To: dengkai <[email protected]>; pgsql-bugs <[email protected]>
On 13/05/2026 16:06, dengkai wrote:
> Recently, in a production environment, we encountered the following SQL
> error:
> `ERROR: dn_6001_6002: dn_6011_6012: invalid memory alloc request size
> 1174405120 in fd.cpp:1094`
> Based on this error message, the database was trying to allocate a
> VfdCache of size 1174405120 bytes, which exceeds the maximum allowable
> memory space `maxCacheSize` for a single allocation.
There is no file called "fd.cpp" or variable called "maxCacheSize" in
PostgreSQL.
There is code like that in fd.c to expand the VfdCache, but it uses
realloc() for the allocation, which can go beyond the 1 GB limit.
In other words, you've hit a bug on something other than PostgreSQL.
- Heikki
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2026-05-13 13:41 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-05-13 13:06 [bug report] The backend process cannot reuse VfdCache cache entries =?utf-8?B?ZGVuZ2thaQ==?= <[email protected]>
2026-05-13 13:41 ` Heikki Linnakangas <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox