public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v6 05/15] Improve sentences in overview of system configuration parameters
95+ messages / 13 participants
[nested] [flat]
* [PATCH v6 05/15] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/74urtnrsBSymuH7bJczNOGS
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v6-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v2 05/11] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/RgO6TscyR9fMvkEm1k5N=yu
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v2-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v3 05/11] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/.arW0LC=Yi8JO9BRih4YlyS
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v3-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v4 05/12] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/OOXZvOwbpccKfGOtE9/SwX6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v2 05/11] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/RgO6TscyR9fMvkEm1k5N=yu
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v2-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v3 05/11] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/.arW0LC=Yi8JO9BRih4YlyS
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v3-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v4 05/12] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/OOXZvOwbpccKfGOtE9/SwX6
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v4-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v7 05/16] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_/VSGM3xNEmY7iJyL2wuWRCjV
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v7-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* [PATCH v5 05/14] Improve sentences in overview of system configuration parameters
@ 2023-09-25 20:32 Karl O. Pinc <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Karl O. Pinc @ 2023-09-25 20:32 UTC (permalink / raw)
Get rid of "we" wording. Remove extra words in sentences.
Line break after end of each sentence to ease future patch reading.
---
doc/src/sgml/config.sgml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6bc1b215db..97f9838bfb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -10,9 +10,10 @@
<para>
There are many configuration parameters that affect the behavior of
- the database system. In the first section of this chapter we
- describe how to interact with configuration parameters. The subsequent sections
- discuss each parameter in detail.
+ the database system.
+ The first section of this chapter describes how to interact with
+ configuration parameters.
+ Subsequent sections discuss each parameter in detail.
</para>
<sect1 id="config-setting">
--
2.30.2
--MP_//gBZFZYGWm7urqUgDbu6PSe
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=v5-0006-Provide-examples-of-listing-all-settings.patch
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-13 15:28 Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-02-13 15:28 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]>
Hi,
On Fri, Feb 07, 2025 at 03:32:43PM +0100, Jakub Wartak wrote:
> As I have promised to Andres on the Discord hacking server some time
> ago, I'm attaching the very brief (and potentially way too rushed)
> draft of the first step into NUMA observability on PostgreSQL that was
> based on his presentation [0]. It might be rough, but it is to get us
> started. The patches were not really even basically tested, they are
> more like input for discussion - rather than solid code - to shake out
> what should be the proper form of this.
>
> Right now it gives:
>
> postgres=# select numa_zone_id, count(*) from pg_buffercache group by
> numa_zone_id;
> NOTICE: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
> numa_zone_id | count
> --------------+-------
> | 16127
> 6 | 256
> 1 | 1
Thanks for the patch!
Not doing a code review but sharing some experimentation.
First, I had to:
@@ -99,7 +100,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
Size os_page_size;
void **os_page_ptrs;
int *os_pages_status;
- int os_page_count;
+ uint64 os_page_count;
and
- os_page_count = (NBuffers * BLCKSZ) / os_page_size;
+ os_page_count = ((uint64)NBuffers * BLCKSZ) / os_page_size;
to make it work with non tiny shared_buffers.
Observations:
when using 2 sessions:
Session 1 first loads buffers (e.g., by querying a relation) and then runs
'select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;'
Session 2 does nothing but runs 'select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;'
I see a lot of '-2' for the numa_zone_id in session 2, indicating that pages appear
as unmapped when viewed from a process that hasn't accessed them, even though
those same pages appear as allocated on a NUMA node in session 1.
To double check, I created a function pg_buffercache_pages_from_pid() that is
exactly the same as pg_buffercache_pages() (with your patch) except that it
takes a pid as input and uses it in move_pages(<pid>, …).
Let me show the results:
In session 1 (that "accessed/loaded" the ~65K buffers):
postgres=# select numa_zone_id, count(*) from pg_buffercache group by
numa_zone_id;
NOTICE: os_page_count=10485760 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+---------
| 5177310
0 | 65192
-2 | 378
(3 rows)
postgres=# select pg_backend_pid();
pg_backend_pid
----------------
1662580
In session 2:
postgres=# select numa_zone_id, count(*) from pg_buffercache group by
numa_zone_id;
NOTICE: os_page_count=10485760 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+---------
| 5177301
0 | 85
-2 | 65494
(3 rows)
^
postgres=# select numa_zone_id, count(*) from pg_buffercache_pages_from_pid(pg_backend_pid()) group by numa_zone_id;
NOTICE: os_page_count=10485760 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+---------
| 5177301
0 | 90
-2 | 65489
(3 rows)
But when session's 1 pid is used:
postgres=# select numa_zone_id, count(*) from pg_buffercache_pages_from_pid(1662580) group by numa_zone_id;
NOTICE: os_page_count=10485760 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+---------
| 5177301
0 | 65195
-2 | 384
(3 rows)
Results show:
Correct NUMA distribution in session 1
Correct NUMA distribution in session 2 only when using pg_buffercache_pages_from_pid()
with the pid of session 1 as a parameter (the session that actually accessed the buffers)
Which makes me wondering if using numa_move_pages()/move_pages is the
right approach. Would be curious to know if you observe the same behavior though.
The initial idea that you shared on discord was to use get_mempolicy() but
as Andres stated:
"
One annoying thing about get_mempolicy() is this:
If no page has yet been allocated for the specified address, get_mempolicy() will allocate a page as if the thread
had performed a read (load) access to that address, and return the ID of the node where that page was allocated.
Forcing the allocation to happen inside a monitoring function is decidedly not great.
"
The man page looks correct (verified with "perf record -e page-faults,kmem:mm_page_alloc -p <pid>")
while using get_mempolicy().
But maybe we could use get_mempolicy() only on "valid" buffers i.e
((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID)), thoughts?
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-17 12:02 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-02-17 12:02 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]>
On Thu, Feb 13, 2025 at 4:28 PM Bertrand Drouvot
<[email protected]> wrote:
Hi Bertrand,
Thanks for playing with this!
> Which makes me wonder if using numa_move_pages()/move_pages is the right approach. Would be curious to know if you observe the same behavior though.
You are correct, I'm observing identical behaviour, please see attached.
> Forcing the allocation to happen inside a monitoring function is decidedly not great.
We probably would need to split it to some separate and new view
within the pg_buffercache extension, but that is going to be slow, yet
still provide valid results. In the previous approach that
get_mempolicy() was allocating on 1st access, but it was slow not only
because it was allocating but also because it was just 1 syscall per
1x addr (yikes!). I somehow struggle to imagine how e.g. scanning
(really allocating) a 128GB buffer cache in future won't cause issues
- that's like 16-17mln (* 2) syscalls to be issued when not using
move_pages(2)
Another thing is that numa_maps(5) won't help us a lot too (not enough
granularity).
> But maybe we could use get_mempolicy() only on "valid" buffers i.e ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID)), thoughts?
Different perspective: I wanted to use the same approach in the new
pg_shmemallocations_numa, but that won't cut it there. The other idea
that came to my mind is to issue move_pages() from the backend that
has already used all of those pages. That literally mean on of the
below ideas:
1. from somewhere like checkpointer / bgwriter?
2. add touching memory on backend startup like always (sic!)
3. or just attempt to read/touch memory addr just before calling
move_pages(). E.g. this last options is just two lines:
if(os_page_ptrs[blk2page+j] == 0) {
+ volatile uint64 touch pg_attribute_unused();
os_page_ptrs[blk2page+j] = (char *)BufHdrGetBlock(bufHdr) +
(os_page_size*j);
+ touch = *(uint64 *)os_page_ptrs[blk2page+j];
}
and it seems to work while still issuing much less syscalls with
move_pages() across backends, well at least here.
Frankly speaking I do not know which path to take with this, maybe
that's good enough?
-J.
postgres=# select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;
NOTICE: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+-------
| 16149
4 | 59
0 | 59
6 | 58
2 | 59
(5 rows)
postgres=# create table xx as select generate_series(1, 1000000);
SELECT 1000000
postgres=# select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;
NOTICE: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+-------
| 14095
4 | 572
0 | 572
6 | 571
2 | 572
-2 | 2
(6 rows)
postgres=# show shared_buffers ;
shared_buffers
----------------
128MB
(1 row)
postgres=# select pg_backend_pid();
pg_backend_pid
----------------
14121
(1 row)
## and now from 14121:
postgres=# select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;
NOTICE: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+-------
| 13439
4 | 46
0 | 48
6 | 47
2 | 47
-2 | 2757
## also, this wont give us detailed addr <-> specific NUMA node info:
postgres@jw-test3:~$ grep --color /dev/zero /proc/14121/numa_maps
7f5dd6004000 interleave:0-7 file=/dev/zero\040(deleted) dirty=6829 mapmax=7 active=0 N0=853 N1=853 N2=855 N3=853 N4=852 N5=854 N6=854 N7=855 kernelpagesize_kB=4
Attachments:
[text/plain] numa_test.txt (1.5K, ../../CAKZiRmzgaN-vZeoDjSHCbavU7dDyBLa1Vyp4sW=WQaZ4R43mvw@mail.gmail.com/2-numa_test.txt)
download | inline:
postgres=# select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;
NOTICE: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+-------
| 16149
4 | 59
0 | 59
6 | 58
2 | 59
(5 rows)
postgres=# create table xx as select generate_series(1, 1000000);
SELECT 1000000
postgres=# select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;
NOTICE: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+-------
| 14095
4 | 572
0 | 572
6 | 571
2 | 572
-2 | 2
(6 rows)
postgres=# show shared_buffers ;
shared_buffers
----------------
128MB
(1 row)
postgres=# select pg_backend_pid();
pg_backend_pid
----------------
14121
(1 row)
## and now from 14121:
postgres=# select numa_zone_id, count(*) from pg_buffercache group by numa_zone_id;
NOTICE: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
--------------+-------
| 13439
4 | 46
0 | 48
6 | 47
2 | 47
-2 | 2757
## also, this wont give us detailed addr <-> specific NUMA node info:
postgres@jw-test3:~$ grep --color /dev/zero /proc/14121/numa_maps
7f5dd6004000 interleave:0-7 file=/dev/zero\040(deleted) dirty=6829 mapmax=7 active=0 N0=853 N1=853 N2=855 N3=853 N4=852 N5=854 N6=854 N7=855 kernelpagesize_kB=4
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-20 08:32 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-02-20 08:32 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]>
Hi Jakub,
On Mon, Feb 17, 2025 at 01:02:04PM +0100, Jakub Wartak wrote:
> On Thu, Feb 13, 2025 at 4:28 PM Bertrand Drouvot
> <[email protected]> wrote:
>
> Hi Bertrand,
>
> Thanks for playing with this!
>
> > Which makes me wonder if using numa_move_pages()/move_pages is the right approach. Would be curious to know if you observe the same behavior though.
>
> You are correct, I'm observing identical behaviour, please see attached.
Thanks for confirming!
>
> We probably would need to split it to some separate and new view
> within the pg_buffercache extension, but that is going to be slow, yet
> still provide valid results.
Yup.
> In the previous approach that
> get_mempolicy() was allocating on 1st access, but it was slow not only
> because it was allocating but also because it was just 1 syscall per
> 1x addr (yikes!). I somehow struggle to imagine how e.g. scanning
> (really allocating) a 128GB buffer cache in future won't cause issues
> - that's like 16-17mln (* 2) syscalls to be issued when not using
> move_pages(2)
Yeah, get_mempolicy() not working on a range is not great.
> > But maybe we could use get_mempolicy() only on "valid" buffers i.e ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID)), thoughts?
>
> Different perspective: I wanted to use the same approach in the new
> pg_shmemallocations_numa, but that won't cut it there. The other idea
> that came to my mind is to issue move_pages() from the backend that
> has already used all of those pages. That literally mean on of the
> below ideas:
> 1. from somewhere like checkpointer / bgwriter?
> 2. add touching memory on backend startup like always (sic!)
> 3. or just attempt to read/touch memory addr just before calling
> move_pages(). E.g. this last options is just two lines:
>
> if(os_page_ptrs[blk2page+j] == 0) {
> + volatile uint64 touch pg_attribute_unused();
> os_page_ptrs[blk2page+j] = (char *)BufHdrGetBlock(bufHdr) +
> (os_page_size*j);
> + touch = *(uint64 *)os_page_ptrs[blk2page+j];
> }
>
> and it seems to work while still issuing much less syscalls with
> move_pages() across backends, well at least here.
One of the main issue I see with 1. and 2. is that we would not get accurate
results should the kernel decides to migrate the pages. Indeed, the process doing
the move_pages() call needs to have accessed the pages more recently than any
kernel migrations to see accurate locations.
OTOH, one of the main issue that I see with 3. is that the monitoring could
probably influence the kernel's decision to start pages migration (I'm not 100%
sure but I could imagine it may influence the kernel's decision due to having to
read/touch the pages).
But I'm thinking: do we really need to know the page location of every single page?
I think what we want to see is if the pages are "equally" distributed on all
the nodes or are somehow "stuck" to one (or more) nodes. In that case what about
using get_mempolicy() but on a subset of the buffer cache? (say every Nth buffer
or contiguous chunks). We could create a new function that would accept a
"sampling distance" as parameter for example, thoughts?
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-24 11:57 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-02-24 11:57 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]>
Hi Bertrand,
TL;DR; the main problem seems choosing which way to page-fault the
shared memory before the backend is going to use numa_move_pages() as
the memory mappings (fresh after fork()/CoW) seem to be not ready for
numa_move_pages() inquiry.
On Thu, Feb 20, 2025 at 9:32 AM Bertrand Drouvot
<[email protected]> wrote:
> > We probably would need to split it to some separate and new view
> > within the pg_buffercache extension, but that is going to be slow, yet
> > still provide valid results.
>
> Yup.
OK so I've made that NUMA inquiry (now with that "volatile touch" to
get valid results for not used memory) into a new and separate
pg_buffercache_numa view. This avoids the problem that somebody would
automatically run into this slow path when using pg_buffercache.
> > > But maybe we could use get_mempolicy() only on "valid" buffers i.e ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID)), thoughts?
> >
> > Different perspective: I wanted to use the same approach in the new
> > pg_shmemallocations_numa, but that won't cut it there. The other idea
> > that came to my mind is to issue move_pages() from the backend that
> > has already used all of those pages. That literally mean on of the
> > below ideas:
> > 1. from somewhere like checkpointer / bgwriter?
> > 2. add touching memory on backend startup like always (sic!)
> > 3. or just attempt to read/touch memory addr just before calling
> > move_pages(). E.g. this last options is just two lines:
> >
> > if(os_page_ptrs[blk2page+j] == 0) {
> > + volatile uint64 touch pg_attribute_unused();
> > os_page_ptrs[blk2page+j] = (char *)BufHdrGetBlock(bufHdr) +
> > (os_page_size*j);
> > + touch = *(uint64 *)os_page_ptrs[blk2page+j];
> > }
> >
> > and it seems to work while still issuing much less syscalls with
> > move_pages() across backends, well at least here.
>
> One of the main issue I see with 1. and 2. is that we would not get accurate
> results should the kernel decides to migrate the pages. Indeed, the process doing
> the move_pages() call needs to have accessed the pages more recently than any
> kernel migrations to see accurate locations.
We never get fully accurate state as the zone memory migration might
be happening as we query it, but in theory we could add something to
e.g. checkpointer/bgwriter that would inquiry it on demand and report
it back somewhat through shared memory (?), but I'm somehow afraid
because as stated at the end of email, it might take some time (well
we probably wouldn't need to "touch memory" then after all, as all of
it is active), but that's still impact to those bgworkers. Somehow I
feel safer if that code is NOT part of bgworker.
> OTOH, one of the main issue that I see with 3. is that the monitoring could
> probably influence the kernel's decision to start pages migration (I'm not 100%
> sure but I could imagine it may influence the kernel's decision due to having to
> read/touch the pages).
>
> But I'm thinking: do we really need to know the page location of every single page?
> I think what we want to see is if the pages are "equally" distributed on all
> the nodes or are somehow "stuck" to one (or more) nodes. In that case what about
> using get_mempolicy() but on a subset of the buffer cache? (say every Nth buffer
> or contiguous chunks). We could create a new function that would accept a
> "sampling distance" as parameter for example, thoughts?
The way I envision it (and I think what Andres wanted, not sure, still
yet to see him comment on all of this) is to give PG devs a way to
quickly spot NUMA imbalances, even for single relation. Probably some
DBA in the wild could also query it to see how PG/kernel distributes
memory from time to time. It seems to be more debugging and coding aid
for future NUMA optimizations, rather than being used by some
monitoring constantly. I would even dare to say it would require
--enable-debug (or some other developer-only toggle), but apparently
there's no need to hide it like that if those are separate views.
Changes since previous version:
0. rebase due the recent OAuth commit introducing libcurl
1. cast uint64 for NBuffers as You found out
2. put stuff into pg_buffercache_numa
3. 0003 adds pg_shmem_numa_allocations Or should we rather call it
pg_shmem_numa_zones or maybe just pg_shm_numa ?
If there would be agreement that this is the way we want to have it
(from the backend and not from checkpointer), here's what's left on
the table to be done here:
a. isn't there something quicker for touching / page-faulting memory ?
If not then maybe add CHECKS_FOR_INTERRUPTS() there? BTW I've tried
additional MAP_POPULATE for PG_MMAP_FLAGS, but that didn't help (it
probably only works for parent//postmaster). I've also tried
MADV_POPULATE_READ (5.14+ kernels only) and that seems to work too:
+ rc = madvise(ShmemBase, ShmemSegHdr->totalsize, MADV_POPULATE_READ);
+ if(rc != 0) {
+ elog(NOTICE, "madvice() failed");
+ }
[..]
- volatile uint64 touch pg_attribute_unused();
os_page_ptrs[i] = (char *)ent->location + (i *
os_page_size);
- touch = *(uint64 *)os_page_ptrs[i];
with volatile touching memory or MADV_POPULATE_READ the result seems
to reliable (s_b 128MB here):
postgres@postgres:1234 : 14442 # select * from
pg_shmem_numa_allocations order by numa_size desc;
name | numa_zone_id | numa_size
------------------------------------------------+--------------+-----------
Buffer Blocks | 0 | 134221824
XLOG Ctl | 0 | 4206592
Buffer Descriptors | 0 | 1048576
transaction | 0 | 528384
Checkpointer Data | 0 | 524288
Checkpoint BufferIds | 0 | 327680
Shared Memory Stats | 0 | 311296
[..]
without at least one of those two, new backend reports complete garbage:
name | numa_zone_id | numa_size
------------------------------------------------+--------------+-----------
Buffer Blocks | 0 | 995328
Shared Memory Stats | 0 | 245760
shmInvalBuffer | 0 | 65536
Buffer Descriptors | 0 | 65536
Backend Status Array | 0 | 61440
serializable | 0 | 57344
[..]
b. refactor shared code so that it goes into src/port (but with
Linux-only support so far)
c. should we use MemoryContext in pg_get_shmem_numa_allocations or not?
d. fix tests, indent it, docs, make cfbot happy
As for the sampling, dunno, fine for me. As an optional argument? but
wouldn't it be better to find a way to actually for it to be quick?
OK, so here's larger test, on 512GB with 8x NUMA nodes and s_b set to
128GB with numactl --interleave=all pg_ctl start:
postgres=# select * from pg_shmem_numa_allocations ;
name | numa_zone_id | numa_size
------------------------------------------------+--------------+-------------
[..]
Buffer Blocks | 0 | 17179869184
Buffer Blocks | 1 | 17179869184
Buffer Blocks | 2 | 17179869184
Buffer Blocks | 3 | 17179869184
Buffer Blocks | 4 | 17179869184
Buffer Blocks | 5 | 17179869184
Buffer Blocks | 6 | 17179869184
Buffer Blocks | 7 | 17179869184
Buffer IO Condition Variables | 0 | 33554432
Buffer IO Condition Variables | 1 | 33554432
Buffer IO Condition Variables | 2 | 33554432
[..]
but it takes 23s. Yes it takes 23s to just gather that info with
memory touch, but that's ~128GB of memory and is hardly responsible
(lack of C_F_I()). By default without numactl's interleave=all, you
get clear picture of lack of NUMA awareness in PG shared segment (just
as Andres presented, but now it is evident; well it is subject to
autobalancing of course):
postgres=# select * from pg_shmem_numa_allocations ;
name | numa_zone_id | numa_size
------------------------------------------------+--------------+-------------
[..]
commit_timestamp | 0 | 2097152
commit_timestamp | 1 | 6291456
commit_timestamp | 2 | 0
commit_timestamp | 3 | 0
commit_timestamp | 4 | 0
[..]
transaction | 0 | 14680064
transaction | 1 | 0
transaction | 2 | 0
transaction | 3 | 0
transaction | 4 | 2097152
[..]
Somehow without interleave it is very quick too.
-J.
Attachments:
[application/octet-stream] v3-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch (5.1K, ../../CAKZiRmz7FWPAJkU6_-5S+e_fd184SWYnLa888wTske15MD_12A@mail.gmail.com/2-v3-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch)
download | inline diff:
From d34b7b5d082c3ea3f8806a5202a3f0fa1e9cca7c Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v3 1/3] Add optional dependency to libnuma for basic NUMA
awareness routines
---
.cirrus.tasks.yml | 4 ++++
configure.ac | 13 +++++++++++++
meson.build | 17 +++++++++++++++++
meson_options.txt | 3 +++
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +++
src/include/pg_config.h.in | 3 +++
src/makefiles/meson.build | 3 +++
8 files changed, 47 insertions(+)
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 91b51142d2e..e3b7554d9e8 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -436,6 +436,10 @@ task:
SANITIZER_FLAGS: -fsanitize=address
PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range
+
+ # FIXME: use or not the libnuma?
+ # --with-libnuma \
+ #
# Normally, the "relation segment" code basically has no coverage in our
# tests, because we (quite reasonably) don't generate tables large
# enough in tests. We've had plenty bugs that we didn't notice due the
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/meson.build b/meson.build
index 574f992ed49..cf9dead5d02 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('libnuma', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt, dirs: test_lib_d)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3821,6 +3837,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5aa..bff9f077a8c 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
--
2.39.5
[application/octet-stream] v3-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch (11.1K, ../../CAKZiRmz7FWPAJkU6_-5S+e_fd184SWYnLa888wTske15MD_12A@mail.gmail.com/3-v3-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch)
download | inline diff:
From e7239148d6dab4482e0f97958c077102295c31c7 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v3 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone
---
contrib/pg_buffercache/Makefile | 3 +-
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 30 +++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 113 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
7 files changed, 146 insertions(+), 6 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..9b2e9393410 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..e5b3d1f7dd2
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,30 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the function.
+DROP FUNCTION pg_buffercache_pages() CASCADE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages(boolean)
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages(false) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_pages(true) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages(boolean) FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..a5aab07fc99 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -6,6 +6,7 @@
* contrib/pg_buffercache/pg_buffercache_pages.c
*-------------------------------------------------------------------------
*/
+#include "pg_config.h"
#include "postgres.h"
#include "access/htup_details.h"
@@ -13,10 +14,16 @@
#include "funcapi.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#include <unistd.h>
+#endif
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +50,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -65,6 +73,15 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+static void
+pg_buffercache_mark_numa_invalid(BufferCachePagesContext *fctx, int n)
+{
+ int i;
+ for (i = 0; i < n; i++) {
+ fctx->record[i].numa_zone_id = -1;
+ }
+}
+
Datum
pg_buffercache_pages(PG_FUNCTION_ARGS)
{
@@ -75,10 +92,16 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
HeapTuple tuple;
+ Buffer query_numa = PG_GETARG_BOOL(0);
if (SRF_IS_FIRSTCALL())
{
- int i;
+ int i, blk2page, j;
+ Size os_page_size;
+ void **os_page_ptrs;
+ int *os_pages_status;
+ int os_page_count;
+ float pages_per_blk;
funcctx = SRF_FIRSTCALL_INIT();
@@ -122,10 +145,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
INT2OID, -1, 0);
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM-1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -140,6 +167,30 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Return to original context when allocating transient memory */
MemoryContextSwitchTo(oldcontext);
+ /* This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before
+ * calling move_pages()
+ */
+ os_page_size = sysconf(_SC_PAGESIZE);
+ if(huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ os_page_count = (NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(NOTICE, "os_page_count=%d os_page_size=%ld pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
/*
* Scan through all the buffers, saving the relevant fields in the
* fctx->record structure.
@@ -177,8 +228,61 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
else
fctx->record[i].isvalid = false;
+#ifdef USE_LIBNUMA
+/* FIXME: taken from bufmgr.c, maybe move to .h ? */
+#define BufHdrGetBlock(bufHdr) ((Block) (BufferBlocks + ((Size) (bufHdr)->buf_id) * BLCKSZ))
+ blk2page = (int) i * pages_per_blk;
+ j = 0;
+ do {
+ /*
+ * Many buffers can point to the same page, but we want to
+ * query just first address.
+ *
+ * In order to get reliable results we also need to touch memory pages
+ * so that inquiry about NUMA zone doesn't return -2.
+ */
+ if(os_page_ptrs[blk2page+j] == 0) {
+ volatile uint64 touch pg_attribute_unused();
+ os_page_ptrs[blk2page+j] = (char *)BufHdrGetBlock(bufHdr) + (os_page_size*j);
+ touch = *(uint64 *)os_page_ptrs[blk2page+j];
+ }
+ j++;
+ } while(j < (int)pages_per_blk);
+#endif
+
UnlockBufHdr(bufHdr, buf_state);
}
+
+
+#ifdef USE_LIBNUMA
+ if(query_numa) {
+ /* According to numa(3) it is required to initialize library even if that's no-op. */
+ if(numa_available() == -1) {
+ pg_buffercache_mark_numa_invalid(fctx, NBuffers);
+ elog(NOTICE, "libnuma initialization failed, some NUMA data might be unavailable.");;
+ } else {
+ /* Amortize the number of pages we need to query about */
+ if(numa_move_pages(0, os_page_count, os_page_ptrs, NULL, os_pages_status, 0) == -1) {
+ elog(ERROR, "failed NUMA pages inquiry status");
+ }
+ for (i = 0; i < NBuffers; i++) {
+ blk2page = (int) i * pages_per_blk;
+ /* Technically we can get errors too here and pass that to user
+ *
+ * XXX:: also we could somehow report single DB block spanning
+ * more than 2 NUMA zones, but it should be rare (?)
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
+ }
+ } else
+ pg_buffercache_mark_numa_invalid(fctx, NBuffers);
+#else
+ pg_buffercache_mark_numa_invalid(fctx, NBuffers);
+#endif
+
+ pfree(os_page_ptrs);
+ pfree(os_pages_status);
}
funcctx = SRF_PERCALL_SETUP();
@@ -211,6 +315,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[7] = true;
/* unused for v1.0 callers, but the array is always long enough */
nulls[8] = true;
+ nulls[9] = true;
}
else
{
@@ -231,6 +336,8 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* unused for v1.0 callers, but the array is always long enough */
values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
}
/* Build and return the tuple. */
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 03a6dd49154..172309d389a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -562,7 +562,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
[application/octet-stream] v3-0003-Add-pg_shmem_numa_allocations.patch (6.3K, ../../CAKZiRmz7FWPAJkU6_-5S+e_fd184SWYnLa888wTske15MD_12A@mail.gmail.com/4-v3-0003-Add-pg_shmem_numa_allocations.patch)
download | inline diff:
From 995011841cde76e530e2ff12452f54e8b8da5923 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v3 3/3] Add pg_shmem_numa_allocations
---
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 120 +++++++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
3 files changed, 136 insertions(+)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index eff0990957e..c808fb82d75 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..c8881d98e05 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -73,6 +73,11 @@
#include "storage/shmem.h"
#include "storage/spin.h"
#include "utils/builtins.h"
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#include <unistd.h>
+#endif
static void *ShmemAllocRaw(Size size, Size *allocated_size);
@@ -568,3 +573,118 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+//#ifdef LIBNUMA
+#if 1
+
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+
+ /* According to numa(3) it is required to initialize library even if that's no-op. */
+ if(numa_available() == -1) {
+ elog(NOTICE, "libnuma initialization failed, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /* This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before
+ * calling move_pages()
+ */
+ os_page_size = sysconf(_SC_PAGESIZE);
+ if(huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ // MemoryContext!
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+ void **os_page_ptrs;
+ int *os_pages_status;
+ int os_page_count;
+#define MAX_ZONES 32 /* FIXME? */
+ Size zones[MAX_ZONES];
+
+ os_page_count = ent->allocated_size / os_page_size;
+ //elog(NOTICE, "os_page_count=%d os_page_size=%ld ", os_page_count, os_page_size);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ for(i = 0; i < os_page_count; i++) {
+ /*
+ * In order to get reliable results we also need to touch memory pages
+ * so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+ os_page_ptrs[i] = (char *)ent->location + (i * os_page_size);
+ touch = *(uint64 *)os_page_ptrs[i];
+ }
+
+ /* Amortize the number of pages we need to query about */
+ if(numa_move_pages(0, os_page_count, os_page_ptrs, NULL, os_pages_status, 0) == -1) {
+ elog(ERROR, "failed NUMA pages inquiry status");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Counter number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < os_page_count; i++) {
+ int s = os_pages_status[i];
+ if(s >= 0)
+ zones[s]++;
+ }
+
+ pfree(os_page_ptrs);
+ pfree(os_pages_status);
+
+ for(i = 0; i <= numa_max_node(); i++){
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /* XXX: We are ignoring reporting the following regions in pg_get_shmem_allocations() case:
+ * - output shared memory allocated but not counted via the shmem index
+ * - output as-of-yet unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+#else
+ ereport(WARNING,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("NUMA support is not availble"),
+ errdetail("NUMA zone information is not available on this platform due to lack of libnuma"),
+ errhint("It looks like you need to re-compile with libnuma packages available")));
+#endif
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9e803d610d7..1efa342b725 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8463,6 +8463,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int8,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-24 14:06 Andres Freund <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: Andres Freund @ 2025-02-24 14:06 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; pgsql-hackers
Hi,
On 2025-02-24 12:57:16 +0100, Jakub Wartak wrote:
> On Thu, Feb 20, 2025 at 9:32 AM Bertrand Drouvot
> <[email protected]> wrote:
> > OTOH, one of the main issue that I see with 3. is that the monitoring could
> > probably influence the kernel's decision to start pages migration (I'm not 100%
> > sure but I could imagine it may influence the kernel's decision due to having to
> > read/touch the pages).
> >
> > But I'm thinking: do we really need to know the page location of every single page?
> > I think what we want to see is if the pages are "equally" distributed on all
> > the nodes or are somehow "stuck" to one (or more) nodes. In that case what about
> > using get_mempolicy() but on a subset of the buffer cache? (say every Nth buffer
> > or contiguous chunks). We could create a new function that would accept a
> > "sampling distance" as parameter for example, thoughts?
>
> The way I envision it (and I think what Andres wanted, not sure, still
> yet to see him comment on all of this) is to give PG devs a way to
> quickly spot NUMA imbalances, even for single relation.
Yea. E.g. for some benchmark workloads the difference whether the root btree
page is on the same NUMA node as the workload or not makes a roughly 2x perf
difference. It's really hard to determine that today.
> If there would be agreement that this is the way we want to have it
> (from the backend and not from checkpointer), here's what's left on
> the table to be done here:
> a. isn't there something quicker for touching / page-faulting memory ?
If you actually fault in a page the kernel actually has to allocate memory and
then zero it out. That rather severely limits the throughput...
> If not then maybe add CHECKS_FOR_INTERRUPTS() there?
Should definitely be there.
> BTW I've tried additional MAP_POPULATE for PG_MMAP_FLAGS, but that didn't
> help (it probably only works for parent//postmaster).
Yes, needs to be in postmaster.
Does the issue with "new" backends seeing pages as not present exist both with
and without huge pages?
FWIW, what you posted fails on CI:
https://cirrus-ci.com/task/5114213770723328
Probably some ifdefs are missing. The sanity-check task configures with
minimal dependencies, which is why you're seeing this even on linux.
> b. refactor shared code so that it goes into src/port (but with
> Linux-only support so far)
> c. should we use MemoryContext in pg_get_shmem_numa_allocations or not?
You mean a specific context instead of CurrentMemoryContext?
> diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
> index 91b51142d2e..e3b7554d9e8 100644
> --- a/.cirrus.tasks.yml
> +++ b/.cirrus.tasks.yml
> @@ -436,6 +436,10 @@ task:
> SANITIZER_FLAGS: -fsanitize=address
> PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range
>
> +
> + # FIXME: use or not the libnuma?
> + # --with-libnuma \
> + #
> # Normally, the "relation segment" code basically has no coverage in our
> # tests, because we (quite reasonably) don't generate tables large
> # enough in tests. We've had plenty bugs that we didn't notice due
> the
I don't see why not.
> diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
> new file mode 100644
> index 00000000000..e5b3d1f7dd2
> --- /dev/null
> +++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
> @@ -0,0 +1,30 @@
> +/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
> +
> +-- complain if script is sourced in psql, rather than via CREATE EXTENSION
> +\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
> +
> +-- Register the function.
> +DROP FUNCTION pg_buffercache_pages() CASCADE;
Why? I think that's going to cause problems, as the pg_buffercache view
depends on it, and user views might turn in depend on pg_buffercache. I think
CASCADE is rarely, if ever, ok to use in an extension scripot.
> +CREATE OR REPLACE FUNCTION pg_buffercache_pages(boolean)
> +RETURNS SETOF RECORD
> +AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
> +LANGUAGE C PARALLEL SAFE;
>
> +-- Create a view for convenient access.
> +CREATE OR REPLACE VIEW pg_buffercache AS
> + SELECT P.* FROM pg_buffercache_pages(false) AS P
> + (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
> + relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
> + pinning_backends int4);
> +
> +CREATE OR REPLACE VIEW pg_buffercache_numa AS
> + SELECT P.* FROM pg_buffercache_pages(true) AS P
> + (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
> + relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
> + pinning_backends int4, numa_zone_id int4);
> +
> +-- Don't want these to be available to public.
> +REVOKE ALL ON FUNCTION pg_buffercache_pages(boolean) FROM PUBLIC;
> +REVOKE ALL ON pg_buffercache FROM PUBLIC;
> +REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
We grant pg_monitor SELECT TO pg_buffercache, I think we should do the same
for _numa?
> @@ -177,8 +228,61 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
> else
> fctx->record[i].isvalid = false;
>
> +#ifdef USE_LIBNUMA
> +/* FIXME: taken from bufmgr.c, maybe move to .h ? */
> +#define BufHdrGetBlock(bufHdr) ((Block) (BufferBlocks + ((Size) (bufHdr)->buf_id) * BLCKSZ))
> + blk2page = (int) i * pages_per_blk;
BufferGetBlock() is public, so I don't think BufHdrGetBlock() is needed here.
> + j = 0;
> + do {
> + /*
> + * Many buffers can point to the same page, but we want to
> + * query just first address.
> + *
> + * In order to get reliable results we also need to touch memory pages
> + * so that inquiry about NUMA zone doesn't return -2.
> + */
> + if(os_page_ptrs[blk2page+j] == 0) {
> + volatile uint64 touch pg_attribute_unused();
> + os_page_ptrs[blk2page+j] = (char *)BufHdrGetBlock(bufHdr) + (os_page_size*j);
> + touch = *(uint64 *)os_page_ptrs[blk2page+j];
> + }
> + j++;
> + } while(j < (int)pages_per_blk);
> +#endif
> +
Why is this done before we even have gotten -2 back? Even if we need it, it
seems like we ought to defer this until necessary.
> +#ifdef USE_LIBNUMA
> + if(query_numa) {
> + /* According to numa(3) it is required to initialize library even if that's no-op. */
> + if(numa_available() == -1) {
> + pg_buffercache_mark_numa_invalid(fctx, NBuffers);
> + elog(NOTICE, "libnuma initialization failed, some NUMA data might be unavailable.");;
> + } else {
> + /* Amortize the number of pages we need to query about */
> + if(numa_move_pages(0, os_page_count, os_page_ptrs, NULL, os_pages_status, 0) == -1) {
> + elog(ERROR, "failed NUMA pages inquiry status");
> + }
I wonder if we ought to override numa_error() so we can display more useful
errors.
> +
> + LWLockAcquire(ShmemIndexLock, LW_SHARED);
Doing multiple memory allocations while holding an lwlock is probably not a
great idea, even if the lock normally isn't contended.
> + os_page_ptrs = palloc(sizeof(void *) * os_page_count);
> + os_pages_status = palloc(sizeof(int) * os_page_count);
Why do this in very loop iteration?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-24 16:11 Bertrand Drouvot <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-02-24 16:11 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Jakub Wartak <[email protected]>; pgsql-hackers
Hi,
On Mon, Feb 24, 2025 at 09:06:20AM -0500, Andres Freund wrote:
> Does the issue with "new" backends seeing pages as not present exist both with
> and without huge pages?
That's a good point and from what I can see it's correct with huge pages being
used (it means all processes see the same NUMA node assignment regardless of
access patterns).
That said, wouldn't that be too strong to impose a restriction that huge_pages
must be enabled?
Jakub, thanks for the new patch version! FWIW, I did not look closely to the
code yet (just did the minor changes already shared to have valid result with non
tiny shared buffer size). I'll look closely at the code for sure once we all agree
on the design part of it.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-26 08:38 Jakub Wartak <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 2 replies; 95+ messages in thread
From: Jakub Wartak @ 2025-02-26 08:38 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; pgsql-hackers
On Mon, Feb 24, 2025 at 3:06 PM Andres Freund <[email protected]> wrote:
> On 2025-02-24 12:57:16 +0100, Jakub Wartak wrote:
Hi Andres, thanks for your review!
OK first sane version attached with new src/port/pg_numa.c boilerplate
in 0001. Fixed some bugs too, there is one remaining optimization to
be done (see that `static` question later). Docs/tests are still
missing.
QQ: I'm still wondering if we there's better way of exposing multiple
pg's shma entries pointing to the same page (think of something hot:
PROCLOCK or ProcArray), so wouldn't it make sense (in some future
thread/patch) to expose this info somehow via additional column
(pg_get_shmem_numa_allocations.shared_pages bool?) ? I'm thinking of
an easy way of showing that a potential NUMA auto balancing could lead
to TLB NUMA shootdowns (not that it is happening or counting , just
identifying it as a problem in allocation). Or that stuff doesn't make
sense as we already have pg_shm_allocations.{off|size} and we could
derive such info from it (after all it is for devs?)?
postgres@postgres:1234 : 18843 # select
name,off,off+allocated_size,allocated_size from pg_shmem_allocations
order by off;
name | off | ?column?
| allocated_size
------------------------------------------------+-----------+-----------+----------------
[..]
Proc Header | 147114112 |
147114240 | 128
Proc Array | 147274752 |
147275392 | 640
KnownAssignedXids | 147275392 |
147310848 | 35456
KnownAssignedXidsValid | 147310848 |
147319808 | 8960
Backend Status Array | 147319808 |
147381248 | 61440
postgres@postgres:1234 : 18924 # select * from
pg_shmem_numa_allocations where name IN ('Proc Header', 'Proc Array',
'KnownAssignedXids', '..') order by name;
name | numa_zone_id | numa_size
-------------------+--------------+-----------
KnownAssignedXids | 0 | 2097152
Proc Array | 0 | 2097152
Proc Header | 0 | 2097152
I.e. ProcArray ends and right afterwards KnownAssignedXids start, both
are hot , but on the same HP and NUMA node
> > If there would be agreement that this is the way we want to have it
> > (from the backend and not from checkpointer), here's what's left on
> > the table to be done here:
>
> > a. isn't there something quicker for touching / page-faulting memory ?
>
> If you actually fault in a page the kernel actually has to allocate memory and
> then zero it out. That rather severely limits the throughput...
OK, no comments about that madvise(MADV_POPULATE_READ), so I'm
sticking to pointers.
> > If not then maybe add CHECKS_FOR_INTERRUPTS() there?
>
> Should definitely be there.
Added.
> > BTW I've tried additional MAP_POPULATE for PG_MMAP_FLAGS, but that didn't
> > help (it probably only works for parent//postmaster).
>
> Yes, needs to be in postmaster.
>
> Does the issue with "new" backends seeing pages as not present exist both with
> and without huge pages?
Please see attached file for more verbose results, but in short it is
like below:
patch(-touchpages) hugepages=off INVALID RESULTS (-2)
patch(-touchpages) hugepages=on INVALID RESULTS (-2)
patch(touchpages) hugepages=off CORRECT RESULT
patch(touchpages) hugepages=on CORRECT RESULT
patch(-touchpages)+MAP_POPULATE hugepages=off INVALID RESULTS (-2)
patch(-touchpages)+MAP_POPULATE hugepages=on INVALID RESULTS (-2)
IMHVO, the only other thing that could work here (but still
page-faulting) is that 5.14+ madvise(MADV_POPULATE_READ). Tests are
welcome, maybe it might be kernel version dependent.
BTW: and yes you can "feel" the timing impact of
MAP_SHARED|MAP_POPULATE during startup, it seems that for our case
child backends that don't come-up with pre-faulted page attachments
across fork() apparently.
> FWIW, what you posted fails on CI:
> https://cirrus-ci.com/task/5114213770723328
>
> Probably some ifdefs are missing. The sanity-check task configures with
> minimal dependencies, which is why you're seeing this even on linux.
Hopefully fixed, we'll see what cfbot tells, I'm flying blind with all
of this CI stuff...
> > b. refactor shared code so that it goes into src/port (but with
> > Linux-only support so far)
Done.
> > c. should we use MemoryContext in pg_get_shmem_numa_allocations or not?
>
> You mean a specific context instead of CurrentMemoryContext?
Yes, I had doubts earlier, but for now I'm going to leave it as it is.
> > diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
> > index 91b51142d2e..e3b7554d9e8 100644
> > --- a/.cirrus.tasks.yml
> > +++ b/.cirrus.tasks.yml
> > @@ -436,6 +436,10 @@ task:
> > SANITIZER_FLAGS: -fsanitize=address
> > PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range
> >
> > +
> > + # FIXME: use or not the libnuma?
> > + # --with-libnuma \
> > + #
> > # Normally, the "relation segment" code basically has no coverage in our
> > # tests, because we (quite reasonably) don't generate tables large
> > # enough in tests. We've had plenty bugs that we didn't notice due
> > the
>
> I don't see why not.
Fixed.
> > diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
> > new file mode 100644
> > index 00000000000..e5b3d1f7dd2
> > --- /dev/null
> > +++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
> > @@ -0,0 +1,30 @@
> > +/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
> > +
> > +-- complain if script is sourced in psql, rather than via CREATE EXTENSION
> > +\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
> > +
> > +-- Register the function.
> > +DROP FUNCTION pg_buffercache_pages() CASCADE;
>
> Why? I think that's going to cause problems, as the pg_buffercache view
> depends on it, and user views might turn in depend on pg_buffercache. I think
> CASCADE is rarely, if ever, ok to use in an extension scripot.
... it's just me cutting corners :^), fixed now.
[..]
> > +-- Don't want these to be available to public.
> > +REVOKE ALL ON FUNCTION pg_buffercache_pages(boolean) FROM PUBLIC;
> > +REVOKE ALL ON pg_buffercache FROM PUBLIC;
> > +REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
>
> We grant pg_monitor SELECT TO pg_buffercache, I think we should do the same
> for _numa?
Yup, fixed.
> > @@ -177,8 +228,61 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
> > else
> > fctx->record[i].isvalid = false;
> >
> > +#ifdef USE_LIBNUMA
> > +/* FIXME: taken from bufmgr.c, maybe move to .h ? */
> > +#define BufHdrGetBlock(bufHdr) ((Block) (BufferBlocks + ((Size) (bufHdr)->buf_id) * BLCKSZ))
> > + blk2page = (int) i * pages_per_blk;
>
> BufferGetBlock() is public, so I don't think BufHdrGetBlock() is needed here.
Fixed, thanks I was looking for something like this! Is that +1 in v4 good?
> > + j = 0;
> > + do {
> > + /*
> > + * Many buffers can point to the same page, but we want to
> > + * query just first address.
> > + *
> > + * In order to get reliable results we also need to touch memory pages
> > + * so that inquiry about NUMA zone doesn't return -2.
> > + */
> > + if(os_page_ptrs[blk2page+j] == 0) {
> > + volatile uint64 touch pg_attribute_unused();
> > + os_page_ptrs[blk2page+j] = (char *)BufHdrGetBlock(bufHdr) + (os_page_size*j);
> > + touch = *(uint64 *)os_page_ptrs[blk2page+j];
> > + }
> > + j++;
> > + } while(j < (int)pages_per_blk);
> > +#endif
> > +
>
> Why is this done before we even have gotten -2 back? Even if we need it, it
> seems like we ought to defer this until necessary.
Not fixed yet: maybe we could even do a `static` with
`has_this_run_earlier` and just perform this work only once during the
first time?
> > +#ifdef USE_LIBNUMA
> > + if(query_numa) {
> > + /* According to numa(3) it is required to initialize library even if that's no-op. */
> > + if(numa_available() == -1) {
> > + pg_buffercache_mark_numa_invalid(fctx, NBuffers);
> > + elog(NOTICE, "libnuma initialization failed, some NUMA data might be unavailable.");;
> > + } else {
> > + /* Amortize the number of pages we need to query about */
> > + if(numa_move_pages(0, os_page_count, os_page_ptrs, NULL, os_pages_status, 0) == -1) {
> > + elog(ERROR, "failed NUMA pages inquiry status");
> > + }
>
> I wonder if we ought to override numa_error() so we can display more useful
> errors.
Another question without an easy answer as I never hit this error from
numa_move_pages(), one gets invalid stuff in *os_pages_status instead.
BUT!: most of our patch just uses things that cannot fail as per
libnuma usage. One way to trigger libnuma warnings is e.g. `chmod 700
/sys` (because it's hard to unmount it) and then still most of numactl
stuff works as euid != 0, but numactl --hardware gets at least
"libnuma: Warning: Cannot parse distance information in sysfs:
Permission denied" or same story with numactl -C 678 date. So unless
we start way more heavy use of libnuma (not just for observability)
there's like no point in that right now (?) Contrary to that: we can
do just do variadic elog() for that, I've put some code, but no idea
if that works fine...
[..]
> Doing multiple memory allocations while holding an lwlock is probably not a
> great idea, even if the lock normally isn't contended.
[..]
> Why do this in very loop iteration?
Both fixed.
-J.
IMPACT OF pg_numa_touch_mem_if_required() from pg_numa.h: so we are touching like ~16k * 8kB buffers =~ 128MB (close to pgbench_accounts size):
postgres=# \dt+ pgbench_accounts
List of tables
Schema | Name | Type | Owner | Persistence | Access method | Size | Description
--------+------------------+-------+----------+-------------+---------------+--------+-------------
public | pgbench_accounts | table | postgres | permanent | heap | 128 MB |
pgbench -i -s 10 ~~ 128MB size, and "select relfilenode, count(*) from pg_buffercache where relfilenode is not null group by relfilenode order by 2 desc limit 3;" shows:
relfilenode | count
-------------+-------
28345 | 16402 // <-- pgbench_accounts
1249 | 46
2619 | 44
EXPECTED RESULT: at least 16k+ buffers split into multiple NUMA zones
patch(-touchpages) hugepages=off INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;'
[..]
done in 2.26 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.40 s, vacuum 0.18 s, primary keys 0.54 s).
huge_pages_status
-------------------
off
(1 row)
numa_zone_id | count
--------------+--------
| 507568
0 | 98
2 | 98
-2 | 16524
patch(-touchpages) hugepages=on INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;'
[..]
done in 2.54 s (drop tables 0.20 s, create tables 0.02 s, client-side generate 1.57 s, vacuum 0.21 s, primary keys 0.55 s).
huge_pages_status
-------------------
on
(1 row)
numa_zone_id | count
--------------+--------
| 507490
0 | 256
2 | 256
-2 | 15774
1 | 512
patch(touchpages) hugepages=off CORRECT RESULT
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;'
[..]
done in 2.42 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.55 s, vacuum 0.19 s, primary keys 0.54 s).
huge_pages_status
-------------------
off
(1 row)
numa_zone_id | count
--------------+--------
| 507589
0 | 8349
2 | 8350
^^ it's interesting (different split without huge with huge pages)
patch(touchpages) hugepages=on CORRECT RESULT
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;' -c '\l+'
[..]
done in 2.21 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.41 s, vacuum 0.19 s, primary keys 0.47 s).
huge_pages_status
-------------------
on
(1 row)
numa_zone_id | count
--------------+--------
| 507602
3 | 4096
0 | 4352
2 | 4096
1 | 4142
^^ it's interesting (different split without huge with huge pages)
patch(-touchpages)+MAP_POPULATE hugepages=off INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;' -c '\l+'
[..]
done in 2.25 s (drop tables 0.13 s, create tables 0.01 s, client-side generate 1.45 s, vacuum 0.19 s, primary keys 0.48 s).
huge_pages_status
-------------------
off
(1 row)
numa_zone_id | count
--------------+--------
| 507594
3 | 98
-2 | 16498
1 | 98
(4 rows)
patch(-touchpages)+MAP_POPULATE hugepages=on INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;' -c '\l+'
[..]
done in 4.89 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.50 s, vacuum 1.32 s, primary keys 1.93 s).
huge_pages_status
-------------------
on
(1 row)
numa_zone_id | count
--------------+--------
| 507596
0 | 256
2 | 256
-2 | 15616
1 | 564
(5 rows)
cross-check that MAP_POPULATE was used above:
postgres@jw-test3:~$ strace -ffe mmap numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart 2>&1 | grep MAP_SHARED
mmap(NULL, 27028, PROT_READ, MAP_SHARED, 3, 0) = 0x7f954aaa1000
[pid 15686] mmap(NULL, 27028, PROT_READ, MAP_SHARED, 3, 0) = 0x7fecd3d46000
[pid 15686] mmap(NULL, 4454350848, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS|MAP_POPULATE|MAP_HUGETLB, -1, 0) = 0x7febc7200000
postgres@jw-test3:~$ uname -r
6.10.11+bpo-cloud-amd64
Attachments:
[text/plain] touchpages_vs_numa_inquiry_results.txt (5.4K, ../../CAKZiRmzpvBtqrz5Jr2DDcfk4Ar1ppsXkUhEX9RpA+s+_5hcTOg@mail.gmail.com/2-touchpages_vs_numa_inquiry_results.txt)
download | inline:
IMPACT OF pg_numa_touch_mem_if_required() from pg_numa.h: so we are touching like ~16k * 8kB buffers =~ 128MB (close to pgbench_accounts size):
postgres=# \dt+ pgbench_accounts
List of tables
Schema | Name | Type | Owner | Persistence | Access method | Size | Description
--------+------------------+-------+----------+-------------+---------------+--------+-------------
public | pgbench_accounts | table | postgres | permanent | heap | 128 MB |
pgbench -i -s 10 ~~ 128MB size, and "select relfilenode, count(*) from pg_buffercache where relfilenode is not null group by relfilenode order by 2 desc limit 3;" shows:
relfilenode | count
-------------+-------
28345 | 16402 // <-- pgbench_accounts
1249 | 46
2619 | 44
EXPECTED RESULT: at least 16k+ buffers split into multiple NUMA zones
patch(-touchpages) hugepages=off INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;'
[..]
done in 2.26 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.40 s, vacuum 0.18 s, primary keys 0.54 s).
huge_pages_status
-------------------
off
(1 row)
numa_zone_id | count
--------------+--------
| 507568
0 | 98
2 | 98
-2 | 16524
patch(-touchpages) hugepages=on INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;'
[..]
done in 2.54 s (drop tables 0.20 s, create tables 0.02 s, client-side generate 1.57 s, vacuum 0.21 s, primary keys 0.55 s).
huge_pages_status
-------------------
on
(1 row)
numa_zone_id | count
--------------+--------
| 507490
0 | 256
2 | 256
-2 | 15774
1 | 512
patch(touchpages) hugepages=off CORRECT RESULT
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;'
[..]
done in 2.42 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.55 s, vacuum 0.19 s, primary keys 0.54 s).
huge_pages_status
-------------------
off
(1 row)
numa_zone_id | count
--------------+--------
| 507589
0 | 8349
2 | 8350
^^ it's interesting (different split without huge with huge pages)
patch(touchpages) hugepages=on CORRECT RESULT
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;' -c '\l+'
[..]
done in 2.21 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.41 s, vacuum 0.19 s, primary keys 0.47 s).
huge_pages_status
-------------------
on
(1 row)
numa_zone_id | count
--------------+--------
| 507602
3 | 4096
0 | 4352
2 | 4096
1 | 4142
^^ it's interesting (different split without huge with huge pages)
patch(-touchpages)+MAP_POPULATE hugepages=off INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;' -c '\l+'
[..]
done in 2.25 s (drop tables 0.13 s, create tables 0.01 s, client-side generate 1.45 s, vacuum 0.19 s, primary keys 0.48 s).
huge_pages_status
-------------------
off
(1 row)
numa_zone_id | count
--------------+--------
| 507594
3 | 98
-2 | 16498
1 | 98
(4 rows)
patch(-touchpages)+MAP_POPULATE hugepages=on INVALID RESULTS (-2)
postgres@jw-test3:~$ numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart ; /usr/pgsql18numa/bin/pgbench -i -s 10; /usr/pgsql18numa/bin/psql -c 'show huge_pages_status;' -c 'select numa_zone_id, count(*) from pg_buffercache_numa group by numa_zone_id;' -c '\l+'
[..]
done in 4.89 s (drop tables 0.12 s, create tables 0.01 s, client-side generate 1.50 s, vacuum 1.32 s, primary keys 1.93 s).
huge_pages_status
-------------------
on
(1 row)
numa_zone_id | count
--------------+--------
| 507596
0 | 256
2 | 256
-2 | 15616
1 | 564
(5 rows)
cross-check that MAP_POPULATE was used above:
postgres@jw-test3:~$ strace -ffe mmap numactl --interleave=all /usr/pgsql18numa/bin/pg_ctl -c -D /tmp/numa -l logfile restart 2>&1 | grep MAP_SHARED
mmap(NULL, 27028, PROT_READ, MAP_SHARED, 3, 0) = 0x7f954aaa1000
[pid 15686] mmap(NULL, 27028, PROT_READ, MAP_SHARED, 3, 0) = 0x7fecd3d46000
[pid 15686] mmap(NULL, 4454350848, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS|MAP_POPULATE|MAP_HUGETLB, -1, 0) = 0x7febc7200000
postgres@jw-test3:~$ uname -r
6.10.11+bpo-cloud-amd64
[application/octet-stream] v4-0003-Add-pg_shmem_numa_allocations.patch (6.4K, ../../CAKZiRmzpvBtqrz5Jr2DDcfk4Ar1ppsXkUhEX9RpA+s+_5hcTOg@mail.gmail.com/3-v4-0003-Add-pg_shmem_numa_allocations.patch)
download | inline diff:
From 7be77950a0e29640204b77616e90a5137b33d154 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v4 3/3] Add pg_shmem_numa_allocations
---
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 127 +++++++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
3 files changed, 143 insertions(+)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index eff0990957e..c808fb82d75 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..dd84a41e3e8 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -73,6 +73,7 @@
#include "storage/shmem.h"
#include "storage/spin.h"
#include "utils/builtins.h"
+#include "port/pg_numa.h"
static void *ShmemAllocRaw(Size size, Size *allocated_size);
@@ -568,3 +569,129 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ /* Every 1GB of scanned memory we give process chance to respond */
+#define ONE_GIGABYTE 1024*1024*1024
+ if ((i * os_page_size) % ONE_GIGABYTE == 0)
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1) {
+ /* FIXME: should we release LWlock and pfree here? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= pg_numa_get_max_node(); i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): - output shared memory
+ * allocated but not counted via the shmem index - output as-of-yet unused
+ * shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+
+ pfree(page_ptrs);
+ pfree(pages_status);
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9e803d610d7..1efa342b725 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8463,6 +8463,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int8,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
--
2.39.5
[application/octet-stream] v4-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch (12.3K, ../../CAKZiRmzpvBtqrz5Jr2DDcfk4Ar1ppsXkUhEX9RpA+s+_5hcTOg@mail.gmail.com/4-v4-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch)
download | inline diff:
From 86a9f778afaeb08ca3e03e81fb49372f392745a7 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v4 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone
---
contrib/pg_buffercache/Makefile | 3 +-
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 35 ++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 156 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
7 files changed, 189 insertions(+), 11 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..9b2e9393410 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..448d08196f3
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,35 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages(boolean)
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages(false) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_pages(true) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages(boolean) FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages(boolean) TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..95f331dc193 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -6,6 +6,7 @@
* contrib/pg_buffercache/pg_buffercache_pages.c
*-------------------------------------------------------------------------
*/
+#include "pg_config.h"
#include "postgres.h"
#include "access/htup_details.h"
@@ -13,10 +14,12 @@
#include "funcapi.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +46,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -65,6 +69,17 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+static void
+pg_buffercache_mark_numa_invalid(BufferCachePagesContext *fctx, int n)
+{
+ int i;
+
+ for (i = 0; i < n; i++)
+ {
+ fctx->record[i].numa_zone_id = -1;
+ }
+}
+
Datum
pg_buffercache_pages(PG_FUNCTION_ARGS)
{
@@ -75,14 +90,33 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
HeapTuple tuple;
+ Buffer query_numa = PG_GETARG_BOOL(0);
if (SRF_IS_FIRSTCALL())
{
- int i;
+ int i,
+ blk2page,
+ j;
+ Size os_page_size;
+ void **os_page_ptrs;
+ int *os_pages_status;
+ int os_page_count;
+ float pages_per_blk;
funcctx = SRF_FIRSTCALL_INIT();
- /* Switch context when allocating stuff to be used in later calls */
+ if (query_numa)
+ {
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
+ }
+ }
+
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
/* Create a user function context for cross-call persistence */
@@ -122,10 +156,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
INT2OID, -1, 0);
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -137,9 +175,37 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
funcctx->max_calls = NBuffers;
funcctx->user_fctx = fctx;
- /* Return to original context when allocating transient memory */
+ /*
+ * Return to original context when allocating transient memory
+ */
MemoryContextSwitchTo(oldcontext);
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first we
+ * need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ os_page_count = ((uint64)NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA os_page_count=%d os_page_size=%ld pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
/*
* Scan through all the buffers, saving the relevant fields in the
* fctx->record structure.
@@ -171,14 +237,79 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
else
fctx->record[i].isdirty = false;
- /* Note if the buffer is valid, and has storage created */
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
fctx->record[i].isvalid = true;
else
fctx->record[i].isvalid = false;
+ if (query_numa)
+ {
+ blk2page = (int) i * pages_per_blk;
+ j = 0;
+ do
+ {
+ /*
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+ if (os_page_ptrs[blk2page + j] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /*
+ * NBuffers count start really from 1
+ */
+ os_page_ptrs[blk2page + j] = (char *) BufferGetBlock(i + 1) + (os_page_size * j);
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2page + j]);
+
+ /*
+ * Every 1GB of scanned memory we give process chance
+ * to respond
+ */
+#define ONE_GIGABYTE 1024*1024*1024
+ if ((i * os_page_size) % ONE_GIGABYTE == 0)
+ CHECK_FOR_INTERRUPTS();
+ }
+ j++;
+ } while (j < (int) pages_per_blk);
+ }
+
UnlockBufHdr(bufHdr, buf_state);
}
+
+
+ if (query_numa)
+ {
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user
+ *
+ * XXX:: also we could somehow report single DB block spanning
+ * more than 2 NUMA zones, but it should be rare (?)
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
+ }
+ else
+ pg_buffercache_mark_numa_invalid(fctx, NBuffers);
+
+ pfree(os_page_ptrs);
+ pfree(os_pages_status);
}
funcctx = SRF_PERCALL_SETUP();
@@ -209,8 +340,12 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[5] = true;
nulls[6] = true;
nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
nulls[8] = true;
+ nulls[9] = true;
}
else
{
@@ -228,9 +363,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[6] = false;
values[7] = Int16GetDatum(fctx->record[i].usagecount);
nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
}
/* Build and return the tuple. */
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 03a6dd49154..172309d389a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -562,7 +562,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
[application/octet-stream] v4-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch (9.5K, ../../CAKZiRmzpvBtqrz5Jr2DDcfk4Ar1ppsXkUhEX9RpA+s+_5hcTOg@mail.gmail.com/5-v4-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch)
download | inline diff:
From 9e8f5376545dd73cd58183bffafcac8df26602a9 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v4 1/3] Add optional dependency to libnuma for basic NUMA
awareness routines
---
.cirrus.tasks.yml | 1 +
configure.ac | 13 ++++
meson.build | 17 ++++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 41 +++++++++++++
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 121 +++++++++++++++++++++++++++++++++++++
12 files changed, 208 insertions(+)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 91b51142d2e..7467e029131 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -448,6 +448,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/meson.build b/meson.build
index 574f992ed49..cf9dead5d02 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('libnuma', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt, dirs: test_lib_d)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3821,6 +3837,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5aa..bff9f077a8c 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..7cdc81d1ed6
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Miscellaneous functions for bit-wise operations.
+ *
+ *
+ * Copyright (c) 2019-2025, PostgreSQL Global Development Group
+ *
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..a9a7d2c964b
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,121 @@
+/*-------------------------------------------------------------------------
+ *
+ * numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+#include <unistd.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+#ifndef FRONTEND
+/* FIXME not tested, might crash */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+#endif
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-26 08:48 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Jakub Wartak @ 2025-02-26 08:48 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Mon, Feb 24, 2025 at 5:11 PM Bertrand Drouvot
<[email protected]> wrote:
>
> Hi,
>
> On Mon, Feb 24, 2025 at 09:06:20AM -0500, Andres Freund wrote:
> > Does the issue with "new" backends seeing pages as not present exist both with
> > and without huge pages?
>
> That's a good point and from what I can see it's correct with huge pages being
> used (it means all processes see the same NUMA node assignment regardless of
> access patterns).
Hi Bertrand , please see that nearby thread. I've got quite the
opposite results. I need page-fault memory or I get invalid results
("-2"). What kernel version are you using ? (I've tried it on two
6.10.x series kernels , virtualized in both cases; one was EPYC [real
NUMA, but not VM so not real hardware]).
> That said, wouldn't that be too strong to impose a restriction that huge_pages
> must be enabled?
>
> Jakub, thanks for the new patch version! FWIW, I did not look closely to the
> code yet (just did the minor changes already shared to have valid result with non
> tiny shared buffer size). I'll look closely at the code for sure once we all agree
> on the design part of it.
Cool, I think we are pretty close actually, but others might have
different perspective.
-J.
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-26 09:58 Andres Freund <[email protected]>
parent: Jakub Wartak <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Andres Freund @ 2025-02-26 09:58 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; pgsql-hackers
Hi,
On 2025-02-26 09:38:20 +0100, Jakub Wartak wrote:
> > FWIW, what you posted fails on CI:
> > https://cirrus-ci.com/task/5114213770723328
> >
> > Probably some ifdefs are missing. The sanity-check task configures with
> > minimal dependencies, which is why you're seeing this even on linux.
>
> Hopefully fixed, we'll see what cfbot tells, I'm flying blind with all
> of this CI stuff...
FYI, you can enable CI on a github repo, to see results without posting to the
list:
https://github.com/postgres/postgres/blob/master/src/tools/ci/README
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-26 13:05 Jakub Wartak <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-02-26 13:05 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; pgsql-hackers
On Wed, Feb 26, 2025 at 10:58 AM Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2025-02-26 09:38:20 +0100, Jakub Wartak wrote:
> > > FWIW, what you posted fails on CI:
> > > https://cirrus-ci.com/task/5114213770723328
> > >
> > > Probably some ifdefs are missing. The sanity-check task configures with
> > > minimal dependencies, which is why you're seeing this even on linux.
> >
> > Hopefully fixed, we'll see what cfbot tells, I'm flying blind with all
> > of this CI stuff...
>
> FYI, you can enable CI on a github repo, to see results without posting to the
> list:
> https://github.com/postgres/postgres/blob/master/src/tools/ci/README
Thanks, I'll take a look into it.
Meanwhile v5 is attached with slight changes to try to make cfbot happy:
1. fixed tests and added tiny copy-cat basic tests for
pg_buffercache_numa and pg_shm_numa_allocations views
2. win32 doesn't have sysconf()
No docs yet.
-J.
Attachments:
[application/octet-stream] v5-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch (15.6K, ../../CAKZiRmz_ERUH0udcZgpAinBSqpWoOYeao_rNNEe9f8Z9asTihw@mail.gmail.com/2-v5-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch)
download | inline diff:
From 01cf3b0d1b0d3f3f484a489158bae9b04d1e27a7 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v5 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache.out | 21 ++-
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 35 ++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 154 +++++++++++++++++-
contrib/pg_buffercache/sql/pg_buffercache.sql | 10 +-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
9 files changed, 216 insertions(+), 13 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache.out b/contrib/pg_buffercache/expected/pg_buffercache.out
index b745dc69eae..c189892f086 100644
--- a/contrib/pg_buffercache/expected/pg_buffercache.out
+++ b/contrib/pg_buffercache/expected/pg_buffercache.out
@@ -8,6 +8,15 @@ from pg_buffercache;
t
(1 row)
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -28,12 +37,16 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts() WHERE buffers >= 0;
SET ROLE pg_database_owner;
SELECT * FROM pg_buffercache;
ERROR: permission denied for view pg_buffercache
-SELECT * FROM pg_buffercache_pages() AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(false) AS p (wrong int);
+ERROR: permission denied for function pg_buffercache_pages
+SELECT * FROM pg_buffercache_pages(true) AS p (wrong int);
ERROR: permission denied for function pg_buffercache_pages
SELECT * FROM pg_buffercache_summary();
ERROR: permission denied for function pg_buffercache_summary
SELECT * FROM pg_buffercache_usage_counts();
ERROR: permission denied for function pg_buffercache_usage_counts
+SELECT * FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
RESET role;
-- Check that pg_monitor is allowed to query view / function
SET ROLE pg_monitor;
@@ -55,3 +68,9 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
t
(1 row)
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..9b2e9393410 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..448d08196f3
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,35 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages(boolean)
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages(false) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_pages(true) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages(boolean) FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages(boolean) TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..dfe53eb8471 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -6,6 +6,7 @@
* contrib/pg_buffercache/pg_buffercache_pages.c
*-------------------------------------------------------------------------
*/
+#include "pg_config.h"
#include "postgres.h"
#include "access/htup_details.h"
@@ -13,10 +14,12 @@
#include "funcapi.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +46,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -65,6 +69,17 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+static void
+pg_buffercache_mark_numa_invalid(BufferCachePagesContext *fctx, int n)
+{
+ int i;
+
+ for (i = 0; i < n; i++)
+ {
+ fctx->record[i].numa_zone_id = -1;
+ }
+}
+
Datum
pg_buffercache_pages(PG_FUNCTION_ARGS)
{
@@ -75,14 +90,33 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
HeapTuple tuple;
+ Buffer query_numa = PG_GETARG_BOOL(0);
if (SRF_IS_FIRSTCALL())
{
- int i;
+ int i,
+ blk2page,
+ j;
+ Size os_page_size;
+ void **os_page_ptrs;
+ int *os_pages_status;
+ int os_page_count;
+ float pages_per_blk;
funcctx = SRF_FIRSTCALL_INIT();
- /* Switch context when allocating stuff to be used in later calls */
+ if (query_numa)
+ {
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
+ }
+ }
+
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
/* Create a user function context for cross-call persistence */
@@ -122,10 +156,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
INT2OID, -1, 0);
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -137,9 +175,35 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
funcctx->max_calls = NBuffers;
funcctx->user_fctx = fctx;
- /* Return to original context when allocating transient memory */
+ /*
+ * Return to original context when allocating transient memory
+ */
MemoryContextSwitchTo(oldcontext);
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first we
+ * need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64)NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA os_page_count=%d os_page_size=%ld pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
/*
* Scan through all the buffers, saving the relevant fields in the
* fctx->record structure.
@@ -171,14 +235,79 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
else
fctx->record[i].isdirty = false;
- /* Note if the buffer is valid, and has storage created */
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
fctx->record[i].isvalid = true;
else
fctx->record[i].isvalid = false;
+ if (query_numa)
+ {
+ blk2page = (int) i * pages_per_blk;
+ j = 0;
+ do
+ {
+ /*
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+ if (os_page_ptrs[blk2page + j] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /*
+ * NBuffers count start really from 1
+ */
+ os_page_ptrs[blk2page + j] = (char *) BufferGetBlock(i + 1) + (os_page_size * j);
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2page + j]);
+
+ /*
+ * Every 1GB of scanned memory we give process chance
+ * to respond
+ */
+#define ONE_GIGABYTE 1024*1024*1024
+ if ((i * os_page_size) % ONE_GIGABYTE == 0)
+ CHECK_FOR_INTERRUPTS();
+ }
+ j++;
+ } while (j < (int) pages_per_blk);
+ }
+
UnlockBufHdr(bufHdr, buf_state);
}
+
+
+ if (query_numa)
+ {
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user
+ *
+ * XXX:: also we could somehow report single DB block spanning
+ * more than 2 NUMA zones, but it should be rare (?)
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
+ }
+ else
+ pg_buffercache_mark_numa_invalid(fctx, NBuffers);
+
+ pfree(os_page_ptrs);
+ pfree(os_pages_status);
}
funcctx = SRF_PERCALL_SETUP();
@@ -209,8 +338,12 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[5] = true;
nulls[6] = true;
nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
nulls[8] = true;
+ nulls[9] = true;
}
else
{
@@ -228,9 +361,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[6] = false;
values[7] = Int16GetDatum(fctx->record[i].usagecount);
nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
}
/* Build and return the tuple. */
diff --git a/contrib/pg_buffercache/sql/pg_buffercache.sql b/contrib/pg_buffercache/sql/pg_buffercache.sql
index 944fbb1beae..96e513a7f98 100644
--- a/contrib/pg_buffercache/sql/pg_buffercache.sql
+++ b/contrib/pg_buffercache/sql/pg_buffercache.sql
@@ -5,6 +5,11 @@ select count(*) = (select setting::bigint
where name = 'shared_buffers')
from pg_buffercache;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -16,9 +21,11 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts() WHERE buffers >= 0;
-- having to create a dedicated user, use the pg_database_owner pseudo-role.
SET ROLE pg_database_owner;
SELECT * FROM pg_buffercache;
-SELECT * FROM pg_buffercache_pages() AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(false) AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(true) AS p (wrong int);
SELECT * FROM pg_buffercache_summary();
SELECT * FROM pg_buffercache_usage_counts();
+SELECT * FROM pg_buffercache_numa;
RESET role;
-- Check that pg_monitor is allowed to query view / function
@@ -26,3 +33,4 @@ SET ROLE pg_monitor;
SELECT count(*) > 0 FROM pg_buffercache;
SELECT buffers_used + buffers_unused > 0 FROM pg_buffercache_summary();
SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
+SELECT count(*) > 0 FROM pg_buffercache_numa;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 03a6dd49154..172309d389a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -562,7 +562,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
[application/octet-stream] v5-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch (10.2K, ../../CAKZiRmz_ERUH0udcZgpAinBSqpWoOYeao_rNNEe9f8Z9asTihw@mail.gmail.com/3-v5-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch)
download | inline diff:
From c548a256e73211dee506762f8efbddcacfc61faf Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v5 1/3] Add optional dependency to libnuma for basic NUMA
awareness routines
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 1 +
configure.ac | 13 ++++
meson.build | 17 +++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 42 +++++++++++
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 150 +++++++++++++++++++++++++++++++++++++
12 files changed, 238 insertions(+)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 91b51142d2e..7467e029131 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -448,6 +448,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/meson.build b/meson.build
index 574f992ed49..cf9dead5d02 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('libnuma', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt, dirs: test_lib_d)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3821,6 +3837,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5aa..bff9f077a8c 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..be85b16b0de
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Miscellaneous functions for bit-wise operations.
+ *
+ *
+ * Copyright (c) 2019-2025, PostgreSQL Global Development Group
+ *
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..e94e68abe42
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+#include <unistd.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* FIXME not tested, might crash */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif;
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
[application/octet-stream] v5-0003-Add-pg_shmem_numa_allocations.patch (10.6K, ../../CAKZiRmz_ERUH0udcZgpAinBSqpWoOYeao_rNNEe9f8Z9asTihw@mail.gmail.com/4-v5-0003-Add-pg_shmem_numa_allocations.patch)
download | inline diff:
From 63b130c3af4de104e0f7e1e8fc2fc990fc71263a Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v5 3/3] Add pg_shmem_numa_allocations
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/privileges.out | 22 +++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/sql/privileges.sql | 7 +-
6 files changed, 170 insertions(+), 4 deletions(-)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index eff0990957e..c808fb82d75 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..48de096f0ea 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -73,6 +73,7 @@
#include "storage/shmem.h"
#include "storage/spin.h"
#include "utils/builtins.h"
+#include "port/pg_numa.h"
static void *ShmemAllocRaw(Size size, Size *allocated_size);
@@ -568,3 +569,127 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ /* Every 1GB of scanned memory we give process chance to respond */
+#define ONE_GIGABYTE 1024*1024*1024
+ if ((i * os_page_size) % ONE_GIGABYTE == 0)
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1) {
+ /* FIXME: should we release LWlock and pfree here? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= pg_numa_get_max_node(); i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): - output shared memory
+ * allocated but not counted via the shmem index - output as-of-yet unused
+ * shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+
+ pfree(page_ptrs);
+ pfree(pages_status);
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9e803d610d7..1efa342b725 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8463,6 +8463,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int8,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 6b01313101b..637f61e6ccc 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3097,8 +3097,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3114,6 +3114,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3127,6 +3133,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
@@ -3141,6 +3153,12 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
t
(1 row)
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
RESET ROLE;
-- clean up
DROP ROLE regress_readallstats;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5baba8d39ff..5cbaa856858 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index 60e7443bf59..38e6909c38c 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1896,8 +1896,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1906,16 +1906,19 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
RESET ROLE;
-- clean up
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-26 17:13 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-02-26 17:13 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Wed, Feb 26, 2025 at 02:05:59PM +0100, Jakub Wartak wrote:
> On Wed, Feb 26, 2025 at 10:58 AM Andres Freund <[email protected]> wrote:
> >
> > Hi,
> >
> > On 2025-02-26 09:38:20 +0100, Jakub Wartak wrote:
> > > > FWIW, what you posted fails on CI:
> > > > https://cirrus-ci.com/task/5114213770723328
> > > >
> > > > Probably some ifdefs are missing. The sanity-check task configures with
> > > > minimal dependencies, which is why you're seeing this even on linux.
> > >
> > > Hopefully fixed, we'll see what cfbot tells, I'm flying blind with all
> > > of this CI stuff...
> >
> > FYI, you can enable CI on a github repo, to see results without posting to the
> > list:
> > https://github.com/postgres/postgres/blob/master/src/tools/ci/README
>
> Thanks, I'll take a look into it.
>
> Meanwhile v5 is attached with slight changes to try to make cfbot happy:
Thanks for the updated version!
FWIW, I had to do a few changes to get an error free compiling experience with
autoconf/or meson and both with or without the libnuma configure option.
Sharing here as .txt files:
v5-0004-configure-changes.txt: changes in configure + add a test on numa.h
availability and a call to numa_available.
v5-0005-pg_numa.c-changes.txt: moving the <unistd.h> outside of USE_LIBNUMA
because the file is still using sysconf() in the non-NUMA code path. Also,
removed a ";" in "#endif;" in the non-NUMA code path.
v5-0006-meson.build-changes.txt.
Those apply on top of your v5.
Also the pg_buffercache test fails without the libnuma configure option. Maybe
some tests should depend of the libnuma configure option.
Still did not look closely to the code.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
From 6871e2d54390d58403bcc36873a5e6e7bf88ed25 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <[email protected]>
Date: Wed, 26 Feb 2025 13:52:15 +0000
Subject: [PATCH v5 4/6] configure changes
---
configure | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 87 insertions(+)
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
--
2.34.1
From ca5449d7091ec724e270b77f5be21e189fa94314 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <[email protected]>
Date: Wed, 26 Feb 2025 15:22:54 +0000
Subject: [PATCH v5 5/6] pg_numa.c changes
---
src/port/pg_numa.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
100.0% src/port/
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index e94e68abe42..3aa1c191f51 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -17,6 +17,7 @@
#include "postgres.h"
#include "port/pg_numa.h"
#include "storage/pg_shmem.h"
+#include <unistd.h>
#ifdef WIN32
#include <windows.h>
#endif
@@ -31,7 +32,6 @@
#include <numa.h>
#include <numaif.h>
-#include <unistd.h>
/* libnuma requires initialization as per numa(3) on Linux */
int
@@ -141,7 +141,7 @@ pg_numa_get_pagesize(void)
SYSTEM_INFO sysinfo;
GetSystemInfo(&sysinfo);
os_page_size = sysinfo.dwPageSize;
-#endif;
+#endif
if (huge_pages_status == HUGE_PAGES_ON)
GetHugePageSize(&os_page_size, NULL);
return os_page_size;
--
2.34.1
From 2de58b2bab5d3c49c285c6d3781923645469e6a2 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <[email protected]>
Date: Wed, 26 Feb 2025 16:41:14 +0000
Subject: [PATCH v5 6/6] meson.build changes
---
meson.build | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index d2178c7d32e..f81092eb661 100644
--- a/meson.build
+++ b/meson.build
@@ -954,9 +954,9 @@ endif
###############################################################
libnumaopt = get_option('libnuma')
-libnuma = dependency('libnuma', required: libnumaopt)
+libnuma = dependency('numa', required: libnumaopt)
if not libnuma.found()
- libnuma = cc.find_library('numa', required: libnumaopt, dirs: test_lib_d)
+ libnuma = cc.find_library('numa', required: libnumaopt)
endif
if libnuma.found()
cdata.set('USE_LIBNUMA', 1)
--
2.34.1
Attachments:
[text/plain] v5-0004-configure-changes.txt (3.3K, ../../[email protected]/2-v5-0004-configure-changes.txt)
download | inline diff:
From 6871e2d54390d58403bcc36873a5e6e7bf88ed25 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <[email protected]>
Date: Wed, 26 Feb 2025 13:52:15 +0000
Subject: [PATCH v5 4/6] configure changes
---
configure | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 87 insertions(+)
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
--
2.34.1
[text/plain] v5-0005-pg_numa.c-changes.txt (1.0K, ../../[email protected]/3-v5-0005-pg_numa.c-changes.txt)
download | inline diff:
From ca5449d7091ec724e270b77f5be21e189fa94314 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <[email protected]>
Date: Wed, 26 Feb 2025 15:22:54 +0000
Subject: [PATCH v5 5/6] pg_numa.c changes
---
src/port/pg_numa.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
100.0% src/port/
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index e94e68abe42..3aa1c191f51 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -17,6 +17,7 @@
#include "postgres.h"
#include "port/pg_numa.h"
#include "storage/pg_shmem.h"
+#include <unistd.h>
#ifdef WIN32
#include <windows.h>
#endif
@@ -31,7 +32,6 @@
#include <numa.h>
#include <numaif.h>
-#include <unistd.h>
/* libnuma requires initialization as per numa(3) on Linux */
int
@@ -141,7 +141,7 @@ pg_numa_get_pagesize(void)
SYSTEM_INFO sysinfo;
GetSystemInfo(&sysinfo);
os_page_size = sysinfo.dwPageSize;
-#endif;
+#endif
if (huge_pages_status == HUGE_PAGES_ON)
GetHugePageSize(&os_page_size, NULL);
return os_page_size;
--
2.34.1
[text/plain] v5-0006-meson.build-changes.txt (863B, ../../[email protected]/4-v5-0006-meson.build-changes.txt)
download | inline diff:
From 2de58b2bab5d3c49c285c6d3781923645469e6a2 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <[email protected]>
Date: Wed, 26 Feb 2025 16:41:14 +0000
Subject: [PATCH v5 6/6] meson.build changes
---
meson.build | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index d2178c7d32e..f81092eb661 100644
--- a/meson.build
+++ b/meson.build
@@ -954,9 +954,9 @@ endif
###############################################################
libnumaopt = get_option('libnuma')
-libnuma = dependency('libnuma', required: libnumaopt)
+libnuma = dependency('numa', required: libnumaopt)
if not libnuma.found()
- libnuma = cc.find_library('numa', required: libnumaopt, dirs: test_lib_d)
+ libnuma = cc.find_library('numa', required: libnumaopt)
endif
if libnuma.found()
cdata.set('USE_LIBNUMA', 1)
--
2.34.1
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-27 09:05 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-02-27 09:05 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Wed, Feb 26, 2025 at 6:13 PM Bertrand Drouvot
<[email protected]> wrote:
[..]
> > Meanwhile v5 is attached with slight changes to try to make cfbot happy:
>
> Thanks for the updated version!
>
> FWIW, I had to do a few changes to get an error free compiling experience with
> autoconf/or meson and both with or without the libnuma configure option.
>
> Sharing here as .txt files:
>
> Also the pg_buffercache test fails without the libnuma configure option. Maybe
> some tests should depend of the libnuma configure option.
[..]
Thank you so much for this Bertrand !
I've applied those , played a little bit with configure and meson and
reproduced the test error and fixed it by silencing that NOTICE in
tests. So v6 is attached even before I get a chance to start using
that CI. Still waiting for some input and tests regarding that earlier
touchpages attempt, docs are still missing...
-J.
Attachments:
[application/octet-stream] v6-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch (11.0K, ../../CAKZiRmxOFTVEtWcOsB6pf36sOohVZZgx4LDGu2ecjXv8kfDniw@mail.gmail.com/2-v6-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch)
download | inline diff:
From 968d02701cce8d0cb1260444ed661d3819a3427a Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v6 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/privileges.out | 25 ++++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/sql/privileges.sql | 10 +-
6 files changed, 176 insertions(+), 4 deletions(-)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index eff0990957e..c808fb82d75 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..48de096f0ea 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -73,6 +73,7 @@
#include "storage/shmem.h"
#include "storage/spin.h"
#include "utils/builtins.h"
+#include "port/pg_numa.h"
static void *ShmemAllocRaw(Size size, Size *allocated_size);
@@ -568,3 +569,127 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ /* Every 1GB of scanned memory we give process chance to respond */
+#define ONE_GIGABYTE 1024*1024*1024
+ if ((i * os_page_size) % ONE_GIGABYTE == 0)
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1) {
+ /* FIXME: should we release LWlock and pfree here? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= pg_numa_get_max_node(); i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): - output shared memory
+ * allocated but not counted via the shmem index - output as-of-yet unused
+ * shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+
+ pfree(page_ptrs);
+ pfree(pages_status);
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9e803d610d7..1efa342b725 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8463,6 +8463,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int8,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 6b01313101b..b20bbcf52b9 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3097,8 +3097,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3114,6 +3114,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3127,6 +3133,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
@@ -3141,6 +3153,15 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
t
(1 row)
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
+RESET client_min_messages;
RESET ROLE;
-- clean up
DROP ROLE regress_readallstats;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5baba8d39ff..5cbaa856858 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index 60e7443bf59..1cf20dfe153 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1896,8 +1896,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1906,16 +1906,22 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+RESET client_min_messages;
RESET ROLE;
-- clean up
--
2.39.5
[application/octet-stream] v6-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch (16.6K, ../../CAKZiRmxOFTVEtWcOsB6pf36sOohVZZgx4LDGu2ecjXv8kfDniw@mail.gmail.com/3-v6-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch)
download | inline diff:
From 7ad8be9e522a9abc95c81c51da332dfb3edc47fc Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v6 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache.out | 30 +++-
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 35 ++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 154 +++++++++++++++++-
contrib/pg_buffercache/sql/pg_buffercache.sql | 19 ++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
9 files changed, 234 insertions(+), 13 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache.out b/contrib/pg_buffercache/expected/pg_buffercache.out
index b745dc69eae..f34f137075e 100644
--- a/contrib/pg_buffercache/expected/pg_buffercache.out
+++ b/contrib/pg_buffercache/expected/pg_buffercache.out
@@ -8,6 +8,18 @@ from pg_buffercache;
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -28,12 +40,19 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts() WHERE buffers >= 0;
SET ROLE pg_database_owner;
SELECT * FROM pg_buffercache;
ERROR: permission denied for view pg_buffercache
-SELECT * FROM pg_buffercache_pages() AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(false) AS p (wrong int);
+ERROR: permission denied for function pg_buffercache_pages
+SELECT * FROM pg_buffercache_pages(true) AS p (wrong int);
ERROR: permission denied for function pg_buffercache_pages
SELECT * FROM pg_buffercache_summary();
ERROR: permission denied for function pg_buffercache_summary
SELECT * FROM pg_buffercache_usage_counts();
ERROR: permission denied for function pg_buffercache_usage_counts
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
SET ROLE pg_monitor;
@@ -55,3 +74,12 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..9b2e9393410 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..448d08196f3
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,35 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages(boolean)
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages(false) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_pages(true) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages(boolean) FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages(boolean) TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..dfe53eb8471 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -6,6 +6,7 @@
* contrib/pg_buffercache/pg_buffercache_pages.c
*-------------------------------------------------------------------------
*/
+#include "pg_config.h"
#include "postgres.h"
#include "access/htup_details.h"
@@ -13,10 +14,12 @@
#include "funcapi.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +46,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -65,6 +69,17 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+static void
+pg_buffercache_mark_numa_invalid(BufferCachePagesContext *fctx, int n)
+{
+ int i;
+
+ for (i = 0; i < n; i++)
+ {
+ fctx->record[i].numa_zone_id = -1;
+ }
+}
+
Datum
pg_buffercache_pages(PG_FUNCTION_ARGS)
{
@@ -75,14 +90,33 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
HeapTuple tuple;
+ Buffer query_numa = PG_GETARG_BOOL(0);
if (SRF_IS_FIRSTCALL())
{
- int i;
+ int i,
+ blk2page,
+ j;
+ Size os_page_size;
+ void **os_page_ptrs;
+ int *os_pages_status;
+ int os_page_count;
+ float pages_per_blk;
funcctx = SRF_FIRSTCALL_INIT();
- /* Switch context when allocating stuff to be used in later calls */
+ if (query_numa)
+ {
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
+ }
+ }
+
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
/* Create a user function context for cross-call persistence */
@@ -122,10 +156,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
INT2OID, -1, 0);
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -137,9 +175,35 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
funcctx->max_calls = NBuffers;
funcctx->user_fctx = fctx;
- /* Return to original context when allocating transient memory */
+ /*
+ * Return to original context when allocating transient memory
+ */
MemoryContextSwitchTo(oldcontext);
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first we
+ * need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64)NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA os_page_count=%d os_page_size=%ld pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
/*
* Scan through all the buffers, saving the relevant fields in the
* fctx->record structure.
@@ -171,14 +235,79 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
else
fctx->record[i].isdirty = false;
- /* Note if the buffer is valid, and has storage created */
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
fctx->record[i].isvalid = true;
else
fctx->record[i].isvalid = false;
+ if (query_numa)
+ {
+ blk2page = (int) i * pages_per_blk;
+ j = 0;
+ do
+ {
+ /*
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+ if (os_page_ptrs[blk2page + j] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /*
+ * NBuffers count start really from 1
+ */
+ os_page_ptrs[blk2page + j] = (char *) BufferGetBlock(i + 1) + (os_page_size * j);
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2page + j]);
+
+ /*
+ * Every 1GB of scanned memory we give process chance
+ * to respond
+ */
+#define ONE_GIGABYTE 1024*1024*1024
+ if ((i * os_page_size) % ONE_GIGABYTE == 0)
+ CHECK_FOR_INTERRUPTS();
+ }
+ j++;
+ } while (j < (int) pages_per_blk);
+ }
+
UnlockBufHdr(bufHdr, buf_state);
}
+
+
+ if (query_numa)
+ {
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user
+ *
+ * XXX:: also we could somehow report single DB block spanning
+ * more than 2 NUMA zones, but it should be rare (?)
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
+ }
+ else
+ pg_buffercache_mark_numa_invalid(fctx, NBuffers);
+
+ pfree(os_page_ptrs);
+ pfree(os_pages_status);
}
funcctx = SRF_PERCALL_SETUP();
@@ -209,8 +338,12 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[5] = true;
nulls[6] = true;
nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
nulls[8] = true;
+ nulls[9] = true;
}
else
{
@@ -228,9 +361,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[6] = false;
values[7] = Int16GetDatum(fctx->record[i].usagecount);
nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
}
/* Build and return the tuple. */
diff --git a/contrib/pg_buffercache/sql/pg_buffercache.sql b/contrib/pg_buffercache/sql/pg_buffercache.sql
index 944fbb1beae..7f2ce683e6c 100644
--- a/contrib/pg_buffercache/sql/pg_buffercache.sql
+++ b/contrib/pg_buffercache/sql/pg_buffercache.sql
@@ -5,6 +5,14 @@ select count(*) = (select setting::bigint
where name = 'shared_buffers')
from pg_buffercache;
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+RESET client_min_messages;
+
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -16,9 +24,14 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts() WHERE buffers >= 0;
-- having to create a dedicated user, use the pg_database_owner pseudo-role.
SET ROLE pg_database_owner;
SELECT * FROM pg_buffercache;
-SELECT * FROM pg_buffercache_pages() AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(false) AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(true) AS p (wrong int);
SELECT * FROM pg_buffercache_summary();
SELECT * FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
@@ -26,3 +39,7 @@ SET ROLE pg_monitor;
SELECT count(*) > 0 FROM pg_buffercache;
SELECT buffers_used + buffers_unused > 0 FROM pg_buffercache_summary();
SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET client_min_messages;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 03a6dd49154..172309d389a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -562,7 +562,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
[application/octet-stream] v6-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch (13.3K, ../../CAKZiRmxOFTVEtWcOsB6pf36sOohVZZgx4LDGu2ecjXv8kfDniw@mail.gmail.com/4-v6-0001-Add-optional-dependency-to-libnuma-for-basic-NUMA.patch)
download | inline diff:
From ae07555fdeb4a8d377ed44925d1ec4795ccf56ce Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v6 1/3] Add optional dependency to libnuma for basic NUMA
awareness routines add minimal src/port/pg_numa.c portability wrapper.
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
---
.cirrus.tasks.yml | 1 +
configure | 87 +++++++++++++++++++++
configure.ac | 13 ++++
meson.build | 17 +++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 42 +++++++++++
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 150 +++++++++++++++++++++++++++++++++++++
13 files changed, 325 insertions(+)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 91b51142d2e..7467e029131 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -448,6 +448,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/meson.build b/meson.build
index 574f992ed49..7302c876b31 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('numa', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3821,6 +3837,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5aa..bff9f077a8c 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..be85b16b0de
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Miscellaneous functions for bit-wise operations.
+ *
+ *
+ * Copyright (c) 2019-2025, PostgreSQL Global Development Group
+ *
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..3aa1c191f51
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+#include <unistd.h>
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* FIXME not tested, might crash */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-27 15:34 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-02-27 15:34 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Thu, Feb 27, 2025 at 10:05:46AM +0100, Jakub Wartak wrote:
> On Wed, Feb 26, 2025 at 6:13 PM Bertrand Drouvot
> <[email protected]> wrote:
> [..]
> > > Meanwhile v5 is attached with slight changes to try to make cfbot happy:
> >
> > Thanks for the updated version!
> >
> > FWIW, I had to do a few changes to get an error free compiling experience with
> > autoconf/or meson and both with or without the libnuma configure option.
> >
> > Sharing here as .txt files:
> >
> > Also the pg_buffercache test fails without the libnuma configure option. Maybe
> > some tests should depend of the libnuma configure option.
> [..]
>
> Thank you so much for this Bertrand !
>
> I've applied those , played a little bit with configure and meson and
> reproduced the test error and fixed it by silencing that NOTICE in
> tests. So v6 is attached even before I get a chance to start using
> that CI. Still waiting for some input and tests regarding that earlier
> touchpages attempt, docs are still missing...
Thanks for the new version!
I did some tests and it looks like it's giving correct results. I don't see -2
anymore and every backend reports correct distribution (with or without hp,
with "small" or "large" shared buffer).
A few random comments:
=== 1
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first we
+ * need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64)NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA os_page_count=%d os_page_size=%ld pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
I think that if (query_numa) check should also wrap that entire section of code.
=== 2
+ if (query_numa)
+ {
+ blk2page = (int) i * pages_per_blk;
+ j = 0;
+ do
+ {
This check is done for every page. I wonder if it would not make sense
to create a brand new function for pg_buffercache_numa and just let the
current pg_buffercache_pages() as it is. That said it would be great to avoid
code duplication as much a possible though, maybe using a shared
populate_buffercache_entry() or such helper function?
=== 3
+#define ONE_GIGABYTE 1024*1024*1024
+ if ((i * os_page_size) % ONE_GIGABYTE == 0)
+ CHECK_FOR_INTERRUPTS();
+ }
Did you observe noticable performance impact if calling CHECK_FOR_INTERRUPTS()
for every page instead? (I don't see with a 30GB shared buffer). I've the
feeling that we could get rid of the "ONE_GIGABYTE" check.
=== 4
+ pfree(os_page_ptrs);
+ pfree(os_pages_status);
Not sure that's needed, we should be in a short-lived memory context here
(ExprContext or such).
=== 5
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
That's a good idea.
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
That sounds right.
Could we also avoid some code duplication with pg_get_shmem_allocations()?
Also same remarks about pfree() and ONE_GIGABYTE as above.
A few other things:
==== 6
+++ b/src/backend/storage/ipc/shmem.c
@@ -73,6 +73,7 @@
#include "storage/shmem.h"
#include "storage/spin.h"
#include "utils/builtins.h"
+#include "port/pg_numa.h"
Not at the right position, should be between those 2:
#include "miscadmin.h"
#include "storage/lwlock.h"
==== 7
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Miscellaneous functions for bit-wise operations.
description is not correct. Also the "Copyright (c) 2019-2025" might be
"Copyright (c) 2025" instead.
=== 8
+++ b/src/port/pg_numa.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * numa.c
+ * Basic NUMA portability routines
s/numa.c/pg_numa.c/ ?
=== 9
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -6,6 +6,7 @@
* contrib/pg_buffercache/pg_buffercache_pages.c
*-------------------------------------------------------------------------
*/
+#include "pg_config.h"
#include "postgres.h"
Is this new include needed?
#include "access/htup_details.h"
@@ -13,10 +14,12 @@
#include "funcapi.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
not in the right order.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-02-27 15:42 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: Bertrand Drouvot @ 2025-02-27 15:42 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Wed, Feb 26, 2025 at 09:38:20AM +0100, Jakub Wartak wrote:
> On Mon, Feb 24, 2025 at 3:06 PM Andres Freund <[email protected]> wrote:
> >
> > Why is this done before we even have gotten -2 back? Even if we need it, it
> > seems like we ought to defer this until necessary.
>
> Not fixed yet: maybe we could even do a `static` with
> `has_this_run_earlier` and just perform this work only once during the
> first time?
Not sure I get your idea, could you share what the code would look like?
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-04 10:48 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-04 10:48 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi!
On Thu, Feb 27, 2025 at 4:34 PM Bertrand Drouvot
<[email protected]> wrote:
> I did some tests and it looks like it's giving correct results. I don't see -2
> anymore and every backend reports correct distribution (with or without hp,
> with "small" or "large" shared buffer).
Cool! Attached is v7 that is fully green on cirrus CI that Andres
recommended, we will see how cfbot reacts to this. BTW docs are still
missing. When started with proper interleave=all with s_b=64GB,
hugepages and 4 NUMA nodes (1socket with 4 CCDs) after small pgbench:
postgres=# select buffers_used, buffers_unused from pg_buffercache_summary();
buffers_used | buffers_unused
--------------+----------------
170853 | 8217755
(
ostgres=# select numa_zone_id, count(*) from pg_buffercache_numa group
by numa_zone_id order by numa_zone_id;
DEBUG: NUMA: os_page_count=32768 os_page_size=2097152 pages_per_blk=0.003906
numa_zone_id | count
--------------+---------
0 | 42752
1 | 42752
2 | 42752
3 | 42597
| 8217755
Time: 5828.100 ms (00:05.828)
postgres=# select 3*42752+42597;
?column?
----------
170853
postgres=# select * from pg_shmem_numa_allocations order by numa_size
desc limit 12;
DEBUG: NUMA: page-faulting shared memory segments for proper NUMA readouts
name | numa_zone_id | numa_size
--------------------+--------------+-------------
Buffer Blocks | 0 | 17179869184
Buffer Blocks | 1 | 17179869184
Buffer Blocks | 3 | 17179869184
Buffer Blocks | 2 | 17179869184
Buffer Descriptors | 2 | 134217728
Buffer Descriptors | 1 | 134217728
Buffer Descriptors | 0 | 134217728
Buffer Descriptors | 3 | 134217728
Checkpointer Data | 1 | 67108864
Checkpointer Data | 0 | 67108864
Checkpointer Data | 2 | 67108864
Checkpointer Data | 3 | 67108864
Time: 68.579 ms
> A few random comments:
>
> === 1
>
[..]
> I think that the if (query_numa) check should also wrap that entire section of code.
Done.
> === 2
>
> + if (query_numa)
> + {
> + blk2page = (int) i * pages_per_blk;
> + j = 0;
> + do
> + {
>
> This check is done for every page. I wonder if it would not make sense
> to create a brand new function for pg_buffercache_numa and just let the
> current pg_buffercache_pages() as it is. That said it would be great to avoid
> code duplication as much a possible though, maybe using a shared
> populate_buffercache_entry() or such helper function?
Well, I've made query_numa a parameter there simply to avoid that code
duplication in the first place, look at those TupleDescInitEntry()...
IMHO rarely anybody uses pg_buffercache, but we could add unlikely()
there maybe to hint compiler with some smaller routine to reduce
complexity of that main routine? (assuming NUMA inquiry is going to be
rare).
> === 3
>
> +#define ONE_GIGABYTE 1024*1024*1024
> + if ((i * os_page_size) % ONE_GIGABYTE == 0)
> + CHECK_FOR_INTERRUPTS();
> + }
>
> Did you observe noticable performance impact if calling CHECK_FOR_INTERRUPTS()
> for every page instead? (I don't see with a 30GB shared buffer). I've the
> feeling that we could get rid of the "ONE_GIGABYTE" check.
You are right and no it was simply my premature optimization attempt,
as apparently CFI on closer looks seems to be already having
unlikely() and looks really cheap, so yeah I've removed that.
> === 4
>
> + pfree(os_page_ptrs);
> + pfree(os_pages_status);
>
> Not sure that's needed, we should be in a short-lived memory context here
> (ExprContext or such).
Yes, I wanted to have it just for illustrative and stylish purposes,
but right, removed.
> === 5
>
> +/* SQL SRF showing NUMA zones for allocated shared memory */
> +Datum
> +pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
> +{
[..]
>
> + for (i = 0; i < shm_ent_page_count; i++)
> + {
> + /*
> + * In order to get reliable results we also need to touch memory
> + * pages so that inquiry about NUMA zone doesn't return -2.
> + */
> + volatile uint64 touch pg_attribute_unused();
> +
> + page_ptrs[i] = (char *) ent->location + (i * os_page_size);
> + pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
>
> That sounds right.
>
> Could we also avoid some code duplication with pg_get_shmem_allocations()?
Not sure I understand do you want to avoid code duplication
pg_get_shmem_allocations() vs pg_get_shmem_numa_allocations() or
pg_get_shmem_numa_allocations() vs pg_buffercache_pages(query_numa =
true) ?
>
> Also same remarks about pfree() and ONE_GIGABYTE as above.
Fixed.
> A few other things:
>
> ==== 6
>
> +#include "port/pg_numa.h"
> Not at the right position, should be between those 2:
>
> #include "miscadmin.h"
> #include "storage/lwlock.h"
Fixed.
> ==== 7
>
> +/*-------------------------------------------------------------------------
> + *
> + * pg_numa.h
> + * Miscellaneous functions for bit-wise operations.
>
> description is not correct. Also the "Copyright (c) 2019-2025" might be
> "Copyright (c) 2025" instead.
Fixed.
> === 8
>
> +++ b/src/port/pg_numa.c
> @@ -0,0 +1,150 @@
> +/*-------------------------------------------------------------------------
> + *
> + * numa.c
> + * Basic NUMA portability routines
>
> s/numa.c/pg_numa.c/ ?
Fixed.
> === 9
>
> +++ b/contrib/pg_buffercache/pg_buffercache_pages.c
> @@ -6,6 +6,7 @@
> * contrib/pg_buffercache/pg_buffercache_pages.c
> *-------------------------------------------------------------------------
> */
> +#include "pg_config.h"
> #include "postgres.h"
>
> Is this new include needed?
Removed, don't remember how it arrived here, most have been some
artifact of earlier attempts.
> #include "access/htup_details.h"
> @@ -13,10 +14,12 @@
> #include "funcapi.h"
> #include "storage/buf_internals.h"
> #include "storage/bufmgr.h"
> +#include "port/pg_numa.h"
> +#include "storage/pg_shmem.h"
>
> not in the right order.
Fixed.
And also those from nearby message:
On Thu, Feb 27, 2025 at 4:42 PM Bertrand Drouvot
<[email protected]> wrote:
> On Wed, Feb 26, 2025 at 09:38:20AM +0100, Jakub Wartak wrote:
> > On Mon, Feb 24, 2025 at 3:06 PM Andres Freund <[email protected]> wrote:
> > >
> > > Why is this done before we even have gotten -2 back? Even if we need it, it
> > > seems like we ought to defer this until necessary.
> >
> > Not fixed yet: maybe we could even do a `static` with
> > `has_this_run_earlier` and just perform this work only once during the
> > first time?
>
> Not sure I get your idea, could you share what the code would look like?
Please see pg_buffercache_pages I've just added static bool firstUseInBackend:
postgres@postgres:1234 : 25103 # select numa_zone_id, count(*) from
pg_buffercache_numa group by numa_zone_id;
DEBUG: NUMA: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
DEBUG: NUMA: page-faulting the buffercache for proper NUMA readouts
[..]
postgres@postgres:1234 : 25103 # select numa_zone_id, count(*) from
pg_buffercache_numa group by numa_zone_id;
DEBUG: NUMA: os_page_count=32768 os_page_size=4096 pages_per_blk=2.000000
numa_zone_id | count
[..]
Same was done to the pg_get_shmem_numa_allocations.
Also CFbot/cirrus was getting:
> [11:01:05.027] In file included from ../../src/include/postgres.h:49,
> [11:01:05.027] from pg_buffercache_pages.c:10:
> [11:01:05.027] pg_buffercache_pages.c: In function ‘pg_buffercache_pages’:
> [11:01:05.027] pg_buffercache_pages.c:194:30: error: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘Size’ {aka ‘long long unsigned int’} [-Werror=format=]
Fixed with %zu (for size_t) instead of %ld.
> Linux - Debian Bookworm - Autoconf got:
> [10:42:59.216] checking numa.h usability... no
> [10:42:59.268] checking numa.h presence... no
> [10:42:59.286] checking for numa.h... no
> [10:42:59.286] configure: error: header file <numa.h> is required for --with-libnuma
I've added libnuma1 to cirrus in a similar vein like libcurl to avoid this.
> [13:50:47.449] gcc -m32 @src/backend/postgres.rsp
> [13:50:47.449] /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libnuma.so: error adding symbols: file in wrong format
I've also got an error in 32-bit build as libnuma.h is there, but
apparently libnuma provides only x86_64. Anyway 32-bit(even with PAE)
and NUMA doesn't seem to make sense so I've added -Dlibnuma=off for
such build.
-J.
Attachments:
[application/octet-stream] v7-0001-Add-optional-dependency-to-libnuma-Linux-only-for.patch (14.4K, ../../CAKZiRmwiLyLbLjoWsC7KrdFha9XcHyfG3BMmVQYRxvdJwgSq_A@mail.gmail.com/2-v7-0001-Add-optional-dependency-to-libnuma-Linux-only-for.patch)
download | inline diff:
From 9795cf53205c6fb1be9ae5364664d6ba7b2baa65 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v7 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper.
Other platforms can be added later.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway as i386 is is very memory-only
limited even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 7 +-
configure | 87 +++++++++++++++++++++
configure.ac | 13 ++++
meson.build | 17 +++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 43 +++++++++++
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 150 +++++++++++++++++++++++++++++++++++++
13 files changed, 331 insertions(+), 1 deletion(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 91b51142d2..584e3e5a44 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -428,6 +428,8 @@ task:
DEBIAN_FRONTEND=noninteractive apt-get -y install \
libcurl4-openssl-dev \
libcurl4-openssl-dev:i386 \
+ libnuma1 \
+ libnuma-dev
matrix:
- name: Linux - Debian Bookworm - Autoconf
@@ -448,6 +450,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -492,6 +495,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -804,7 +808,8 @@ task:
setup_additional_packages_script: |
apt-get update
- DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev \
+ libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd6998..23c33dd997 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc..1a394dfc07 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/meson.build b/meson.build
index 13c13748e5..f81092eb66 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('numa', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3839,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c451714..adaadb5faf 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5a..0bd4b2d7d3 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5a..bff9f077a8 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d..8894f80060 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 0000000000..d3ebe8b5bd
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d5023..f786c19160 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c22431951..a68a29d541 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d4..7ffbd4d88d 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 0000000000..db28578bca
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+#include <unistd.h>
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* FIXME not tested, might crash */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
[application/octet-stream] v7-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch (11.1K, ../../CAKZiRmwiLyLbLjoWsC7KrdFha9XcHyfG3BMmVQYRxvdJwgSq_A@mail.gmail.com/3-v7-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch)
download | inline diff:
From 977c18b0ad8abfcd81a557c9559ae911f4752aac Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v7 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/privileges.out | 25 ++++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/sql/privileges.sql | 10 +-
6 files changed, 176 insertions(+), 4 deletions(-)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf..cc014a62dc 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39..7d83a14390 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port/pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -568,3 +569,127 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count;
+ static bool firstUseInBackend = true;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ {
+ /* FIXME: should we release LWlock here ? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= pg_numa_get_max_node(); i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cd9422d0ba..62f051c194 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8463,6 +8463,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int8,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index a76256405f..02997690e1 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
@@ -3171,6 +3183,15 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
t
(1 row)
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
+RESET client_min_messages;
RESET ROLE;
-- clean up
DROP ROLE regress_readallstats;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b..b63c6e0f74 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index d195aaf137..e969cc3854 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,16 +1921,22 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+RESET client_min_messages;
RESET ROLE;
-- clean up
--
2.39.5
[application/octet-stream] v7-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch (16.8K, ../../CAKZiRmwiLyLbLjoWsC7KrdFha9XcHyfG3BMmVQYRxvdJwgSq_A@mail.gmail.com/4-v7-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch)
download | inline diff:
From 77149659f37dd8943a85ab5cf61c96c6cc9dcebd Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v7 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache.out | 30 +++-
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 35 ++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 156 +++++++++++++++++-
contrib/pg_buffercache/sql/pg_buffercache.sql | 19 ++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
9 files changed, 237 insertions(+), 12 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e..2a33602537 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache.out b/contrib/pg_buffercache/expected/pg_buffercache.out
index b745dc69ea..f34f137075 100644
--- a/contrib/pg_buffercache/expected/pg_buffercache.out
+++ b/contrib/pg_buffercache/expected/pg_buffercache.out
@@ -8,6 +8,18 @@ from pg_buffercache;
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -28,12 +40,19 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts() WHERE buffers >= 0;
SET ROLE pg_database_owner;
SELECT * FROM pg_buffercache;
ERROR: permission denied for view pg_buffercache
-SELECT * FROM pg_buffercache_pages() AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(false) AS p (wrong int);
+ERROR: permission denied for function pg_buffercache_pages
+SELECT * FROM pg_buffercache_pages(true) AS p (wrong int);
ERROR: permission denied for function pg_buffercache_pages
SELECT * FROM pg_buffercache_summary();
ERROR: permission denied for function pg_buffercache_summary
SELECT * FROM pg_buffercache_usage_counts();
ERROR: permission denied for function pg_buffercache_usage_counts
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
SET ROLE pg_monitor;
@@ -55,3 +74,12 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe4871..9b2e939341 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 0000000000..448d08196f
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,35 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages(boolean)
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages(false) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_pages(true) AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages(boolean) FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages(boolean) TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77d..b030ba3a6f 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e1..f32546fdee 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,14 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +45,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -65,6 +68,52 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+static void
+pg_buffercache_mark_numa_invalid(BufferCachePagesContext *fctx, int n)
+{
+ int i;
+
+ for (i = 0; i < n; i++)
+ {
+ fctx->record[i].numa_zone_id = -1;
+ }
+}
+
+/*
+* Many buffers can point to the same page (in case of
+* BLCKSZ < 4kB), but we want to also query just first
+* address.
+*
+* In order to get reliable results we also need to touch
+* memory pages, so that inquiry about NUMA zone doesn't
+* return -2.
+*/
+static inline void
+pg_buffercache_numa_prepare_ptrs(int i, float pages_per_blk, Size os_page_size,
+ void **os_page_ptrs, bool firstUseInBackend)
+{
+ int j = 0,
+ blk2page = (int) i * pages_per_blk;
+
+ do
+ {
+ if (os_page_ptrs[blk2page + j] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers count start really from 1 */
+ os_page_ptrs[blk2page + j] = (char *) BufferGetBlock(i + 1) + (os_page_size * j);
+
+ /* We just need to do it only once in backend */
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2page + j]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ j++;
+ } while (j < (int) pages_per_blk);
+}
+
Datum
pg_buffercache_pages(PG_FUNCTION_ARGS)
{
@@ -75,14 +124,32 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
HeapTuple tuple;
+ Buffer query_numa = PG_GETARG_BOOL(0);
+ static bool firstUseInBackend = true;
if (SRF_IS_FIRSTCALL())
{
int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ int os_page_count = 0;
+ float pages_per_blk = 0;
funcctx = SRF_FIRSTCALL_INIT();
- /* Switch context when allocating stuff to be used in later calls */
+ if (query_numa)
+ {
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
+ }
+ }
+
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
/* Create a user function context for cross-call persistence */
@@ -122,10 +189,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
INT2OID, -1, 0);
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -137,9 +208,41 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
funcctx->max_calls = NBuffers;
funcctx->user_fctx = fctx;
- /* Return to original context when allocating transient memory */
+ /*
+ * Return to original context when allocating transient memory
+ */
MemoryContextSwitchTo(oldcontext);
+ if (query_numa)
+ {
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+ }
+
/*
* Scan through all the buffers, saving the relevant fields in the
* fctx->record structure.
@@ -171,14 +274,41 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
else
fctx->record[i].isdirty = false;
- /* Note if the buffer is valid, and has storage created */
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
fctx->record[i].isvalid = true;
else
fctx->record[i].isvalid = false;
+ if (unlikely(query_numa))
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
+
UnlockBufHdr(bufHdr, buf_state);
}
+
+
+ if (query_numa)
+ {
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user. Also we could somehow report single DB block spanning
+ * more than one NUMA zone, but it should be rare.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
+ }
+ else
+ pg_buffercache_mark_numa_invalid(fctx, NBuffers);
+
}
funcctx = SRF_PERCALL_SETUP();
@@ -209,8 +339,12 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[5] = true;
nulls[6] = true;
nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
nulls[8] = true;
+ nulls[9] = true;
}
else
{
@@ -228,9 +362,14 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
nulls[6] = false;
values[7] = Int16GetDatum(fctx->record[i].usagecount);
nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
}
/* Build and return the tuple. */
@@ -240,7 +379,10 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstUseInBackend = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache.sql b/contrib/pg_buffercache/sql/pg_buffercache.sql
index 944fbb1bea..7f2ce683e6 100644
--- a/contrib/pg_buffercache/sql/pg_buffercache.sql
+++ b/contrib/pg_buffercache/sql/pg_buffercache.sql
@@ -5,6 +5,14 @@ select count(*) = (select setting::bigint
where name = 'shared_buffers')
from pg_buffercache;
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+RESET client_min_messages;
+
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -16,9 +24,14 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts() WHERE buffers >= 0;
-- having to create a dedicated user, use the pg_database_owner pseudo-role.
SET ROLE pg_database_owner;
SELECT * FROM pg_buffercache;
-SELECT * FROM pg_buffercache_pages() AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(false) AS p (wrong int);
+SELECT * FROM pg_buffercache_pages(true) AS p (wrong int);
SELECT * FROM pg_buffercache_summary();
SELECT * FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
@@ -26,3 +39,7 @@ SET ROLE pg_monitor;
SELECT count(*) > 0 FROM pg_buffercache;
SELECT buffers_used + buffers_unused > 0 FROM pg_buffercache_summary();
SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET client_min_messages;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ad25cbb39c..dd34c79f52 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86..5f7d4b83a6 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-04 16:02 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-04 16:02 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Tue, Mar 04, 2025 at 11:48:31AM +0100, Jakub Wartak wrote:
> Hi!
>
> On Thu, Feb 27, 2025 at 4:34 PM Bertrand Drouvot
> <[email protected]> wrote:
>
> > I did some tests and it looks like it's giving correct results. I don't see -2
> > anymore and every backend reports correct distribution (with or without hp,
> > with "small" or "large" shared buffer).
>
> Cool! Attached is v7
Thanks for the new version!
> > === 2
> >
> > + if (query_numa)
> > + {
> > + blk2page = (int) i * pages_per_blk;
> > + j = 0;
> > + do
> > + {
> >
> > This check is done for every page. I wonder if it would not make sense
> > to create a brand new function for pg_buffercache_numa and just let the
> > current pg_buffercache_pages() as it is. That said it would be great to avoid
> > code duplication as much a possible though, maybe using a shared
> > populate_buffercache_entry() or such helper function?
>
> Well, I've made query_numa a parameter there simply to avoid that code
> duplication in the first place, look at those TupleDescInitEntry()...
Yeah, that's why I was mentioning to use a "shared" populate_buffercache_entry()
or such function: to put the "duplicated" code in it and then use this
shared function in pg_buffercache_pages() and in the new numa related one.
> IMHO rarely anybody uses pg_buffercache, but we could add unlikely()
I think unlikely() should be used for optimization based on code path likelihood,
not based on how often users might use a feature.
> > === 5
> >
> > Could we also avoid some code duplication with pg_get_shmem_allocations()?
>
> Not sure I understand do you want to avoid code duplication
> pg_get_shmem_allocations() vs pg_get_shmem_numa_allocations() or
> pg_get_shmem_numa_allocations() vs pg_buffercache_pages(query_numa =
> true) ?
I meant to say avoid code duplication between pg_get_shmem_allocations() and
pg_get_shmem_numa_allocations(). It might be possible to create a shared
function for them too. That said, it looks like that the savings (if any), would
not be that much, so maybe just forget about it.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-05 09:30 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-05 09:30 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Tue, Mar 4, 2025 at 5:02 PM Bertrand Drouvot
<[email protected]> wrote:
> > Cool! Attached is v7
> Thanks for the new version!
... and another one: 7b ;)
> > > === 2
[..]
> > Well, I've made query_numa a parameter there simply to avoid that code
> > duplication in the first place, look at those TupleDescInitEntry()...
>
> Yeah, that's why I was mentioning to use a "shared" populate_buffercache_entry()
> or such function: to put the "duplicated" code in it and then use this
> shared function in pg_buffercache_pages() and in the new numa related one.
OK, so hastily attempted that in 7b , I had to do a larger refactor
there to avoid code duplication between those two. I don't know which
attempt is better though (7 vs 7b)..
> > IMHO rarely anybody uses pg_buffercache, but we could add unlikely()
>
> I think unlikely() should be used for optimization based on code path likelihood,
> not based on how often users might use a feature.
In 7b I've removed the unlikely() - For a moment I was thinking that
you are concerned about this loop for NBuffers to be as much optimized
as it can and that's the reason for splitting the routines.
> > > === 5
[..]
> I meant to say avoid code duplication between pg_get_shmem_allocations() and
> pg_get_shmem_numa_allocations(). It might be possible to create a shared
> function for them too. That said, it looks like that the savings (if any), would
> not be that much, so maybe just forget about it.
Yeah, OK, so let's leave it at that.
-J.
Attachments:
[application/octet-stream] v7b-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch (11.1K, ../../CAKZiRmyMrynZpEdbrYx2qRB=-_FREYc3oXJnNdTmO36VR3NvpQ@mail.gmail.com/2-v7b-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch)
download | inline diff:
From 65dbe35271a3800037858058dcec77b186e0df05 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v7b 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/privileges.out | 25 ++++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/sql/privileges.sql | 10 +-
6 files changed, 176 insertions(+), 4 deletions(-)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf..cc014a62dc 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39..7d83a14390 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port/pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -568,3 +569,127 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count;
+ static bool firstUseInBackend = true;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ {
+ /* FIXME: should we release LWlock here ? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= pg_numa_get_max_node(); i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cd9422d0ba..62f051c194 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8463,6 +8463,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int8,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index a76256405f..02997690e1 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
@@ -3171,6 +3183,15 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
t
(1 row)
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
+RESET client_min_messages;
RESET ROLE;
-- clean up
DROP ROLE regress_readallstats;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b..b63c6e0f74 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index d195aaf137..e969cc3854 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,16 +1921,22 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+RESET client_min_messages;
RESET ROLE;
-- clean up
--
2.39.5
[application/octet-stream] v7b-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (26.3K, ../../CAKZiRmyMrynZpEdbrYx2qRB=-_FREYc3oXJnNdTmO36VR3NvpQ@mail.gmail.com/3-v7b-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From 84dcc240c51c568a192da8d11a49144b027a87de Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v7b 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache.out | 26 +
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 470 +++++++++++++-----
contrib/pg_buffercache/sql/pg_buffercache.sql | 16 +
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
9 files changed, 430 insertions(+), 133 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e..2a33602537 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache.out b/contrib/pg_buffercache/expected/pg_buffercache.out
index b745dc69ea..4da569c20a 100644
--- a/contrib/pg_buffercache/expected/pg_buffercache.out
+++ b/contrib/pg_buffercache/expected/pg_buffercache.out
@@ -8,6 +8,18 @@ from pg_buffercache;
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -34,6 +46,11 @@ SELECT * FROM pg_buffercache_summary();
ERROR: permission denied for function pg_buffercache_summary
SELECT * FROM pg_buffercache_usage_counts();
ERROR: permission denied for function pg_buffercache_usage_counts
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
SET ROLE pg_monitor;
@@ -55,3 +72,12 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe4871..9b2e939341 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 0000000000..52f63aa258
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77d..b030ba3a6f 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e1..96b14f9ed4 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,14 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +45,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -61,84 +64,250 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine to map Buffers into addresses that can be
+ * later consumed by pg_numa_query_pages()
+ *
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int i, float pages_per_blk, Size os_page_size,
+ void **os_page_ptrs, bool firstUseInBackend)
+{
+ int j = 0,
+ blk2page = (int) i * pages_per_blk;
+
+ do
+ {
+ if (os_page_ptrs[blk2page + j] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers count start really from 1 */
+ os_page_ptrs[blk2page + j] = (char *) BufferGetBlock(i + 1) + (os_page_size * j);
+
+ /* We just need to do it only once in backend */
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2page + j]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ j++;
+ } while (j < (int) pages_per_blk);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages.
+ *
+ * We need fcinfo here and we pass it here with PG_FUNCTION_ARGS
+ */
+static BufferCachePagesContext *
+init_buffercache_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static void
+populate_buffercache_entry(int i, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(i);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[i].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[i].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[i].blocknum = bufHdr->tag.blockNum;
+ fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[i].isdirty = true;
+ else
+ fctx->record[i].isdirty = false;
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[i].isvalid = true;
+ else
+ fctx->record[i].isvalid = false;
+
+ fctx->record[i].numa_zone_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static Datum
+get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[i].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[i].blocknum == InvalidBlockNumber ||
+ fctx->record[i].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[i].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[i].usagecount);
+ nulls[7] = false;
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
+ }
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
+
+/*
+ * When updating this routine please sync it with below one:
+ * pg_buffercache_numa_pages()
+ */
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = init_buffercache_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +318,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ populate_buffercache_entry(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +328,129 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+ bool query_numa = true;
+ static bool firstUseInBackend = true;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ int os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
+ }
+ fctx = init_buffercache_entries(funcctx, fcinfo);
+
+ if (query_numa)
+ {
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+ }
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ populate_buffercache_entry(i, fctx);
+ if (query_numa)
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
}
- else
+
+ if (query_numa)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user. Also we could somehow report single DB block spanning
+ * more than one NUMA zone, but it should be rare.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
}
+ }
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ funcctx = SRF_PERCALL_SETUP();
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstUseInBackend = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache.sql b/contrib/pg_buffercache/sql/pg_buffercache.sql
index 944fbb1bea..d982048425 100644
--- a/contrib/pg_buffercache/sql/pg_buffercache.sql
+++ b/contrib/pg_buffercache/sql/pg_buffercache.sql
@@ -5,6 +5,14 @@ select count(*) = (select setting::bigint
where name = 'shared_buffers')
from pg_buffercache;
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+RESET client_min_messages;
+
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -19,6 +27,10 @@ SELECT * FROM pg_buffercache;
SELECT * FROM pg_buffercache_pages() AS p (wrong int);
SELECT * FROM pg_buffercache_summary();
SELECT * FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
@@ -26,3 +38,7 @@ SET ROLE pg_monitor;
SELECT count(*) > 0 FROM pg_buffercache;
SELECT buffers_used + buffers_unused > 0 FROM pg_buffercache_summary();
SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET client_min_messages;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ad25cbb39c..dd34c79f52 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86..5f7d4b83a6 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
[application/octet-stream] v7b-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (14.4K, ../../CAKZiRmyMrynZpEdbrYx2qRB=-_FREYc3oXJnNdTmO36VR3NvpQ@mail.gmail.com/4-v7b-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From 9795cf53205c6fb1be9ae5364664d6ba7b2baa65 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v7b 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper.
Other platforms can be added later.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway as i386 is is very memory-only
limited even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 7 +-
configure | 87 +++++++++++++++++++++
configure.ac | 13 ++++
meson.build | 17 +++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 43 +++++++++++
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 150 +++++++++++++++++++++++++++++++++++++
13 files changed, 331 insertions(+), 1 deletion(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 91b51142d2..584e3e5a44 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -428,6 +428,8 @@ task:
DEBIAN_FRONTEND=noninteractive apt-get -y install \
libcurl4-openssl-dev \
libcurl4-openssl-dev:i386 \
+ libnuma1 \
+ libnuma-dev
matrix:
- name: Linux - Debian Bookworm - Autoconf
@@ -448,6 +450,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -492,6 +495,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -804,7 +808,8 @@ task:
setup_additional_packages_script: |
apt-get update
- DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev \
+ libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd6998..23c33dd997 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc..1a394dfc07 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/meson.build b/meson.build
index 13c13748e5..f81092eb66 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('numa', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3839,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c451714..adaadb5faf 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5a..0bd4b2d7d3 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5a..bff9f077a8 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d..8894f80060 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 0000000000..d3ebe8b5bd
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d5023..f786c19160 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c22431951..a68a29d541 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d4..7ffbd4d88d 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 0000000000..db28578bca
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+#include <unistd.h>
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* FIXME not tested, might crash */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-07 10:20 Jakub Wartak <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-07 10:20 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Wed, Mar 5, 2025 at 10:30 AM Jakub Wartak
<[email protected]> wrote:
>Hi,
> > Yeah, that's why I was mentioning to use a "shared" populate_buffercache_entry()
> > or such function: to put the "duplicated" code in it and then use this
> > shared function in pg_buffercache_pages() and in the new numa related one.
>
> OK, so hastily attempted that in 7b , I had to do a larger refactor
> there to avoid code duplication between those two. I don't know which
> attempt is better though (7 vs 7b)..
>
I'm attaching basically the earlier stuff (v7b) as v8 with the
following minor changes:
- docs are included
- changed int8 to int4 in one function definition for numa_zone_id
-J.
Attachments:
[application/octet-stream] v8-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch (29.5K, ../../CAKZiRmy-4ZuuZ+mYwcC+3-n46zy90vqRxZcXShLzwsopx5j50A@mail.gmail.com/2-v8-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch)
download | inline diff:
From 53b6f8925c7367ab668b39338507942388dbbeca Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v8 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache.out | 26 +
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 470 +++++++++++++-----
contrib/pg_buffercache/sql/pg_buffercache.sql | 16 +
doc/src/sgml/pgbuffercache.sgml | 64 ++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
10 files changed, 493 insertions(+), 134 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e..2a33602537 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache.out b/contrib/pg_buffercache/expected/pg_buffercache.out
index b745dc69ea..4da569c20a 100644
--- a/contrib/pg_buffercache/expected/pg_buffercache.out
+++ b/contrib/pg_buffercache/expected/pg_buffercache.out
@@ -8,6 +8,18 @@ from pg_buffercache;
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -34,6 +46,11 @@ SELECT * FROM pg_buffercache_summary();
ERROR: permission denied for function pg_buffercache_summary
SELECT * FROM pg_buffercache_usage_counts();
ERROR: permission denied for function pg_buffercache_usage_counts
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
SET ROLE pg_monitor;
@@ -55,3 +72,12 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe4871..9b2e939341 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 0000000000..52f63aa258
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77d..b030ba3a6f 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e1..96b14f9ed4 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,14 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +45,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -61,84 +64,250 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine to map Buffers into addresses that can be
+ * later consumed by pg_numa_query_pages()
+ *
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int i, float pages_per_blk, Size os_page_size,
+ void **os_page_ptrs, bool firstUseInBackend)
+{
+ int j = 0,
+ blk2page = (int) i * pages_per_blk;
+
+ do
+ {
+ if (os_page_ptrs[blk2page + j] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers count start really from 1 */
+ os_page_ptrs[blk2page + j] = (char *) BufferGetBlock(i + 1) + (os_page_size * j);
+
+ /* We just need to do it only once in backend */
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2page + j]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ j++;
+ } while (j < (int) pages_per_blk);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages.
+ *
+ * We need fcinfo here and we pass it here with PG_FUNCTION_ARGS
+ */
+static BufferCachePagesContext *
+init_buffercache_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static void
+populate_buffercache_entry(int i, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(i);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[i].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[i].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[i].blocknum = bufHdr->tag.blockNum;
+ fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[i].isdirty = true;
+ else
+ fctx->record[i].isdirty = false;
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[i].isvalid = true;
+ else
+ fctx->record[i].isvalid = false;
+
+ fctx->record[i].numa_zone_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static Datum
+get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[i].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[i].blocknum == InvalidBlockNumber ||
+ fctx->record[i].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[i].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[i].usagecount);
+ nulls[7] = false;
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
+ }
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
+
+/*
+ * When updating this routine please sync it with below one:
+ * pg_buffercache_numa_pages()
+ */
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = init_buffercache_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +318,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ populate_buffercache_entry(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +328,129 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+ bool query_numa = true;
+ static bool firstUseInBackend = true;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ int os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
+ }
+ fctx = init_buffercache_entries(funcctx, fcinfo);
+
+ if (query_numa)
+ {
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+ }
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ populate_buffercache_entry(i, fctx);
+ if (query_numa)
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
}
- else
+
+ if (query_numa)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user. Also we could somehow report single DB block spanning
+ * more than one NUMA zone, but it should be rare.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
}
+ }
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ funcctx = SRF_PERCALL_SETUP();
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstUseInBackend = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache.sql b/contrib/pg_buffercache/sql/pg_buffercache.sql
index 944fbb1bea..d982048425 100644
--- a/contrib/pg_buffercache/sql/pg_buffercache.sql
+++ b/contrib/pg_buffercache/sql/pg_buffercache.sql
@@ -5,6 +5,14 @@ select count(*) = (select setting::bigint
where name = 'shared_buffers')
from pg_buffercache;
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+RESET client_min_messages;
+
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -19,6 +27,10 @@ SELECT * FROM pg_buffercache;
SELECT * FROM pg_buffercache_pages() AS p (wrong int);
SELECT * FROM pg_buffercache_summary();
SELECT * FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
@@ -26,3 +38,7 @@ SET ROLE pg_monitor;
SELECT count(*) > 0 FROM pg_buffercache;
SELECT buffers_used + buffers_unused > 0 FROM pg_buffercache_summary();
SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET client_min_messages;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d7..75978a6eae 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,13 @@
convenient use.
</para>
+ <para>
+ The similiar <function>pg_buffercache_numa_pages()</function> is a slower
+ variant of the above, but also can provide NUMA node ID for shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +209,59 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are almost identical to the previous
+ <structname>pg_buffercache</structname> view, but this one includes one additional
+ column numa_zone_id as defined in <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ ID (number) of the NUMA node for this particular buffer. NULL if the shared buffer
+ has not been used yet.On systems without NUMA this usually returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ This is clone version of the original pg_buffercache view, however it provides
+ additional <structfield>numa_zone_id</structfield> column. Fetching this
+ information from OS is costly and might take much longer and querying it is not
+ recommended by automated or monitoring systems.
+ </para>
+
+ <para>
+ As NUMA node ID inquiry for each page requires memory pages to be paged-in, first
+ execution of this function can take long time especially on systems with bigint
+ shared_buffers and without huge_pages enabled.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ad25cbb39c..dd34c79f52 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86..5f7d4b83a6 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
[application/octet-stream] v8-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch (14.0K, ../../CAKZiRmy-4ZuuZ+mYwcC+3-n46zy90vqRxZcXShLzwsopx5j50A@mail.gmail.com/3-v8-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch)
download | inline diff:
From 0d0319a32a7ea5689d10fe8de7cf5848d547cadf Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v8 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 78 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/privileges.out | 25 ++++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/sql/privileges.sql | 10 +-
7 files changed, 254 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247..6f8fea37de 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,79 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_allocations</structname> view shows NUMA nodes
+ assigned allocations made from the server's main shared memory segment.
+ This includes both memory allocated by <productname>PostgreSQL</productname>
+ itself and memory allocated by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of NUMA node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf..cc014a62dc 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39..7d83a14390 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port/pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -568,3 +569,127 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count;
+ static bool firstUseInBackend = true;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ {
+ /* FIXME: should we release LWlock here ? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= pg_numa_get_max_node(); i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cd9422d0ba..59b40dc8f4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8463,6 +8463,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index a76256405f..02997690e1 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
@@ -3171,6 +3183,15 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
t
(1 row)
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
+RESET client_min_messages;
RESET ROLE;
-- clean up
DROP ROLE regress_readallstats;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b..b63c6e0f74 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index d195aaf137..e969cc3854 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,16 +1921,22 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+RESET client_min_messages;
RESET ROLE;
-- clean up
--
2.39.5
[application/octet-stream] v8-0001-Add-optional-dependency-to-libnuma-Linux-only-for.patch (15.8K, ../../CAKZiRmy-4ZuuZ+mYwcC+3-n46zy90vqRxZcXShLzwsopx5j50A@mail.gmail.com/4-v8-0001-Add-optional-dependency-to-libnuma-Linux-only-for.patch)
download | inline diff:
From cc5272cb4acd864482fa0f5ca051e4e1d659c32e Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v8 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper.
Other platforms can be added later.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway as i386 is is very memory-only
limited even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 7 +-
configure | 87 +++++++++++++++++++
configure.ac | 13 +++
doc/src/sgml/installation.sgml | 20 +++++
meson.build | 17 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 43 ++++++++++
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 150 +++++++++++++++++++++++++++++++++
14 files changed, 351 insertions(+), 1 deletion(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 91b51142d2..584e3e5a44 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -428,6 +428,8 @@ task:
DEBIAN_FRONTEND=noninteractive apt-get -y install \
libcurl4-openssl-dev \
libcurl4-openssl-dev:i386 \
+ libnuma1 \
+ libnuma-dev
matrix:
- name: Linux - Debian Bookworm - Autoconf
@@ -448,6 +450,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -492,6 +495,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -804,7 +808,8 @@ task:
setup_additional_packages_script: |
apt-get update
- DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev \
+ libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd6998..23c33dd997 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc..1a394dfc07 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b..79203e45a8 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on Linux.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,16 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on Linux. The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 13c13748e5..f81092eb66 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('numa', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3839,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c451714..adaadb5faf 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5a..0bd4b2d7d3 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5a..bff9f077a8 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d..8894f80060 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 0000000000..d3ebe8b5bd
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d5023..f786c19160 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c22431951..a68a29d541 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d4..7ffbd4d88d 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 0000000000..db28578bca
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+#include <unistd.h>
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* FIXME not tested, might crash */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-07 11:33 Jakub Wartak <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-07 11:33 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Fri, Mar 7, 2025 at 11:20 AM Jakub Wartak
<[email protected]> wrote:
>
> Hi,
> On Wed, Mar 5, 2025 at 10:30 AM Jakub Wartak
> <[email protected]> wrote:
> >Hi,
>
> > > Yeah, that's why I was mentioning to use a "shared" populate_buffercache_entry()
> > > or such function: to put the "duplicated" code in it and then use this
> > > shared function in pg_buffercache_pages() and in the new numa related one.
> >
> > OK, so hastily attempted that in 7b , I had to do a larger refactor
> > there to avoid code duplication between those two. I don't know which
> > attempt is better though (7 vs 7b)..
> >
>
> I'm attaching basically the earlier stuff (v7b) as v8 with the
> following minor changes:
> - docs are included
> - changed int8 to int4 in one function definition for numa_zone_id
.. and v9 attached because cfbot partially complained about
.cirrus.tasks.yml being adjusted recently (it seems master is hot
these days).
-J.
Attachments:
[application/octet-stream] v9-0001-Add-optional-dependency-to-libnuma-Linux-only-for.patch (15.9K, ../../CAKZiRmwp5_X6n7bsezkuOAn8=sAcGpme9WwMVgmF7czo0E1UGg@mail.gmail.com/2-v9-0001-Add-optional-dependency-to-libnuma-Linux-only-for.patch)
download | inline diff:
From ade181e760e9cb36e1688fa9cfd67172fea01509 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v9 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper.
Other platforms can be added later.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway as i386 is is very memory-only
limited even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 ++-
configure | 87 +++++++++++++++++++
configure.ac | 13 +++
doc/src/sgml/installation.sgml | 20 +++++
meson.build | 17 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/Makefile | 3 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 43 ++++++++++
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 150 +++++++++++++++++++++++++++++++++
14 files changed, 353 insertions(+), 4 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 5849cbb839a..7010dff7aef 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -519,6 +522,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -835,8 +839,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b9..79203e45a83 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on Linux.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,16 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on Linux. The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 13c13748e5d..f81092eb661 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,21 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+libnuma = dependency('numa', required: libnumaopt)
+if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+endif
+if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3839,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 42d4a28e5aa..bff9f077a8c 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -54,6 +54,9 @@ ifeq ($(with_systemd),yes)
LIBS += -lsystemd
endif
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
+
override LDFLAGS := $(LDFLAGS) $(LDFLAGS_EX) $(LDFLAGS_EX_BE)
##########################################################################
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..d3ebe8b5bd8
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..db28578bcaf
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+#include <unistd.h>
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* FIXME not tested, might crash */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
[application/octet-stream] v9-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch (14.0K, ../../CAKZiRmwp5_X6n7bsezkuOAn8=sAcGpme9WwMVgmF7czo0E1UGg@mail.gmail.com/3-v9-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones-.patch)
download | inline diff:
From 74fbe3c042533409ef4ae847826d918451d3bf68 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v9 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 78 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/privileges.out | 25 ++++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/sql/privileges.sql | 10 +-
7 files changed, 254 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..6f8fea37de6 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,79 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_allocations</structname> view shows NUMA nodes
+ assigned allocations made from the server's main shared memory segment.
+ This includes both memory allocated by <productname>PostgreSQL</productname>
+ itself and memory allocated by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of NUMA node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf5..cc014a62dc2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..7d83a143900 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port/pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -568,3 +569,127 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count;
+ static bool firstUseInBackend = true;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ {
+ /* FIXME: should we release LWlock here ? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+ }
+
+ memset(zones, 0, sizeof(zones));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= pg_numa_get_max_node(); i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cede992b6e2..ac9b8003fbc 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8482,6 +8482,14 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index a76256405fe..02997690e1b 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
@@ -3171,6 +3183,15 @@ SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
t
(1 row)
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
+RESET client_min_messages;
RESET ROLE;
-- clean up
DROP ROLE regress_readallstats;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b2..b63c6e0f744 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index d195aaf1377..e969cc38545 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,16 +1921,22 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_allocations;
+-- to ignore potenital NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+RESET client_min_messages;
RESET ROLE;
-- clean up
--
2.39.5
[application/octet-stream] v9-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch (29.5K, ../../CAKZiRmwp5_X6n7bsezkuOAn8=sAcGpme9WwMVgmF7czo0E1UGg@mail.gmail.com/4-v9-0002-Extend-pg_buffercache-with-new-view-pg_buffercach.patch)
download | inline diff:
From 3bc82e2aa255c4669ccf81bfee546a131cacedc3 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v9 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache.out | 26 +
contrib/pg_buffercache/meson.build | 1 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 470 +++++++++++++-----
contrib/pg_buffercache/sql/pg_buffercache.sql | 16 +
doc/src/sgml/pgbuffercache.sgml | 64 ++-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/storage/pg_shmem.h | 1 +
10 files changed, 493 insertions(+), 134 deletions(-)
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache.out b/contrib/pg_buffercache/expected/pg_buffercache.out
index b745dc69eae..4da569c20ae 100644
--- a/contrib/pg_buffercache/expected/pg_buffercache.out
+++ b/contrib/pg_buffercache/expected/pg_buffercache.out
@@ -8,6 +8,18 @@ from pg_buffercache;
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -34,6 +46,11 @@ SELECT * FROM pg_buffercache_summary();
ERROR: permission denied for function pg_buffercache_summary
SELECT * FROM pg_buffercache_usage_counts();
ERROR: permission denied for function pg_buffercache_usage_counts
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
SET ROLE pg_monitor;
@@ -55,3 +72,12 @@ SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
t
(1 row)
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET client_min_messages;
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..9b2e9393410 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..52f63aa258c
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..96b14f9ed49 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,14 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "storage/pg_shmem.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +45,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -61,84 +64,250 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine to map Buffers into addresses that can be
+ * later consumed by pg_numa_query_pages()
+ *
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int i, float pages_per_blk, Size os_page_size,
+ void **os_page_ptrs, bool firstUseInBackend)
+{
+ int j = 0,
+ blk2page = (int) i * pages_per_blk;
+
+ do
+ {
+ if (os_page_ptrs[blk2page + j] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers count start really from 1 */
+ os_page_ptrs[blk2page + j] = (char *) BufferGetBlock(i + 1) + (os_page_size * j);
+
+ /* We just need to do it only once in backend */
+ if (firstUseInBackend == true)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2page + j]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ j++;
+ } while (j < (int) pages_per_blk);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages.
+ *
+ * We need fcinfo here and we pass it here with PG_FUNCTION_ARGS
+ */
+static BufferCachePagesContext *
+init_buffercache_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static void
+populate_buffercache_entry(int i, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(i);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[i].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[i].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[i].blocknum = bufHdr->tag.blockNum;
+ fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[i].isdirty = true;
+ else
+ fctx->record[i].isdirty = false;
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[i].isvalid = true;
+ else
+ fctx->record[i].isvalid = false;
+
+ fctx->record[i].numa_zone_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static Datum
+get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[i].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[i].blocknum == InvalidBlockNumber ||
+ fctx->record[i].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[i].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[i].usagecount);
+ nulls[7] = false;
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
+ }
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
+
+/*
+ * When updating this routine please sync it with below one:
+ * pg_buffercache_numa_pages()
+ */
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = init_buffercache_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +318,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ populate_buffercache_entry(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +328,129 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+ bool query_numa = true;
+ static bool firstUseInBackend = true;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ int os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
+ }
+ fctx = init_buffercache_entries(funcctx, fcinfo);
+
+ if (query_numa)
+ {
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstUseInBackend == true)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+ }
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ populate_buffercache_entry(i, fctx);
+ if (query_numa)
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
}
- else
+
+ if (query_numa)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user. Also we could somehow report single DB block spanning
+ * more than one NUMA zone, but it should be rare.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
+ }
}
+ }
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ funcctx = SRF_PERCALL_SETUP();
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstUseInBackend = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache.sql b/contrib/pg_buffercache/sql/pg_buffercache.sql
index 944fbb1beae..d982048425f 100644
--- a/contrib/pg_buffercache/sql/pg_buffercache.sql
+++ b/contrib/pg_buffercache/sql/pg_buffercache.sql
@@ -5,6 +5,14 @@ select count(*) = (select setting::bigint
where name = 'shared_buffers')
from pg_buffercache;
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+RESET client_min_messages;
+
select buffers_used + buffers_unused > 0,
buffers_dirty <= buffers_used,
buffers_pinned <= buffers_used
@@ -19,6 +27,10 @@ SELECT * FROM pg_buffercache;
SELECT * FROM pg_buffercache_pages() AS p (wrong int);
SELECT * FROM pg_buffercache_summary();
SELECT * FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT * FROM pg_buffercache_numa;
+RESET client_min_messages;
RESET role;
-- Check that pg_monitor is allowed to query view / function
@@ -26,3 +38,7 @@ SET ROLE pg_monitor;
SELECT count(*) > 0 FROM pg_buffercache;
SELECT buffers_used + buffers_unused > 0 FROM pg_buffercache_summary();
SELECT count(*) > 0 FROM pg_buffercache_usage_counts();
+-- to ignore potential NOTICE: libnuma initialization failed..
+SET client_min_messages TO warning ;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET client_min_messages;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..75978a6eaed 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,13 @@
convenient use.
</para>
+ <para>
+ The similiar <function>pg_buffercache_numa_pages()</function> is a slower
+ variant of the above, but also can provide NUMA node ID for shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +209,59 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are almost identical to the previous
+ <structname>pg_buffercache</structname> view, but this one includes one additional
+ column numa_zone_id as defined in <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ ID (number) of the NUMA node for this particular buffer. NULL if the shared buffer
+ has not been used yet.On systems without NUMA this usually returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ This is clone version of the original pg_buffercache view, however it provides
+ additional <structfield>numa_zone_id</structfield> column. Fetching this
+ information from OS is costly and might take much longer and querying it is not
+ recommended by automated or monitoring systems.
+ </para>
+
+ <para>
+ As NUMA node ID inquiry for each page requires memory pages to be paged-in, first
+ execution of this function can take long time especially on systems with bigint
+ shared_buffers and without huge_pages enabled.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ad25cbb39c5..dd34c79f521 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-10 10:14 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-10 10:14 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Fri, Mar 07, 2025 at 12:33:27PM +0100, Jakub Wartak wrote:
> On Fri, Mar 7, 2025 at 11:20 AM Jakub Wartak
> <[email protected]> wrote:
> >
> > Hi,
> > On Wed, Mar 5, 2025 at 10:30 AM Jakub Wartak
> > <[email protected]> wrote:
> > >Hi,
> >
> > > > Yeah, that's why I was mentioning to use a "shared" populate_buffercache_entry()
> > > > or such function: to put the "duplicated" code in it and then use this
> > > > shared function in pg_buffercache_pages() and in the new numa related one.
> > >
> > > OK, so hastily attempted that in 7b , I had to do a larger refactor
> > > there to avoid code duplication between those two. I don't know which
> > > attempt is better though (7 vs 7b)..
> > >
> >
> > I'm attaching basically the earlier stuff (v7b) as v8 with the
> > following minor changes:
> > - docs are included
> > - changed int8 to int4 in one function definition for numa_zone_id
>
> .. and v9 attached because cfbot partially complained about
> .cirrus.tasks.yml being adjusted recently (it seems master is hot
> these days).
>
Thanks for the new version!
Some random comments on 0001:
=== 1
It does not compiles "alone". It's missing:
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
and
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
That come with 0002. So it has to be in 0001 instead.
=== 2
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
Maybe we should add the same test (checking for numa.h) for meson?
=== 3
+# FIXME: filter-out / with/without with_libnuma?
+LIBS += $(LIBNUMA_LIBS)
It looks to me that we can remove those 2 lines.
=== 4
+ Only supported on Linux.
s/on Linux/on platforms for which the libnuma library is implemented/?
I did a quick grep on "Only supported on" and it looks like that could be
a more consistent wording.
=== 5
+#include "c.h"
+#include "postgres.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+#include <unistd.h>
I had a closer look to other header files and it looks like it "should" be:
#include "c.h"
#include "postgres.h"
#include <unistd.h>
#ifdef WIN32
#include <windows.h>
#endif
#include "port/pg_numa.h"
#include "storage/pg_shmem.h"
And is "#include "c.h"" really needed?
=== 6
+/* FIXME not tested, might crash */
That's a bit scary.
=== 7
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here
s/here/here./
Did not look carefully at all the comments in 0001, 0002 and 0003 though.
A few random comments regarding 0002:
=== 8
# create extension pg_buffercache;
ERROR: could not load library "/home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so": /home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so: undefined symbol: pg_numa_query_pages
CONTEXT: SQL statement "CREATE FUNCTION pg_buffercache_pages()
RETURNS SETOF RECORD
AS '$libdir/pg_buffercache', 'pg_buffercache_pages'
LANGUAGE C PARALLEL SAFE"
extension script file "pg_buffercache--1.2.sql", near line 7
While that's ok if 0003 is applied. I think that each individual patch should
"fully" work individually.
=== 9
+ do
+ {
+ if (os_page_ptrs[blk2page + j] == 0)
blk2page + j will be repeated multiple times, better to store it in a local
variable instead?
=== 10
+ if (firstUseInBackend == true)
if (firstUseInBackend) instead?
=== 11
+ int j = 0,
+ blk2page = (int) i * pages_per_blk;
I wonder if size_t is more appropriate for blk2page:
size_t blk2page = (size_t)(i * pages_per_blk) maybe?
=== 12
as we know that we'll iterate until pages_per_blk then would a for loop be more
appropriate here, something like?
"
for (size_t j = 0; j < pages_per_blk; j++)
{
if (os_page_ptrs[blk2page + j] == 0)
{
"
=== 13
+ if (buf_state & BM_DIRTY)
+ fctx->record[i].isdirty = true;
+ else
+ fctx->record[i].isdirty = false;
fctx->record[i].isdirty = (buf_state & BM_DIRTY) != 0 maybe?
=== 14
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[i].isvalid = true;
+ else
+ fctx->record[i].isvalid = false;
fctx->record[i].isvalid = ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
maybe?
=== 15
+populate_buffercache_entry(int i, BufferCachePagesContext *fctx)
I wonder if "i" should get a more descriptive name?
=== 16
s/populate_buffercache_entry/pg_buffercache_build_tuple/? (to be consistent
with pg_stat_io_build_tuples for example).
I now realize that I did propose populate_buffercache_entry() up-thread,
sorry for changing my mind.
=== 17
+ static bool firstUseInBackend = true;
maybe we should give it a more descriptive name?
Also I need to think more about how firstUseInBackend is used, for example:
==== 17.1
would it be better to define it as a file scope variable? (at least that
would avoid to pass it as an extra parameter in some functions).
=== 17.2
what happens if firstUseInBackend is set to false and later on the pages
are moved to different NUMA nodes. Then pg_buffercache_numa_pages() is
called again by a backend that already had set firstUseInBackend to false,
would that provide accurate information?
=== 18
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+ bool query_numa = true;
I don't think we need query_numa anymore in pg_buffercache_numa_pages().
I think that we can just ERROR (or NOTICE and return) here:
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ query_numa = false;
and fully get rid of query_numa.
=== 19
And then here:
for (i = 0; i < NBuffers; i++)
{
populate_buffercache_entry(i, fctx);
if (query_numa)
pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
}
if (query_numa)
{
if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
elog(ERROR, "failed NUMA pages inquiry: %m");
for (i = 0; i < NBuffers; i++)
{
int blk2page = (int) i * pages_per_blk;
/*
* Technically we can get errors too here and pass that to
* user. Also we could somehow report single DB block spanning
* more than one NUMA zone, but it should be rare.
*/
fctx->record[i].numa_zone_id = os_pages_status[blk2page];
}
maybe we can just loop a single time over "for (i = 0; i < NBuffers; i++)"?
A few random comments regarding 0003:
=== 20
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_allocations</structname> view shows NUMA nodes
s/pg_shmem_allocations/pg_shmem_numa_allocations/?
=== 21
+ /* FIXME: should we release LWlock here ? */
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
There is no need, see src/backend/storage/lmgr/README.
=== 22
+#define MAX_NUMA_ZONES 32 /* FIXME? */
+ Size zones[MAX_NUMA_ZONES];
could we rely on pg_numa_get_max_node() instead?
=== 23
+ if (s >= 0)
+ zones[s]++;
should we also check that s is below a limit?
=== 24
Regarding how we make use of pg_numa_get_max_node(), are we sure there is
no possible holes? I mean could a system have node 0,1 and 3 but not 2?
Also I don't think I'm a Co-author, I think I'm just a reviewer (even if I
did a little in 0001 though)
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-11 10:44 Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-11 10:44 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Feb 25, 2025 at 11:59 AM Ashutosh Bapat
<[email protected]> wrote:
>
> On Tue, Feb 11, 2025 at 5:53 PM Ashutosh Bapat
> <[email protected]> wrote:
> >
> > Hi Michael,
> >
> >
> > On Sun, Feb 9, 2025 at 1:25 PM Michael Paquier <[email protected]> wrote:
> > >
> > > On Fri, Feb 07, 2025 at 07:11:25AM +0900, Michael Paquier wrote:
> > > > Okay, thanks for the feedback. We have been relying on diff -u for
> > > > the parts of the tests touched by 0001 for some time now, so if there
> > > > are no objections I would like to apply 0001 in a couple of days.
> > >
> > > This part has been applied as 169208092f5c.
> >
> > Thanks. PFA rebased patches.
>
> PFA rebased patches.
>
> After rebasing I found another bug and reported it at [1].
This bug has been fixed. But now that it's fixed, it's easy to see
another bug related to materialized view statistics. I have reported
it at [2]. That's the fourth bug identified by this test.
>
> For the time being I have added --no-statistics to the pg_dump command
> when taking a dump for comparison.
>
I have not taken out this option because of materialized view bug.
> [1] https://www.postgresql.org/message-id/[email protected]...
[2] https://www.postgresql.org/message-id/[email protected]...
--
Best Wishes,
Ashutosh Bapat
Attachments:
[text/x-patch] 0002-Filter-COPY-statements-with-differing-colum-20250311.patch (5.7K, ../../CAExHW5uQoyOddBKLBBJpfxXqqok=BTeMvt5OpnM6gw0SroiUUw@mail.gmail.com/2-0002-Filter-COPY-statements-with-differing-colum-20250311.patch)
download | inline diff:
From a140d50245249894a49c39a908163e9bac2fe4bb Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Tue, 11 Feb 2025 16:31:10 +0530
Subject: [PATCH 2/3] Filter COPY statements with differing column order
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 10 +---
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 59 +++++++++++++++------
2 files changed, 45 insertions(+), 24 deletions(-)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index c6b99125d9e..e6d8ac9a757 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -581,9 +581,6 @@ sub test_regression_dump_restore
my $dump_file = "$tempdir/regression_dump.$format";
my $restored_db = 'regression_' . $format;
- # Even though we compare only schema from the original and the restored
- # database (See get_dump_for_comparison() for details.), we dump and
- # restore data as well to catch any errors while doing so.
$src_node->command_ok(
[
'pg_dump', "-F$format", '--no-sync',
@@ -645,13 +642,10 @@ sub get_dump_for_comparison
my $dump_adjusted = "${dumpfile}_adjusted";
- # The order of columns in COPY statements dumped from the original database
- # and that from the restored database differs. These differences are hard to
- # adjust. Hence we compare only schema dumps for now.
$node->command_ok(
[
- 'pg_dump', '-s', '--no-sync', '-d',
- $node->connstr($db), '-f', $dumpfile
+ 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ $dumpfile
],
'dump for comparison succeeded');
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
index e3e152b88fa..e00a00d1b2c 100644
--- a/src/test/perl/PostgreSQL/Test/AdjustDump.pm
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -44,22 +44,36 @@ our @EXPORT = qw(
If we take dump of the regression database left behind after running regression
tests, restore the dump, and take dump of the restored regression database, the
-outputs of both the dumps differ. Some regression tests purposefully create
-some child tables in such a way that their column orders differ from column
-orders of their respective parents. In the restored database, however, their
-column orders are same as that of their respective parents. Thus the column
+outputs of both the dumps differ in the following cases. This routine adjusts
+the given dump so that dump outputs from the original and restored database,
+respectively, match.
+
+Case 1: Some regression tests purposefully create child tables in such a way
+that the order of their inherited columns differ from column orders of their
+respective parents. In the restored database, however, the order of their
+inherited columns are same as that of their respective parents. Thus the column
orders of these child tables in the original database and those in the restored
database differ, causing difference in the dump outputs. See MergeAttributes()
-and dumpTableSchema() for details.
-
-This routine rearranges the column declarations in the relevant
-C<CREATE TABLE... INHERITS> statements in the dump file from original database
-to match those from the restored database. We could instead adjust the
-statements in the dump from the restored database to match those from original
-database or adjust both to a canonical order. But we have chosen to adjust the
-statements in the dump from original database for no particular reason.
-
-Additionally it adjusts blank and new lines to avoid noise.
+and dumpTableSchema() for details. This routine rearranges the column
+declarations in the relevant C<CREATE TABLE... INHERITS> statements in the dump
+file from original database to match those from the restored database. We could,
+instead, adjust the statements in the dump from the restored database to match
+those from original database or adjust both to a canonical order. But we have
+chosen to adjust the statements in the dump from original database for no
+particular reason.
+
+Case 2: When dumping COPY statements the columns are ordered by their attribute
+number by fmtCopyColumnList(). If a column is added to a parent table after a
+child has inherited the parent and the child has its own columns, the attribute
+number of the column changes after restoring the child table. This is because
+when executing the dumped C<CREATE TABLE... INHERITS> statement all the parent
+attributes are created before any child attributes. Thus the order of columns in
+COPY statements dumped from the original and the restored databases,
+respectively, differs. Such tables in regression tests are listed below. It is
+hard to adjust the column order in the COPY statement along with the data. Hence
+we just remove such COPY statements from the dump output.
+
+Additionally the routine adjusts blank and new lines to avoid noise.
Arguments:
@@ -84,8 +98,6 @@ sub adjust_regress_dumpfile
# use Unix newlines
$dump =~ s/\r\n/\n/g;
- # Suppress blank lines, as some places in pg_dump emit more or fewer.
- $dump =~ s/\n\n+/\n/g;
# Adjust the CREATE TABLE ... INHERITS statements.
if ($adjust_child_columns)
@@ -122,6 +134,21 @@ sub adjust_regress_dumpfile
'applied public.test_type_diff2_c2 adjustments');
}
+ # Remove COPY statements with differing column order
+ for my $table (
+ 'public\.b_star', 'public\.c_star',
+ 'public\.cc2', 'public\.d_star',
+ 'public\.e_star', 'public\.f_star',
+ 'public\.renamecolumnanother', 'public\.renamecolumnchild',
+ 'public\.test_type_diff2_c1', 'public\.test_type_diff2_c2',
+ 'public\.test_type_diff_c')
+ {
+ $dump =~ s/^COPY\s$table\s\(.+?^\\\.$//sm;
+ }
+
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
return $dump;
}
--
2.34.1
[text/x-patch] 0001-Test-pg_dump-restore-of-regression-objects-20250311.patch (14.4K, ../../CAExHW5uQoyOddBKLBBJpfxXqqok=BTeMvt5OpnM6gw0SroiUUw@mail.gmail.com/3-0001-Test-pg_dump-restore-of-regression-objects-20250311.patch)
download | inline diff:
From 25c4c7e4ee754dd989d3fd8f015c7355fd9992d6 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Thu, 27 Jun 2024 10:03:53 +0530
Subject: [PATCH 1/3] Test pg_dump/restore of regression objects
002_pg_upgrade.pl tests pg_upgrade of the regression database left
behind by regression run. Modify it to test dump and restore of the
regression database as well.
Regression database created by regression run contains almost all the
database objects supported by PostgreSQL in various states. Hence the
new testcase covers dump and restore scenarios not covered by individual
dump/restore cases. Till now 002_pg_upgrade only tested dump/restore
through pg_upgrade which only uses binary mode. Many regression tests
mention that they leave objects behind for dump/restore testing but they
are not tested in a non-binary mode. The new testcase closes that
gap.
Testing dump and restore of regression database makes this test run
longer for a relatively smaller benefit. Hence run it only when
explicitly requested by user by specifying "regress_dump_test" in
PG_TEST_EXTRA.
Note For the reviewers:
The new test has uncovered two bugs so far in one year.
1. Introduced by 14e87ffa5c54. Fixed in fd41ba93e4630921a72ed5127cd0d552a8f3f8fc.
2. Introduced by 0413a556990ba628a3de8a0b58be020fd9a14ed0. Reverted in 74563f6b90216180fc13649725179fc119dddeb5.
Author: Ashutosh Bapat
Reviewed by: Michael Pacquire, Daniel Gustafsson, Tom Lane
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
---
doc/src/sgml/regress.sgml | 12 ++
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 142 +++++++++++++++++++-
src/test/perl/Makefile | 2 +
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 134 ++++++++++++++++++
src/test/perl/meson.build | 1 +
5 files changed, 289 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/AdjustDump.pm
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 0e5e8e8f309..237b974b3ab 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,6 +357,18 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>regress_dump_test</literal></term>
+ <listitem>
+ <para>
+ When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
+ tests dump and restore of regression database left behind by the
+ regression run. Not enabled by default because it is time and resource
+ consuming.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index c00cf68d660..c6b99125d9e 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -12,6 +12,7 @@ use File::Path qw(rmtree);
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::AdjustUpgrade;
+use PostgreSQL::Test::AdjustDump;
use Test::More;
# Can be changed to test the other modes.
@@ -35,8 +36,8 @@ sub generate_db
"created database with ASCII characters from $from_char to $to_char");
}
-# Filter the contents of a dump before its use in a content comparison.
-# This returns the path to the filtered dump.
+# Filter the contents of a dump before its use in a content comparison for
+# upgrade testing. This returns the path to the filtered dump.
sub filter_dump
{
my ($is_old, $old_version, $dump_file) = @_;
@@ -261,6 +262,21 @@ else
}
}
is($rc, 0, 'regression tests pass');
+
+ # Test dump/restore of the objects left behind by regression. Ideally it
+ # should be done in a separate TAP test, but doing it here saves us one full
+ # regression run.
+ #
+ # This step takes several extra seconds and some extra disk space, so
+ # requires an opt-in with the PG_TEST_EXTRA environment variable.
+ #
+ # Do this while the old cluster is running before it is shut down by the
+ # upgrade test.
+ if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
+ {
+ test_regression_dump_restore($oldnode, %node_params);
+ }
}
# Initialize a new node for the upgrade.
@@ -527,4 +543,126 @@ my $dump2_filtered = filter_dump(0, $oldnode->pg_version, $dump2_file);
compare_files($dump1_filtered, $dump2_filtered,
'old and new dumps match after pg_upgrade');
+# Test dump and restore of objects left behind by the regression run.
+#
+# It is expected that regression tests, which create `regression` database, are
+# run on `src_node`, which in turn, is left in running state. The dump from
+# `src_node` is restored on a fresh node created using given `node_params`.
+# Plain dumps from both the nodes are compared to make sure that all the dumped
+# objects are restored faithfully.
+sub test_regression_dump_restore
+{
+ my ($src_node, %node_params) = @_;
+ my $dst_node = PostgreSQL::Test::Cluster->new('dst_node');
+
+ # Make sure that the source and destination nodes have the same version and
+ # do not use custom install paths. In both the cases, the dump files may
+ # require additional adjustments unknown to code here. Do not run this test
+ # in such a case to avoid utilizing the time and resources unnecessarily.
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default installation");
+ return;
+ }
+
+ # Dump the original database for comparison later.
+ my $src_dump =
+ get_dump_for_comparison($src_node, 'regression', 'src_dump', 1);
+
+ # Setup destination database cluster
+ $dst_node->init(%node_params);
+ # Stabilize stats for comparison.
+ $dst_node->append_conf('postgresql.conf', 'autovacuum = off');
+ $dst_node->start;
+
+ for my $format ('plain', 'tar', 'directory', 'custom')
+ {
+ my $dump_file = "$tempdir/regression_dump.$format";
+ my $restored_db = 'regression_' . $format;
+
+ # Even though we compare only schema from the original and the restored
+ # database (See get_dump_for_comparison() for details.), we dump and
+ # restore data as well to catch any errors while doing so.
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ # Create a new database for restoring dump from every format so that it
+ # is available for debugging in case the test fails.
+ $dst_node->command_ok([ 'createdb', $restored_db ],
+ "created destination database '$restored_db'");
+
+ # Restore into destination database.
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [
+ 'psql', '-d', $dst_node->connstr($restored_db),
+ '-f', $dump_file
+ ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '-d',
+ $dst_node->connstr($restored_db), $dump_file
+ ];
+ }
+ $dst_node->command_ok(@restore_command,
+ "restored dump taken in $format format on destination instance");
+
+ my $dst_dump =
+ get_dump_for_comparison($dst_node, $restored_db,
+ 'dest_dump.' . $format, 0);
+
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database (using $format format) match"
+ );
+ }
+}
+
+# Dump database `db` from the given `node` in plain format and adjust it for
+# comparing dumps from the original and the restored database.
+#
+# `file_prefix` is used to create unique names for all dump files so that they
+# remain available for debugging in case the test fails.
+#
+# `adjust_child_columns` is passed to adjust_regress_dumpfile() which actually
+# adjusts the dump output.
+#
+# The name of the file containting adjusted dump is returned.
+sub get_dump_for_comparison
+{
+ my ($node, $db, $file_prefix, $adjust_child_columns) = @_;
+
+ my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
+ my $dump_adjusted = "${dumpfile}_adjusted";
+
+
+ # The order of columns in COPY statements dumped from the original database
+ # and that from the restored database differs. These differences are hard to
+ # adjust. Hence we compare only schema dumps for now.
+ $node->command_ok(
+ [
+ 'pg_dump', '-s', '--no-sync', '-d',
+ $node->connstr($db), '-f', $dumpfile
+ ],
+ 'dump for comparison succeeded');
+
+ open(my $dh, '>', $dump_adjusted)
+ || die
+ "could not open $dump_adjusted for writing the adjusted dump: $!";
+ print $dh adjust_regress_dumpfile(slurp_file($dumpfile),
+ $adjust_child_columns);
+ close($dh);
+
+ return $dump_adjusted;
+}
+
done_testing();
diff --git a/src/test/perl/Makefile b/src/test/perl/Makefile
index d82fb67540e..def89650ead 100644
--- a/src/test/perl/Makefile
+++ b/src/test/perl/Makefile
@@ -26,6 +26,7 @@ install: all installdirs
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/Cluster.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/BackgroundPsql.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustUpgrade.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustDump.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Version.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
uninstall:
@@ -36,6 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
endif
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..e3e152b88fa
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,134 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::AdjustDump - helper module for dump and restore tests
+
+=head1 SYNOPSIS
+
+ use PostgreSQL::Test::AdjustDump;
+
+ # Adjust contents of dump output file so that dump output from original
+ # regression database and that from the restored regression database match
+ $dump = adjust_regress_dumpfile($dump, $adjust_child_columns);
+
+=head1 DESCRIPTION
+
+C<PostgreSQL::Test::AdjustDump> encapsulates various hacks needed to
+compare the results of dump and restore tests
+
+=cut
+
+package PostgreSQL::Test::AdjustDump;
+
+use strict;
+use warnings FATAL => 'all';
+
+use Exporter 'import';
+use Test::More;
+
+our @EXPORT = qw(
+ adjust_regress_dumpfile
+);
+
+=pod
+
+=head1 ROUTINES
+
+=over
+
+=item $dump = adjust_regress_dumpfile($dump, $adjust_child_columns)
+
+If we take dump of the regression database left behind after running regression
+tests, restore the dump, and take dump of the restored regression database, the
+outputs of both the dumps differ. Some regression tests purposefully create
+some child tables in such a way that their column orders differ from column
+orders of their respective parents. In the restored database, however, their
+column orders are same as that of their respective parents. Thus the column
+orders of these child tables in the original database and those in the restored
+database differ, causing difference in the dump outputs. See MergeAttributes()
+and dumpTableSchema() for details.
+
+This routine rearranges the column declarations in the relevant
+C<CREATE TABLE... INHERITS> statements in the dump file from original database
+to match those from the restored database. We could instead adjust the
+statements in the dump from the restored database to match those from original
+database or adjust both to a canonical order. But we have chosen to adjust the
+statements in the dump from original database for no particular reason.
+
+Additionally it adjusts blank and new lines to avoid noise.
+
+Arguments:
+
+=over
+
+=item C<dump>: Contents of dump file
+
+=item C<adjust_child_columns>: 1 indicates that the given dump file requires
+adjusting columns in the child tables; usually when the dump is from original
+database. 0 indicates no such adjustment is needed; usually when the dump is
+from restored database.
+
+=back
+
+Returns the adjusted dump text.
+
+=cut
+
+sub adjust_regress_dumpfile
+{
+ my ($dump, $adjust_child_columns) = @_;
+
+ # use Unix newlines
+ $dump =~ s/\r\n/\n/g;
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
+ # Adjust the CREATE TABLE ... INHERITS statements.
+ if ($adjust_child_columns)
+ {
+ my $saved_dump = $dump;
+
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_stored_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_stored_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_virtual_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_virtual_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c1\s\()
+ (\n\s+int_four\sbigint),
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint)/$1$4,$2,$3/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c1 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c2\s\()
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint),
+ (\n\s+int_four\sbigint)/$1$3,$4,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c2 adjustments');
+ }
+
+ return $dump;
+}
+
+=pod
+
+=back
+
+=cut
+
+1;
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
index 58e30f15f9d..492ca571ff8 100644
--- a/src/test/perl/meson.build
+++ b/src/test/perl/meson.build
@@ -14,4 +14,5 @@ install_data(
'PostgreSQL/Test/Cluster.pm',
'PostgreSQL/Test/BackgroundPsql.pm',
'PostgreSQL/Test/AdjustUpgrade.pm',
+ 'PostgreSQL/Test/AdjustDump.pm',
install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
base-commit: dabccf45139a8c7c3c2e7683a943c31077e55a78
--
2.34.1
[text/x-patch] 0003-Do-not-dump-statistics-in-the-file-dumped-f-20250311.patch (1.2K, ../../CAExHW5uQoyOddBKLBBJpfxXqqok=BTeMvt5OpnM6gw0SroiUUw@mail.gmail.com/4-0003-Do-not-dump-statistics-in-the-file-dumped-f-20250311.patch)
download | inline diff:
From 4258ed1bcad537418c4c3f4ba0e3712ec515e09e Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Tue, 25 Feb 2025 11:42:51 +0530
Subject: [PATCH 3/3] Do not dump statistics in the file dumped for comparison
As reported at [1], the dumped and restored statistics may differ if there's a
primary key on the table. Hence do not dump the statistics to avoid differences
in the dump output from the original and restored database.
[1] https://www.postgresql.org/message-id/CAExHW5vf9D+8-a5_BEX3y=2y_xY9hiCxV1=C+FnxDvfprWvkng@mail.gmail.com
Ashutosh Bapat
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index e6d8ac9a757..8924cf8344a 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -644,7 +644,7 @@ sub get_dump_for_comparison
$node->command_ok(
[
- 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ 'pg_dump', '--no-sync', '--no-statistics', '-d', $node->connstr($db), '-f',
$dumpfile
],
'dump for comparison succeeded');
--
2.34.1
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-12 12:05 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-12 12:05 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Hello
When running these tests, I encounter this strange diff in the dumps,
which seems to be that the locale for type money does not match. I
imagine the problem is that the locale is not set correctly when
initdb'ing one of them? Grepping the regress_log for initdb, I see
this:
$ grep -B1 'Running: initdb' tmp_check/log/regress_log_002_pg_upgrade
[13:00:57.580](0.003s) # initializing database system by running initdb
# Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_old_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale C.UTF-8 -k
--
[13:01:12.879](0.044s) # initializing database system by running initdb
# Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_dst_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale C.UTF-8 -k
--
[13:01:28.000](0.033s) # initializing database system by running initdb
# Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_new_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding SQL_ASCII --locale-provider libc
[12:50:31.838](0.102s) not ok 15 - dump outputs from original and restored regression database (using plain format) match
[12:50:31.839](0.000s)
[12:50:31.839](0.000s) # Failed test 'dump outputs from original and restored regression database (using plain format) match'
# at /pgsql/source/master/src/test/perl/PostgreSQL/Test/Utils.pm line 797.
[12:50:31.839](0.000s) # got: '1'
# expected: '0'
=== diff of /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/src_dump.sql_adjusted and /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/dest_dump.plain.sql_adjusted
=== stdout ===
--- /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/src_dump.sql_adjusted 2025-03-12 12:50:27.674918597 +0100
+++ /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/dest_dump.plain.sql_adjusted 2025-03-12 12:50:31.778840338 +0100
@@ -208972,7 +208972,7 @@
-- Data for Name: money_data; Type: TABLE DATA; Schema: public; Owner: alvherre
--
COPY public.money_data (m) FROM stdin;
-$123.46
+$ 12.346,00
\.
--
-- Data for Name: mvtest_t; Type: TABLE DATA; Schema: public; Owner: alvherre
@@ -376231,7 +376231,7 @@
-- Data for Name: tab_core_types; Type: TABLE DATA; Schema: public; Owner: alvherre
--
COPY public.tab_core_types (point, line, lseg, box, openedpath, closedpath, polygon, circle, date, "time", "timestamp", timetz, timestamptz, "interval", "json", jsonb, jsonpath, inet, cidr, macaddr8, macaddr, int2, int4, int8, float4, float8, pi, "char", bpchar, "varchar", name, text, bool, bytea, "bit", varbit, money, refcursor, int2vector, oidvector, aclitem, tsvector, tsquery, uuid, xid8, regclass, type, regrole, oid, tid, xid, cid, txid_snapshot, pg_snapshot, pg_lsn, cardinal_number, character_data, sql_identifier, time_stamp, yes_or_no, int4range, int4multirange, int8range, int8multirange, numrange, nummultirange, daterange, datemultirange, tsrange, tsmultirange, tstzrange, tstzmultirange) FROM stdin;
-(11,12) {1,-1,0} [(11,11),(12,12)] (13,13),(11,11) ((11,12),(13,13),(14,14)) [(11,12),(13,13),(14,14)] ((11,12),(13,13),(14,14)) <(1,1),1> 2025-03-12 04:50:14.125899 2025-03-12 04:50:14.125899 04:50:14.125899-07 2025-03-12 12:50:14.125899+01 00:00:12 {"reason":"because"} {"when": "now"} $."a"[*]?(@ > 2) 127.0.0.1 127.0.0.0/8 00:01:03:ff:fe:86:1c:ba 00:01:03:86:1c:ba 2 4 8 4 8 3.14159265358979 f c abc name txt t \\xdeadbeef 1 10001 $12.34 abc 1 2 1 2 alvherre=UC/alvherre 'a' 'and' 'ate' 'cat' 'fat' 'mat' 'on' 'rat' 'sat' 'fat' & 'rat' a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11 11 pg_class regtype pg_monitor 1259 (1,1) 2 3 10:20:10,14,15 10:20:10,14,15 16/B374D848 1 l n 2025-03-12 12:50:14.13+01 YES empty {} empty {} (3,4) {(3,4)} [2020-01-03,2021-02-03) {[2020-01-03,2021-02-03)} ("2020-01-02 03:04:05","2021-02-03 06:07:08") {("2020-01-02 03:04:05","2021-02-03 06:07:08")} ("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01") {("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01")}
+(11,12) {1,-1,0} [(11,11),(12,12)] (13,13),(11,11) ((11,12),(13,13),(14,14)) [(11,12),(13,13),(14,14)] ((11,12),(13,13),(14,14)) <(1,1),1> 2025-03-12 04:50:14.125899 2025-03-12 04:50:14.125899 04:50:14.125899-07 2025-03-12 12:50:14.125899+01 00:00:12 {"reason":"because"} {"when": "now"} $."a"[*]?(@ > 2) 127.0.0.1 127.0.0.0/8 00:01:03:ff:fe:86:1c:ba 00:01:03:86:1c:ba 2 4 8 4 8 3.14159265358979 f c abc name txt t \\xdeadbeef 1 10001 $ 1.234,00 abc 1 2 1 2 alvherre=UC/alvherre 'a' 'and' 'ate' 'cat' 'fat' 'mat' 'on' 'rat' 'sat' 'fat' & 'rat' a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11 11 pg_class regtype pg_monitor 1259 (1,1) 2 3 10:20:10,14,15 10:20:10,14,15 16/B374D848 1 l n 2025-03-12 12:50:14.13+01 YES empty {} empty {} (3,4) {(3,4)} [2020-01-03,2021-02-03) {[2020-01-03,2021-02-03)} ("2020-01-02 03:04:05","2021-02-03 06:07:08") {("2020-01-02 03:04:05","2021-02-03 06:07:08")} ("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01") {("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01")}
\.
--
-- Data for Name: tableam_parted_a_heap2; Type: TABLE DATA; Schema: public; Owner: alvherre=== stderr ===
=== EOF ===
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"¿Qué importan los años? Lo que realmente importa es comprobar que
a fin de cuentas la mejor edad de la vida es estar vivo" (Mafalda)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-12 15:41 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: Jakub Wartak @ 2025-03-12 15:41 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Mon, Mar 10, 2025 at 11:14 AM Bertrand Drouvot
<[email protected]> wrote:
> Thanks for the new version!
v10 is attached with most fixes after review and one new thing
introduced: pg_numa_available() for run-time decision inside tests
which was needed after simplifying code a little bit as you wanted.
I've also fixed -Dlibnuma=disabled as it was not properly implemented.
There are couple open items (minor/decision things), but most is fixed
or answered:
> Some random comments on 0001:
>
> === 1
>
> It does not compiles "alone". It's missing:
>
[..]
> +extern PGDLLIMPORT int huge_pages_status;
[..]
> -static int huge_pages_status = HUGE_PAGES_UNKNOWN;
> +int huge_pages_status = HUGE_PAGES_UNKNOWN;
>
> That come with 0002. So it has to be in 0001 instead.
Ugh, good catch, I haven't thought about it in isolation, they are
separate to just ease review, but should be committed together. Fixed.
> === 2
>
> +else
> + as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
> +fi
>
> Maybe we should add the same test (checking for numa.h) for meson?
TBH, I have no idea, libnuma.h may exist but it may not link e.g. when
cross-compiling 32-bit on 64-bit. Or is this more about keeping sync
between meson and autoconf?
> === 3
>
> +# FIXME: filter-out / with/without with_libnuma?
> +LIBS += $(LIBNUMA_LIBS)
>
> It looks to me that we can remove those 2 lines.
Done.
> === 4
>
> + Only supported on Linux.
>
> s/on Linux/on platforms for which the libnuma library is implemented/?
>
> I did a quick grep on "Only supported on" and it looks like that could be
> a more consistent wording.
Fixed.
> === 5
>
> +#include "c.h"
> +#include "postgres.h"
> +#include "port/pg_numa.h"
> +#include "storage/pg_shmem.h"
> +#include <unistd.h>
>
> I had a closer look to other header files and it looks like it "should" be:
>
> #include "c.h"
> #include "postgres.h"
>
> #include <unistd.h>
> #ifdef WIN32
> #include <windows.h>
> #endif
>
> #include "port/pg_numa.h"
> #include "storage/pg_shmem.h"
>
> And is "#include "c.h"" really needed?
Fixed both. It seems to compile without c.h.
> === 6
>
> +/* FIXME not tested, might crash */
>
> That's a bit scary.
When you are in support for long enough it is becoming the norm ;) But
on serious note Andres wanted have numa error/warning handlers (used
by libnuma), but current code has no realistic code-path to hit it
from numa_available(3), numa_move_pages(3) or numa_max_node(3). The
situation won't change until one day in future (I hope!) we start
using some more advanced libnuma functionality for interleaving
memory, please see my earlier reply:
https://www.postgresql.org/message-id/CAKZiRmzpvBtqrz5Jr2DDcfk4Ar1ppsXkUhEX9RpA%2Bs%2B_5hcTOg%40mail...
E.g. numa_available(3) is tiny wrapper , see
https://github.com/numactl/numactl/blob/master/libnuma.c#L872
For now, I've adjusted that FIXME to XXX, but still don't know we
could inject failure to see this triggered...
> === 7
>
> + * XXX: for now we issue just WARNING, but long-term that might depend on
> + * numa_set_strict() here
>
> s/here/here./
Done.
> Did not look carefully at all the comments in 0001, 0002 and 0003 though.
>
> A few random comments regarding 0002:
>
> === 8
>
> # create extension pg_buffercache;
> ERROR: could not load library "/home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so": /home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so: undefined symbol: pg_numa_query_pages
> CONTEXT: SQL statement "CREATE FUNCTION pg_buffercache_pages()
> RETURNS SETOF RECORD
> AS '$libdir/pg_buffercache', 'pg_buffercache_pages'
> LANGUAGE C PARALLEL SAFE"
> extension script file "pg_buffercache--1.2.sql", near line 7
>
> While that's ok if 0003 is applied. I think that each individual patch should
> "fully" work individually.
STILL OPEN QUESTION: Not sure I understand: you need to have 0001 +
0002 or 0001 + 0003, but here 0002 is complaining about lack of
pg_numa_query_pages() which is part of 0001 (?) Should I merge those
patches or keep them separate?
> === 9
>
> + do
> + {
> + if (os_page_ptrs[blk2page + j] == 0)
>
> blk2page + j will be repeated multiple times, better to store it in a local
> variable instead?
Done.
> === 10
>
> + if (firstUseInBackend == true)
>
>
> if (firstUseInBackend) instead?
Done everywhere.
> === 11
>
> + int j = 0,
> + blk2page = (int) i * pages_per_blk;
>
> I wonder if size_t is more appropriate for blk2page:
>
> size_t blk2page = (size_t)(i * pages_per_blk) maybe?
Sure, done.
> === 12
>
> as we know that we'll iterate until pages_per_blk then would a for loop be more
> appropriate here, something like?
>
> "
> for (size_t j = 0; j < pages_per_blk; j++)
> {
> if (os_page_ptrs[blk2page + j] == 0)
> {
> "
Sure.
> === 13
>
> + if (buf_state & BM_DIRTY)
> + fctx->record[i].isdirty = true;
> + else
> + fctx->record[i].isdirty = false;
>
> fctx->record[i].isdirty = (buf_state & BM_DIRTY) != 0 maybe?
>
> === 14
>
> + if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
> + fctx->record[i].isvalid = true;
> + else
> + fctx->record[i].isvalid = false;
>
> fctx->record[i].isvalid = ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
> maybe?
It is coming from the original pg_buffercache and it is less readable
to me, so I don't want to touch that too much because that might open
refactoring doors too wide.
> === 15
>
> +populate_buffercache_entry(int i, BufferCachePagesContext *fctx)
>
> I wonder if "i" should get a more descriptive name?
Done, buffer_id.
> === 16
>
> s/populate_buffercache_entry/pg_buffercache_build_tuple/? (to be consistent
> with pg_stat_io_build_tuples for example).
>
> I now realize that I did propose populate_buffercache_entry() up-thread,
> sorry for changing my mind.
OK, I've tried to create a better name, but all my ideas were too
long, but pg_buffercache_build_tuple sounds nice, so lets use that.
> === 17
>
> + static bool firstUseInBackend = true;
>
> maybe we should give it a more descriptive name?
I couldn't come up with anything that wouldn't look too long, so
instead I've added a comment explaining the meaning behind this
variable, hope that's good enough.
> Also I need to think more about how firstUseInBackend is used, for example:
>
> ==== 17.1
>
> would it be better to define it as a file scope variable? (at least that
> would avoid to pass it as an extra parameter in some functions).
I have no strong opinion on this, but I have one doubt (for future):
isn't creating global variables making life harder for upcoming
multithreading guys ?
> === 17.2
>
> what happens if firstUseInBackend is set to false and later on the pages
> are moved to different NUMA nodes. Then pg_buffercache_numa_pages() is
> called again by a backend that already had set firstUseInBackend to false,
> would that provide accurate information?
It is still the correct result. That "touching" (paging-in) is only
necessary probably to properly resolve PTEs as the fork() does not
seem to carry them over from parent:
postgres=# select 'create table foo' || s || ' as select
generate_series(1, 100000);' from generate_series(1, 4) s;
postgres=# \gexec
SELECT 100000
SELECT 100000
SELECT 100000
SELECT 100000
postgres=# select numa_zone_id, count(*) from pg_buffercache_numa
group by numa_zone_id order by numa_zone_id; -- before:
numa_zone_id | count
--------------+---------
0 | 256
1 | 4131
| 8384221
postgres=# select pg_backend_pid();
pg_backend_pid
----------------
1451
-- now use another root (!) session to "migratepages(8)" from numactl
will also shift shm segment:
# migratepages 1451 1 3
-- while above will be in progress for a lot of time , but the outcome
is visible much faster in that backend (pg is functioning):
postgres=# select numa_zone_id, count(*) from pg_buffercache_numa
group by numa_zone_id order by numa_zone_id;
numa_zone_id | count
--------------+---------
0 | 256
3 | 4480
| 8383872
So the above clearly shows that initial touching of shm is required,
but just once and it stays valid afterwards.
BTW: migratepages(8) was stuck for 1-2 minutes there on
"__wait_rcu_gp" according to it's wchan, without any sign of activity
on the OS and then out of blue completed just fine, s_b=64GB,HP=on.
> === 18
>
> +Datum
> +pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
> +{
> + FuncCallContext *funcctx;
> + BufferCachePagesContext *fctx; /* User function context. */
> + bool query_numa = true;
>
> I don't think we need query_numa anymore in pg_buffercache_numa_pages().
>
> I think that we can just ERROR (or NOTICE and return) here:
>
> + if (pg_numa_init() == -1)
> + {
> + elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
> + query_numa = false;
>
> and fully get rid of query_numa.
Right... fixed it here, but it made the tests blow up, so I've had to
find a way to conditionally launch tests based on NUMA availability
and that's how pg_numa_available() was born. It's in ipc/shmem.c
because I couldn't find a better place for it...
> === 19
>
> And then here:
>
> for (i = 0; i < NBuffers; i++)
> {
> populate_buffercache_entry(i, fctx);
> if (query_numa)
> pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
> }
>
> if (query_numa)
> {
> if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
> elog(ERROR, "failed NUMA pages inquiry: %m");
>
> for (i = 0; i < NBuffers; i++)
> {
> int blk2page = (int) i * pages_per_blk;
>
> /*
> * Technically we can get errors too here and pass that to
> * user. Also we could somehow report single DB block spanning
> * more than one NUMA zone, but it should be rare.
> */
> fctx->record[i].numa_zone_id = os_pages_status[blk2page];
> }
>
> maybe we can just loop a single time over "for (i = 0; i < NBuffers; i++)"?
Well, pg_buffercache_numa_prepare_ptrs() is just an inlined wrapper
that prepares **os_page_ptrs, which is then used by
pg_numa_query_pages() and then we fill the data. But now after
removing query_numa , it reads smoother anyway. Can you please take a
look again on this, is this up to the project standards?
> A few random comments regarding 0003:
>
> === 20
>
> + The <structname>pg_shmem_allocations</structname> view shows NUMA nodes
>
> s/pg_shmem_allocations/pg_shmem_numa_allocations/?
Fixed.
> === 21
>
> + /* FIXME: should we release LWlock here ? */
> + elog(ERROR, "failed NUMA pages inquiry status: %m");
>
> There is no need, see src/backend/storage/lmgr/README.
Thanks, fixed.
> === 22
>
> +#define MAX_NUMA_ZONES 32 /* FIXME? */
> + Size zones[MAX_NUMA_ZONES];
>
> could we rely on pg_numa_get_max_node() instead?
Sure, done.
> === 23
>
> + if (s >= 0)
> + zones[s]++;
>
> should we also check that s is below a limit?
STILL OPEN QUESTION: I'm not sure it would give us value to report
e.g. -2 on per shm entry/per numa node entry, would it? If it would we
could somehow overallocate that array and remember negative ones too.
> === 24
>
> Regarding how we make use of pg_numa_get_max_node(), are we sure there is
> no possible holes? I mean could a system have node 0,1 and 3 but not 2?
I have never seen a hole in numbering as it is established during
boot, and the only way that could get close to adjusting it could be
making processor books (CPU and RAM together) offline. Even *if*
someone would be doing some preventive hardware maintenance, that
still wouldn't hurt, as we are just using the Id of the zone to
display it -- the max would be already higher. I mean, technically one
could use lsmem(1) (it mentions removable flag, after let's pages and
processes migrated away and from there) and then use chcpu(1) to
--disable CPUs on that zone (to prevent new ones coming there and
allocating new local memory there) and then offlining that memory
region via chmem(1) --disable. Even with all of that, that still
shouldn't cause issues for this code I think, because
`numa_max_node()` says it `returns the >> highest node number <<<
available on the current system.`
> Also I don't think I'm a Co-author, I think I'm just a reviewer (even if I
> did a little in 0001 though)
That was an attempt to say "Thank You", OK, I've aligned it that way,
so you are still referenced in 0001. I wouldn't find motivation to
work on this if you won't respond to those emails ;)
There is one known issue, CI returned for numa.out test, that I need
to get bottom to (it does not reproduce for me locally) inside
numa.out/sql:
SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations
WARNING: detected write past chunk end in ExprContext 0x55bb7f1a8f90
WARNING: detected write past chunk end in ExprContext 0x55bb7f1a8f90
WARNING: detected write past chunk end in ExprContext 0x55bb7f1a8f90
-J.
Attachments:
[application/octet-stream] v10-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (19.0K, ../../CAKZiRmzwO_KEXH4hkrHmx3qja0kER9pKW=3yyLvEP+b38vVLQg@mail.gmail.com/2-v10-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From f77b7e1a680a7aad370cf2ac9233b9c17bd5a4a2 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v10 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper.
Other platforms can be added later.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway as i386 is is very memory-only
limited even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 ++-
configure | 87 ++++++++++++++++
configure.ac | 13 +++
doc/src/sgml/func.sgml | 13 +++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 21 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/storage/ipc/shmem.c | 11 ++
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 5 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 43 ++++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 151 ++++++++++++++++++++++++++++
18 files changed, 387 insertions(+), 5 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 5849cbb839a..7010dff7aef 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -519,6 +522,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -835,8 +839,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 51dd8ad6571..071b98e6c9a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25061,6 +25061,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if a <acronym>NUMA</acronym> support is available.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b9..9f56205a1d7 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 13c13748e5d..19500ebdfb2 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,25 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ else
+ libnuma = not_found_dep
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3187,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3843,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..4c9c3cb320f 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port/pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -568,3 +569,13 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL level function returning whether NUMA support was compiled in. */
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ if(pg_numa_init() == -1)
+ PG_RETURN_BOOL(false);
+ PG_RETURN_BOOL(true);
+}
+
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ad25cbb39c5..dd34c79f521 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 42e427f8fe8..38612d8ae12 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8489,6 +8489,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '5102', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
@@ -12477,3 +12481,4 @@
prosrc => 'gist_stratnum_common' },
]
+
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..d3ebe8b5bd8
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..b9348caaca9
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,151 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* XXX: not tested */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here.
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
[application/octet-stream] v10-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (27.8K, ../../CAKZiRmzwO_KEXH4hkrHmx3qja0kER9pKW=3yyLvEP+b38vVLQg@mail.gmail.com/3-v10-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From df7f25e69e7e39d564e78783dab4d4fa4287dc4d Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v10 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 ++
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 462 +++++++++++++-----
.../sql/pg_buffercache_numa.sql | 21 +
doc/src/sgml/pgbuffercache.sgml | 64 ++-
9 files changed, 493 insertions(+), 134 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..52f63aa258c
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..9cc04320d63 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,12 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +43,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -61,84 +62,249 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine to map Buffers into addresses that can be
+ * later consumed by pg_numa_query_pages()
+ *
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk, Size os_page_size,
+ void **os_page_ptrs, bool firstUseInBackend)
+{
+ size_t blk2page = (size_t)(buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers count start really from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) + (os_page_size * j);
+
+ /* We just need to do it only once in backend */
+ if (firstUseInBackend)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ }
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages.
+ *
+ * We need fcinfo here and we pass it here with PG_FUNCTION_ARGS
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static void
+pg_buffercache_build_tuple(int i, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(i);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[i].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[i].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[i].blocknum = bufHdr->tag.blockNum;
+ fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[i].isdirty = true;
+ else
+ fctx->record[i].isdirty = false;
+
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[i].isvalid = true;
+ else
+ fctx->record[i].isvalid = false;
+
+ fctx->record[i].numa_zone_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static Datum
+get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[i].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[i].blocknum == InvalidBlockNumber ||
+ fctx->record[i].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[i].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[i].usagecount);
+ nulls[7] = false;
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
+ }
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
+
+/*
+ * When updating this routine please sync it with below one:
+ * pg_buffercache_numa_pages()
+ */
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +315,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_build_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +325,122 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+ /*
+ * To get reliable results we need to "touch pages" once, see
+ * comments nearby pg_buffercache_numa_prepare_ptrs().
+ */
+ static bool firstUseInBackend = true;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ int os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstUseInBackend)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ pg_buffercache_build_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
}
- else
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user. Also we could somehow report single DB block spanning
+ * more than one NUMA zone, but it should be rare.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
}
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstUseInBackend = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..e2e8cd6a241
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,21 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..75978a6eaed 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,13 @@
convenient use.
</para>
+ <para>
+ The similiar <function>pg_buffercache_numa_pages()</function> is a slower
+ variant of the above, but also can provide NUMA node ID for shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +209,59 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are almost identical to the previous
+ <structname>pg_buffercache</structname> view, but this one includes one additional
+ column numa_zone_id as defined in <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ ID (number) of the NUMA node for this particular buffer. NULL if the shared buffer
+ has not been used yet.On systems without NUMA this usually returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ This is clone version of the original pg_buffercache view, however it provides
+ additional <structfield>numa_zone_id</structfield> column. Fetching this
+ information from OS is costly and might take much longer and querying it is not
+ recommended by automated or monitoring systems.
+ </para>
+
+ <para>
+ As NUMA node ID inquiry for each page requires memory pages to be paged-in, first
+ execution of this function can take long time especially on systems with bigint
+ shared_buffers and without huge_pages enabled.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
[application/octet-stream] v10-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch (15.4K, ../../CAKZiRmzwO_KEXH4hkrHmx3qja0kER9pKW=3yyLvEP+b38vVLQg@mail.gmail.com/4-v10-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch)
download | inline diff:
From 189be69e5a256bf966ab7f883f6635f377e0a6bc Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v10 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 78 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 123 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 10 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 265 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..5164083131a 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,79 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> view shows NUMA nodes
+ assigned allocations made from the server's main shared memory segment.
+ This includes both memory allocated by <productname>PostgreSQL</productname>
+ itself and memory allocated by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of NUMA node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf5..cc014a62dc2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 4c9c3cb320f..61e603fc42a 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -579,3 +579,126 @@ pg_numa_available(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(true);
}
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count,
+ max_zones;
+ /* To get reliable results we need to "touch pages" once */
+ static bool firstUseInBackend = true;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+ max_zones = pg_numa_get_max_node();
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+ Size *zones;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ zones = palloc0(sizeof(Size) * max_zones);
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= max_zones; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+
+ pfree(zones);
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38612d8ae12..7fd68702be9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8493,6 +8493,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index a76256405fe..38ff8bcabe8 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b2..b63c6e0f744 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 37b6d21e1f9..c07a4c7633a 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..e748434c2fe
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,10 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index d195aaf1377..28261fd774b 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-12 15:58 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-12 15:58 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, Mar 12, 2025 at 5:35 PM Alvaro Herrera <[email protected]> wrote:
>
> Hello
>
> When running these tests, I encounter this strange diff in the dumps,
> which seems to be that the locale for type money does not match. I
> imagine the problem is that the locale is not set correctly when
> initdb'ing one of them? Grepping the regress_log for initdb, I see
> this:
>
> $ grep -B1 'Running: initdb' tmp_check/log/regress_log_002_pg_upgrade
> [13:00:57.580](0.003s) # initializing database system by running initdb
> # Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_old_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale C.UTF-8 -k
> --
> [13:01:12.879](0.044s) # initializing database system by running initdb
> # Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_dst_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale C.UTF-8 -k
> --
> [13:01:28.000](0.033s) # initializing database system by running initdb
> # Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_new_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding SQL_ASCII --locale-provider libc
>
The original node and the node where dump is restored have the same
initdb commands. It's the upgraded node which has different initdb
command. But that's how the test is written originally.
>
>
> [12:50:31.838](0.102s) not ok 15 - dump outputs from original and restored regression database (using plain format) match
> [12:50:31.839](0.000s)
> [12:50:31.839](0.000s) # Failed test 'dump outputs from original and restored regression database (using plain format) match'
> # at /pgsql/source/master/src/test/perl/PostgreSQL/Test/Utils.pm line 797.
> [12:50:31.839](0.000s) # got: '1'
> # expected: '0'
> === diff of /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/src_dump.sql_adjusted and /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/dest_dump.plain.sql_adjusted
> === stdout ===
> --- /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/src_dump.sql_adjusted 2025-03-12 12:50:27.674918597 +0100
> +++ /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/tmp_test_vVew/dest_dump.plain.sql_adjusted 2025-03-12 12:50:31.778840338 +0100
> @@ -208972,7 +208972,7 @@
> -- Data for Name: money_data; Type: TABLE DATA; Schema: public; Owner: alvherre
> --
> COPY public.money_data (m) FROM stdin;
> -$123.46
> +$ 12.346,00
> \.
> --
> -- Data for Name: mvtest_t; Type: TABLE DATA; Schema: public; Owner: alvherre
> @@ -376231,7 +376231,7 @@
> -- Data for Name: tab_core_types; Type: TABLE DATA; Schema: public; Owner: alvherre
> --
> COPY public.tab_core_types (point, line, lseg, box, openedpath, closedpath, polygon, circle, date, "time", "timestamp", timetz, timestamptz, "interval", "json", jsonb, jsonpath, inet, cidr, macaddr8, macaddr, int2, int4, int8, float4, float8, pi, "char", bpchar, "varchar", name, text, bool, bytea, "bit", varbit, money, refcursor, int2vector, oidvector, aclitem, tsvector, tsquery, uuid, xid8, regclass, type, regrole, oid, tid, xid, cid, txid_snapshot, pg_snapshot, pg_lsn, cardinal_number, character_data, sql_identifier, time_stamp, yes_or_no, int4range, int4multirange, int8range, int8multirange, numrange, nummultirange, daterange, datemultirange, tsrange, tsmultirange, tstzrange, tstzmultirange) FROM stdin;
> -(11,12) {1,-1,0} [(11,11),(12,12)] (13,13),(11,11) ((11,12),(13,13),(14,14)) [(11,12),(13,13),(14,14)] ((11,12),(13,13),(14,14)) <(1,1),1> 2025-03-12 04:50:14.125899 2025-03-12 04:50:14.125899 04:50:14.125899-07 2025-03-12 12:50:14.125899+01 00:00:12 {"reason":"because"} {"when": "now"} $."a"[*]?(@ > 2) 127.0.0.1 127.0.0.0/8 00:01:03:ff:fe:86:1c:ba 00:01:03:86:1c:ba 2 4 8 4 8 3.14159265358979 f c abc name txt t \\xdeadbeef 1 10001 $12.34 abc 1 2 1 2 alvherre=UC/alvherre 'a' 'and' 'ate' 'cat' 'fat' 'mat' 'on' 'rat' 'sat' 'fat' & 'rat' a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11 11 pg_class regtype pg_monitor 1259 (1,1) 2 3 10:20:10,14,15 10:20:10,14,15 16/B374D848 1 l n 2025-03-12 12:50:14.13+01 YES empty {} empty {} (3,4) {(3,4)} [2020-01-03,2021-02-03) {[2020-01-03,2021-02-03)} ("2020-01-02 03:04:05","2021-02-03 06:07:08") {("2020-01-02 03:04:05","2021-02-03 06:07:08")} ("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01") {("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01")}
> +(11,12) {1,-1,0} [(11,11),(12,12)] (13,13),(11,11) ((11,12),(13,13),(14,14)) [(11,12),(13,13),(14,14)] ((11,12),(13,13),(14,14)) <(1,1),1> 2025-03-12 04:50:14.125899 2025-03-12 04:50:14.125899 04:50:14.125899-07 2025-03-12 12:50:14.125899+01 00:00:12 {"reason":"because"} {"when": "now"} $."a"[*]?(@ > 2) 127.0.0.1 127.0.0.0/8 00:01:03:ff:fe:86:1c:ba 00:01:03:86:1c:ba 2 4 8 4 8 3.14159265358979 f c abc name txt t \\xdeadbeef 1 10001 $ 1.234,00 abc 1 2 1 2 alvherre=UC/alvherre 'a' 'and' 'ate' 'cat' 'fat' 'mat' 'on' 'rat' 'sat' 'fat' & 'rat' a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11 11 pg_class regtype pg_monitor 1259 (1,1) 2 3 10:20:10,14,15 10:20:10,14,15 16/B374D848 1 l n 2025-03-12 12:50:14.13+01 YES empty {} empty {} (3,4) {(3,4)} [2020-01-03,2021-02-03) {[2020-01-03,2021-02-03)} ("2020-01-02 03:04:05","2021-02-03 06:07:08") {("2020-01-02 03:04:05","2021-02-03 06:07:08")} ("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01") {("2020-01-02 12:04:05+01","2021-02-03 15:07:08+01")}
> \.
> --
> -- Data for Name: tableam_parted_a_heap2; Type: TABLE DATA; Schema: public; Owner: alvherre=== stderr ===
> === EOF ===
>
However these differences are coming from original and restored
database which are using the same initdb options.
Does the test pass for you if you don't apply my patches?
Over at [1], I had seen a locale related failure without applying my patches.
[1] https://www.postgresql.org/message-id/[email protected]...
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-12 16:09 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-12 16:09 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-12, Ashutosh Bapat wrote:
> Does the test pass for you if you don't apply my patches?
Yes. It also passes if I keep PG_TEST_EXTRA empty.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-13 06:52 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-13 06:52 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi Alvaro,
On Wed, Mar 12, 2025 at 9:39 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-12, Ashutosh Bapat wrote:
>
> > Does the test pass for you if you don't apply my patches?
>
> Yes. It also passes if I keep PG_TEST_EXTRA empty.
I am not able to reproduce this problem locally.
The test uses
In my case the money is printed $<digits before decimal>.<digits after
decimal> format in both the dumps. But in your case the money printed
from restored database has a space between $ and amount and the amount
also has decimal and comma in odd places - I can't figure out what
that means or what lc_monetary value would print something like that.
Can you please help me with
1. can you please run the test again and share the dump outputs. They
will be located in a temporary directory with names
src_dump.sql_adjusted and dest_dump.<format>.sql_adjusted.
2. Are you seeing this diff only with plain format or other formats as well?
Sorry for the trouble.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-13 08:42 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-13 08:42 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Hello
On 2025-Mar-13, Ashutosh Bapat wrote:
> 1. can you please run the test again and share the dump outputs. They
> will be located in a temporary directory with names
> src_dump.sql_adjusted and dest_dump.<format>.sql_adjusted.
Ah, I see the problem :-) The first initdb does this:
# Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_old_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale C.UTF-8 -k
The files belonging to this database system will be owned by user "alvherre".
This user must also own the server process.
The database cluster will be initialized with this locale configuration:
locale provider: builtin
default collation: C.UTF-8
LC_COLLATE: C
LC_CTYPE: C
LC_MESSAGES: C
LC_MONETARY: es_CL.UTF-8
LC_NUMERIC: es_CL.UTF-8
LC_TIME: es_CL.UTF-8
The default text search configuration will be set to "english".
Data page checksums are enabled.
which for some reason used my environment setting for LC_MONETARY.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"But static content is just dynamic content that isn't moving!"
http://smylers.hates-software.com/2007/08/15/fe244d0c.html
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-13 09:22 Jakub Wartak <[email protected]>
parent: Jakub Wartak <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: Jakub Wartak @ 2025-03-13 09:22 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Wed, Mar 12, 2025 at 4:41 PM Jakub Wartak
<[email protected]> wrote:
>
> On Mon, Mar 10, 2025 at 11:14 AM Bertrand Drouvot
> <[email protected]> wrote:
>
> > Thanks for the new version!
>
> v10 is attached with most fixes after review and one new thing
> introduced: pg_numa_available() for run-time decision inside tests
> which was needed after simplifying code a little bit as you wanted.
> I've also fixed -Dlibnuma=disabled as it was not properly implemented.
> There are couple open items (minor/decision things), but most is fixed
> or answered:
>
> > Some random comments on 0001:
> >
> > === 1
> >
> > It does not compiles "alone". It's missing:
> >
> [..]
> > +extern PGDLLIMPORT int huge_pages_status;
> [..]
> > -static int huge_pages_status = HUGE_PAGES_UNKNOWN;
> > +int huge_pages_status = HUGE_PAGES_UNKNOWN;
> >
> > That come with 0002. So it has to be in 0001 instead.
>
> Ugh, good catch, I haven't thought about it in isolation, they are
> separate to just ease review, but should be committed together. Fixed.
>
> > === 2
> >
> > +else
> > + as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
> > +fi
> >
> > Maybe we should add the same test (checking for numa.h) for meson?
>
> TBH, I have no idea, libnuma.h may exist but it may not link e.g. when
> cross-compiling 32-bit on 64-bit. Or is this more about keeping sync
> between meson and autoconf?
>
> > === 3
> >
> > +# FIXME: filter-out / with/without with_libnuma?
> > +LIBS += $(LIBNUMA_LIBS)
> >
> > It looks to me that we can remove those 2 lines.
>
> Done.
>
> > === 4
> >
> > + Only supported on Linux.
> >
> > s/on Linux/on platforms for which the libnuma library is implemented/?
> >
> > I did a quick grep on "Only supported on" and it looks like that could be
> > a more consistent wording.
>
> Fixed.
>
> > === 5
> >
> > +#include "c.h"
> > +#include "postgres.h"
> > +#include "port/pg_numa.h"
> > +#include "storage/pg_shmem.h"
> > +#include <unistd.h>
> >
> > I had a closer look to other header files and it looks like it "should" be:
> >
> > #include "c.h"
> > #include "postgres.h"
> >
> > #include <unistd.h>
> > #ifdef WIN32
> > #include <windows.h>
> > #endif
> >
> > #include "port/pg_numa.h"
> > #include "storage/pg_shmem.h"
> >
> > And is "#include "c.h"" really needed?
>
> Fixed both. It seems to compile without c.h.
>
> > === 6
> >
> > +/* FIXME not tested, might crash */
> >
> > That's a bit scary.
>
> When you are in support for long enough it is becoming the norm ;) But
> on serious note Andres wanted have numa error/warning handlers (used
> by libnuma), but current code has no realistic code-path to hit it
> from numa_available(3), numa_move_pages(3) or numa_max_node(3). The
> situation won't change until one day in future (I hope!) we start
> using some more advanced libnuma functionality for interleaving
> memory, please see my earlier reply:
> https://www.postgresql.org/message-id/CAKZiRmzpvBtqrz5Jr2DDcfk4Ar1ppsXkUhEX9RpA%2Bs%2B_5hcTOg%40mail...
> E.g. numa_available(3) is tiny wrapper , see
> https://github.com/numactl/numactl/blob/master/libnuma.c#L872
>
> For now, I've adjusted that FIXME to XXX, but still don't know we
> could inject failure to see this triggered...
>
> > === 7
> >
> > + * XXX: for now we issue just WARNING, but long-term that might depend on
> > + * numa_set_strict() here
> >
> > s/here/here./
>
> Done.
>
> > Did not look carefully at all the comments in 0001, 0002 and 0003 though.
> >
> > A few random comments regarding 0002:
> >
> > === 8
> >
> > # create extension pg_buffercache;
> > ERROR: could not load library "/home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so": /home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so: undefined symbol: pg_numa_query_pages
> > CONTEXT: SQL statement "CREATE FUNCTION pg_buffercache_pages()
> > RETURNS SETOF RECORD
> > AS '$libdir/pg_buffercache', 'pg_buffercache_pages'
> > LANGUAGE C PARALLEL SAFE"
> > extension script file "pg_buffercache--1.2.sql", near line 7
> >
> > While that's ok if 0003 is applied. I think that each individual patch should
> > "fully" work individually.
>
> STILL OPEN QUESTION: Not sure I understand: you need to have 0001 +
> 0002 or 0001 + 0003, but here 0002 is complaining about lack of
> pg_numa_query_pages() which is part of 0001 (?) Should I merge those
> patches or keep them separate?
>
> > === 9
> >
> > + do
> > + {
> > + if (os_page_ptrs[blk2page + j] == 0)
> >
> > blk2page + j will be repeated multiple times, better to store it in a local
> > variable instead?
>
> Done.
>
> > === 10
> >
> > + if (firstUseInBackend == true)
> >
> >
> > if (firstUseInBackend) instead?
>
> Done everywhere.
>
> > === 11
> >
> > + int j = 0,
> > + blk2page = (int) i * pages_per_blk;
> >
> > I wonder if size_t is more appropriate for blk2page:
> >
> > size_t blk2page = (size_t)(i * pages_per_blk) maybe?
>
> Sure, done.
>
> > === 12
> >
> > as we know that we'll iterate until pages_per_blk then would a for loop be more
> > appropriate here, something like?
> >
> > "
> > for (size_t j = 0; j < pages_per_blk; j++)
> > {
> > if (os_page_ptrs[blk2page + j] == 0)
> > {
> > "
>
> Sure.
>
> > === 13
> >
> > + if (buf_state & BM_DIRTY)
> > + fctx->record[i].isdirty = true;
> > + else
> > + fctx->record[i].isdirty = false;
> >
> > fctx->record[i].isdirty = (buf_state & BM_DIRTY) != 0 maybe?
> >
> > === 14
> >
> > + if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
> > + fctx->record[i].isvalid = true;
> > + else
> > + fctx->record[i].isvalid = false;
> >
> > fctx->record[i].isvalid = ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
> > maybe?
>
> It is coming from the original pg_buffercache and it is less readable
> to me, so I don't want to touch that too much because that might open
> refactoring doors too wide.
>
> > === 15
> >
> > +populate_buffercache_entry(int i, BufferCachePagesContext *fctx)
> >
> > I wonder if "i" should get a more descriptive name?
>
> Done, buffer_id.
>
> > === 16
> >
> > s/populate_buffercache_entry/pg_buffercache_build_tuple/? (to be consistent
> > with pg_stat_io_build_tuples for example).
> >
> > I now realize that I did propose populate_buffercache_entry() up-thread,
> > sorry for changing my mind.
>
> OK, I've tried to create a better name, but all my ideas were too
> long, but pg_buffercache_build_tuple sounds nice, so lets use that.
>
> > === 17
> >
> > + static bool firstUseInBackend = true;
> >
> > maybe we should give it a more descriptive name?
>
> I couldn't come up with anything that wouldn't look too long, so
> instead I've added a comment explaining the meaning behind this
> variable, hope that's good enough.
>
> > Also I need to think more about how firstUseInBackend is used, for example:
> >
> > ==== 17.1
> >
> > would it be better to define it as a file scope variable? (at least that
> > would avoid to pass it as an extra parameter in some functions).
>
> I have no strong opinion on this, but I have one doubt (for future):
> isn't creating global variables making life harder for upcoming
> multithreading guys ?
>
> > === 17.2
> >
> > what happens if firstUseInBackend is set to false and later on the pages
> > are moved to different NUMA nodes. Then pg_buffercache_numa_pages() is
> > called again by a backend that already had set firstUseInBackend to false,
> > would that provide accurate information?
>
> It is still the correct result. That "touching" (paging-in) is only
> necessary probably to properly resolve PTEs as the fork() does not
> seem to carry them over from parent:
>
> postgres=# select 'create table foo' || s || ' as select
> generate_series(1, 100000);' from generate_series(1, 4) s;
> postgres=# \gexec
> SELECT 100000
> SELECT 100000
> SELECT 100000
> SELECT 100000
> postgres=# select numa_zone_id, count(*) from pg_buffercache_numa
> group by numa_zone_id order by numa_zone_id; -- before:
> numa_zone_id | count
> --------------+---------
> 0 | 256
> 1 | 4131
> | 8384221
> postgres=# select pg_backend_pid();
> pg_backend_pid
> ----------------
> 1451
>
> -- now use another root (!) session to "migratepages(8)" from numactl
> will also shift shm segment:
> # migratepages 1451 1 3
>
> -- while above will be in progress for a lot of time , but the outcome
> is visible much faster in that backend (pg is functioning):
> postgres=# select numa_zone_id, count(*) from pg_buffercache_numa
> group by numa_zone_id order by numa_zone_id;
> numa_zone_id | count
> --------------+---------
> 0 | 256
> 3 | 4480
> | 8383872
>
> So the above clearly shows that initial touching of shm is required,
> but just once and it stays valid afterwards.
>
> BTW: migratepages(8) was stuck for 1-2 minutes there on
> "__wait_rcu_gp" according to it's wchan, without any sign of activity
> on the OS and then out of blue completed just fine, s_b=64GB,HP=on.
>
> > === 18
> >
> > +Datum
> > +pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
> > +{
> > + FuncCallContext *funcctx;
> > + BufferCachePagesContext *fctx; /* User function context. */
> > + bool query_numa = true;
> >
> > I don't think we need query_numa anymore in pg_buffercache_numa_pages().
> >
> > I think that we can just ERROR (or NOTICE and return) here:
> >
> > + if (pg_numa_init() == -1)
> > + {
> > + elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
> > + query_numa = false;
> >
> > and fully get rid of query_numa.
>
> Right... fixed it here, but it made the tests blow up, so I've had to
> find a way to conditionally launch tests based on NUMA availability
> and that's how pg_numa_available() was born. It's in ipc/shmem.c
> because I couldn't find a better place for it...
>
> > === 19
> >
> > And then here:
> >
> > for (i = 0; i < NBuffers; i++)
> > {
> > populate_buffercache_entry(i, fctx);
> > if (query_numa)
> > pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
> > }
> >
> > if (query_numa)
> > {
> > if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
> > elog(ERROR, "failed NUMA pages inquiry: %m");
> >
> > for (i = 0; i < NBuffers; i++)
> > {
> > int blk2page = (int) i * pages_per_blk;
> >
> > /*
> > * Technically we can get errors too here and pass that to
> > * user. Also we could somehow report single DB block spanning
> > * more than one NUMA zone, but it should be rare.
> > */
> > fctx->record[i].numa_zone_id = os_pages_status[blk2page];
> > }
> >
> > maybe we can just loop a single time over "for (i = 0; i < NBuffers; i++)"?
>
> Well, pg_buffercache_numa_prepare_ptrs() is just an inlined wrapper
> that prepares **os_page_ptrs, which is then used by
> pg_numa_query_pages() and then we fill the data. But now after
> removing query_numa , it reads smoother anyway. Can you please take a
> look again on this, is this up to the project standards?
>
> > A few random comments regarding 0003:
> >
> > === 20
> >
> > + The <structname>pg_shmem_allocations</structname> view shows NUMA nodes
> >
> > s/pg_shmem_allocations/pg_shmem_numa_allocations/?
>
> Fixed.
>
> > === 21
> >
> > + /* FIXME: should we release LWlock here ? */
> > + elog(ERROR, "failed NUMA pages inquiry status: %m");
> >
> > There is no need, see src/backend/storage/lmgr/README.
>
> Thanks, fixed.
>
> > === 22
> >
> > +#define MAX_NUMA_ZONES 32 /* FIXME? */
> > + Size zones[MAX_NUMA_ZONES];
> >
> > could we rely on pg_numa_get_max_node() instead?
>
> Sure, done.
>
> > === 23
> >
> > + if (s >= 0)
> > + zones[s]++;
> >
> > should we also check that s is below a limit?
>
> STILL OPEN QUESTION: I'm not sure it would give us value to report
> e.g. -2 on per shm entry/per numa node entry, would it? If it would we
> could somehow overallocate that array and remember negative ones too.
>
> > === 24
> >
> > Regarding how we make use of pg_numa_get_max_node(), are we sure there is
> > no possible holes? I mean could a system have node 0,1 and 3 but not 2?
>
> I have never seen a hole in numbering as it is established during
> boot, and the only way that could get close to adjusting it could be
> making processor books (CPU and RAM together) offline. Even *if*
> someone would be doing some preventive hardware maintenance, that
> still wouldn't hurt, as we are just using the Id of the zone to
> display it -- the max would be already higher. I mean, technically one
> could use lsmem(1) (it mentions removable flag, after let's pages and
> processes migrated away and from there) and then use chcpu(1) to
> --disable CPUs on that zone (to prevent new ones coming there and
> allocating new local memory there) and then offlining that memory
> region via chmem(1) --disable. Even with all of that, that still
> shouldn't cause issues for this code I think, because
> `numa_max_node()` says it `returns the >> highest node number <<<
> available on the current system.`
>
> > Also I don't think I'm a Co-author, I think I'm just a reviewer (even if I
> > did a little in 0001 though)
>
> That was an attempt to say "Thank You", OK, I've aligned it that way,
> so you are still referenced in 0001. I wouldn't find motivation to
> work on this if you won't respond to those emails ;)
>
> There is one known issue, CI returned for numa.out test, that I need
> to get bottom to (it does not reproduce for me locally) inside
> numa.out/sql:
>
> SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations
> WARNING: detected write past chunk end in ExprContext 0x55bb7f1a8f90
> WARNING: detected write past chunk end in ExprContext 0x55bb7f1a8f90
> WARNING: detected write past chunk end in ExprContext 0x55bb7f1a8f90
>
Hi, ok, so v11 should fix this last problem and make cfbot happy.
-J.
Attachments:
[application/octet-stream] v11-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (27.8K, ../../CAKZiRmw+EtapdJrageWrMhn35=cP2XZD4JK6ox_gpHnKsbG-Mg@mail.gmail.com/2-v11-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From df7f25e69e7e39d564e78783dab4d4fa4287dc4d Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v11 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 ++
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 462 +++++++++++++-----
.../sql/pg_buffercache_numa.sql | 21 +
doc/src/sgml/pgbuffercache.sgml | 64 ++-
9 files changed, 493 insertions(+), 134 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..52f63aa258c
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..9cc04320d63 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,12 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +43,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -61,84 +62,249 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine to map Buffers into addresses that can be
+ * later consumed by pg_numa_query_pages()
+ *
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk, Size os_page_size,
+ void **os_page_ptrs, bool firstUseInBackend)
+{
+ size_t blk2page = (size_t)(buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers count start really from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) + (os_page_size * j);
+
+ /* We just need to do it only once in backend */
+ if (firstUseInBackend)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ }
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages.
+ *
+ * We need fcinfo here and we pass it here with PG_FUNCTION_ARGS
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static void
+pg_buffercache_build_tuple(int i, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(i);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[i].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[i].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[i].blocknum = bufHdr->tag.blockNum;
+ fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[i].isdirty = true;
+ else
+ fctx->record[i].isdirty = false;
+
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[i].isvalid = true;
+ else
+ fctx->record[i].isvalid = false;
+
+ fctx->record[i].numa_zone_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static Datum
+get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[i].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[i].blocknum == InvalidBlockNumber ||
+ fctx->record[i].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[i].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[i].usagecount);
+ nulls[7] = false;
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
+ }
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
+
+/*
+ * When updating this routine please sync it with below one:
+ * pg_buffercache_numa_pages()
+ */
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +315,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_build_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +325,122 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+ /*
+ * To get reliable results we need to "touch pages" once, see
+ * comments nearby pg_buffercache_numa_prepare_ptrs().
+ */
+ static bool firstUseInBackend = true;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ int os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstUseInBackend)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ pg_buffercache_build_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs, firstUseInBackend);
}
- else
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user. Also we could somehow report single DB block spanning
+ * more than one NUMA zone, but it should be rare.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
}
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstUseInBackend = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..e2e8cd6a241
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,21 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..75978a6eaed 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,13 @@
convenient use.
</para>
+ <para>
+ The similiar <function>pg_buffercache_numa_pages()</function> is a slower
+ variant of the above, but also can provide NUMA node ID for shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +209,59 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are almost identical to the previous
+ <structname>pg_buffercache</structname> view, but this one includes one additional
+ column numa_zone_id as defined in <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ ID (number) of the NUMA node for this particular buffer. NULL if the shared buffer
+ has not been used yet.On systems without NUMA this usually returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ This is clone version of the original pg_buffercache view, however it provides
+ additional <structfield>numa_zone_id</structfield> column. Fetching this
+ information from OS is costly and might take much longer and querying it is not
+ recommended by automated or monitoring systems.
+ </para>
+
+ <para>
+ As NUMA node ID inquiry for each page requires memory pages to be paged-in, first
+ execution of this function can take long time especially on systems with bigint
+ shared_buffers and without huge_pages enabled.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
[application/octet-stream] v11-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch (15.4K, ../../CAKZiRmw+EtapdJrageWrMhn35=cP2XZD4JK6ox_gpHnKsbG-Mg@mail.gmail.com/3-v11-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch)
download | inline diff:
From 1161550310b14529d4a42d0ab2ab58f5116708d7 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v11 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 78 +++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 122 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 10 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 264 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..5164083131a 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,79 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> view shows NUMA nodes
+ assigned allocations made from the server's main shared memory segment.
+ This includes both memory allocated by <productname>PostgreSQL</productname>
+ itself and memory allocated by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of NUMA node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf5..cc014a62dc2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 4c9c3cb320f..0dda3c99dd8 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -579,3 +579,125 @@ pg_numa_available(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(true);
}
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count,
+ max_zones;
+ /* To get reliable results we need to "touch pages" once */
+ static bool firstUseInBackend = true;
+ Size *zones;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+ max_zones = pg_numa_get_max_node();
+ zones = palloc(sizeof(Size) * (max_zones + 1));
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ memset(zones, 0, sizeof(Size) * (max_zones + 1));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ if (s >= 0)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= max_zones; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38612d8ae12..7fd68702be9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8493,6 +8493,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '5101', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index a76256405fe..38ff8bcabe8 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b2..b63c6e0f744 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 37b6d21e1f9..c07a4c7633a 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..e748434c2fe
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,10 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index d195aaf1377..28261fd774b 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
[application/octet-stream] v11-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (19.0K, ../../CAKZiRmw+EtapdJrageWrMhn35=cP2XZD4JK6ox_gpHnKsbG-Mg@mail.gmail.com/4-v11-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From f77b7e1a680a7aad370cf2ac9233b9c17bd5a4a2 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v11 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper.
Other platforms can be added later.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway as i386 is is very memory-only
limited even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 ++-
configure | 87 ++++++++++++++++
configure.ac | 13 +++
doc/src/sgml/func.sgml | 13 +++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 21 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/storage/ipc/shmem.c | 11 ++
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 5 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 43 ++++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 151 ++++++++++++++++++++++++++++
18 files changed, 387 insertions(+), 5 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 5849cbb839a..7010dff7aef 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -519,6 +522,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -835,8 +839,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 51dd8ad6571..071b98e6c9a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25061,6 +25061,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if a <acronym>NUMA</acronym> support is available.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b9..9f56205a1d7 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 13c13748e5d..19500ebdfb2 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,25 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ else
+ libnuma = not_found_dep
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3187,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3843,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..4c9c3cb320f 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port/pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -568,3 +569,13 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL level function returning whether NUMA support was compiled in. */
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ if(pg_numa_init() == -1)
+ PG_RETURN_BOOL(false);
+ PG_RETURN_BOOL(true);
+}
+
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ad25cbb39c5..dd34c79f521 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 42e427f8fe8..38612d8ae12 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8489,6 +8489,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '5102', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
@@ -12477,3 +12481,4 @@
prosrc => 'gist_stratnum_common' },
]
+
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..d3ebe8b5bd8
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..b9348caaca9
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,151 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/* XXX: not tested */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here.
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-13 12:39 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-13 12:39 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 13, 2025 at 2:12 PM Alvaro Herrera <[email protected]> wrote:
>
> Hello
>
> On 2025-Mar-13, Ashutosh Bapat wrote:
>
> > 1. can you please run the test again and share the dump outputs. They
> > will be located in a temporary directory with names
> > src_dump.sql_adjusted and dest_dump.<format>.sql_adjusted.
>
> Ah, I see the problem :-) The first initdb does this:
>
> # Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_old_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale C.UTF-8 -k
> The files belonging to this database system will be owned by user "alvherre".
> This user must also own the server process.
>
> The database cluster will be initialized with this locale configuration:
> locale provider: builtin
> default collation: C.UTF-8
> LC_COLLATE: C
> LC_CTYPE: C
> LC_MESSAGES: C
> LC_MONETARY: es_CL.UTF-8
> LC_NUMERIC: es_CL.UTF-8
> LC_TIME: es_CL.UTF-8
> The default text search configuration will be set to "english".
>
> Data page checksums are enabled.
>
> which for some reason used my environment setting for LC_MONETARY.
>
Thanks. This is super helpful. I am able to reproduce the problem
$ unset LC_MONETARY
$ export PG_TEST_EXTRA=regress_dump_test
$ meson test --suite setup && meson test pg_upgrade/002_pg_upgrade
... snip ...
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
72.38s 44 subtests passed
Ok: 1
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 0
Timeout: 0
Full log written to
/home/ashutosh/work/units/pg_dump_test/build/dev/meson-logs/testlog.txt
$ export LC_MONETARY="es_CL.UTF-8"
$ meson test --suite setup && meson test pg_upgrade/002_pg_upgrade
... snip ...
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade ERROR
69.18s exit status 4
>>> with_icu=no LD_LIBRARY_PATH=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install//home/ashutosh/work/units/pg_dump_test/build/dev/lib/x86_64-linux-gnu REGRESS_SHLIB=/home/ashutosh/work/units/pg_dump_test/build/dev/src/test/regress/regress.so PATH=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install//home/ashutosh/work/units/pg_dump_test/build/dev/bin:/home/ashutosh/work/units/pg_dump_test/build/dev/src/bin/pg_upgrade:/home/ashutosh/work/units/pg_dump_test/build/dev/src/bin/pg_upgrade/test:/home/ashutosh/work/units/pg_dump_test/build/dev/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin MALLOC_PERTURB_=30 share_contrib_dir=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install//home/ashutosh/work/units/pg_dump_test/build/dev/share/postgresql/contrib PG_REGRESS=/home/ashutosh/work/units/pg_dump_test/build/dev/src/test/regress/pg_regress top_builddir=/home/ashutosh/work/units/pg_dump_test/build/dev INITDB_TEMPLATE=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install/initdb-template /usr/bin/python3 /home/ashutosh/work/units/pg_dump_test/build/dev/../../coderoot/pg/src/tools/testwrap --basedir /home/ashutosh/work/units/pg_dump_test/build/dev --srcdir /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/bin/pg_upgrade --pg-test-extra '' --testgroup pg_upgrade --testname 002_pg_upgrade -- /usr/bin/perl -I /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/test/perl -I /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/bin/pg_upgrade /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/bin/pg_upgrade/t/002_pg_upgrade.pl
Ok: 0
Expected Fail: 0
Fail: 1
Unexpected Pass: 0
Skipped: 0
Timeout: 0
I see what's happening. If I set LC_MONETARY environment explicitly,
that's taken by initdb
$ export LC_MONETARY="es_CL.UTF-8";rm -rf $DataDir; $BinDir/initdb -D
$DataDir -A trust -N --wal-segsize 1 --allow-group-access --encoding
UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin
--builtin-locale C.UTF-8 -k
The files belonging to this database system will be owned by user "ashutosh".
This user must also own the server process.
The database cluster will be initialized with this locale configuration:
locale provider: builtin
default collation: C.UTF-8
LC_COLLATE: C
LC_CTYPE: C
LC_MESSAGES: en_US.UTF-8
LC_MONETARY: es_CL.UTF-8
LC_NUMERIC: en_US.UTF-8
LC_TIME: en_US.UTF-8
The default text search configuration will be set to "english".
If I don't set it explicitly, it's taken from default settings
$ unset LC_MONETARY;rm -rf $DataDir; $BinDir/initdb -D $DataDir -A
trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8
--lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale
C.UTF-8 -k
The files belonging to this database system will be owned by user "ashutosh".
This user must also own the server process.
The database cluster will be initialized with this locale configuration:
locale provider: builtin
default collation: C.UTF-8
LC_COLLATE: C
LC_CTYPE: C
LC_MESSAGES: en_US.UTF-8
LC_MONETARY: en_US.UTF-8
LC_NUMERIC: en_US.UTF-8
LC_TIME: en_US.UTF-8
The default text search configuration will be set to "english".
In your case probably your default setting is es_CL.UTF-8 or have set
LC_MONETARY explicitly in your environment.
I think the fix is to explicitly pass --lc-monetary to the old cluster
and the restored cluster. 003 patch in the attached patch set does
that. Please check if it fixes the issue for you.
Additionally we should check that it gets copied to the new cluster as
well. But I haven't figured out how to get those settings yet. This
treatment is similar to how --lc-collate and --lc-ctype are treated. I
am wondering whether we should explicitly pass --lc-messages,
--lc-time and --lc-numeric as well.
2d819a08a1cbc11364e36f816b02e33e8dcc030b introduced buildin locale
provider and added overrides to LC_COLLATE and LC_TYPE. But it did not
override other LC_, which I think it should have. In pure upgrade
test, the upgraded node inherits the locale settings of the original
cluster, so this wasn't apparent. But with pg_dump testing, the
original and restored databases are independent. Hence I think we have
to override all LC_* settings by explicitly mentioning --lc-* options
to initdb. Please let me know what you think about this?
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-13 12:40 Ashutosh Bapat <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-13 12:40 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Here are patches missing in the previous email.
On Thu, Mar 13, 2025 at 6:09 PM Ashutosh Bapat
<[email protected]> wrote:
>
> On Thu, Mar 13, 2025 at 2:12 PM Alvaro Herrera <[email protected]> wrote:
> >
> > Hello
> >
> > On 2025-Mar-13, Ashutosh Bapat wrote:
> >
> > > 1. can you please run the test again and share the dump outputs. They
> > > will be located in a temporary directory with names
> > > src_dump.sql_adjusted and dest_dump.<format>.sql_adjusted.
> >
> > Ah, I see the problem :-) The first initdb does this:
> >
> > # Running: initdb -D /home/alvherre/Code/pgsql-build/master/src/bin/pg_upgrade/tmp_check/t_002_pg_upgrade_old_node_data/pgdata -A trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale C.UTF-8 -k
> > The files belonging to this database system will be owned by user "alvherre".
> > This user must also own the server process.
> >
> > The database cluster will be initialized with this locale configuration:
> > locale provider: builtin
> > default collation: C.UTF-8
> > LC_COLLATE: C
> > LC_CTYPE: C
> > LC_MESSAGES: C
> > LC_MONETARY: es_CL.UTF-8
> > LC_NUMERIC: es_CL.UTF-8
> > LC_TIME: es_CL.UTF-8
> > The default text search configuration will be set to "english".
> >
> > Data page checksums are enabled.
> >
> > which for some reason used my environment setting for LC_MONETARY.
> >
>
> Thanks. This is super helpful. I am able to reproduce the problem
> $ unset LC_MONETARY
> $ export PG_TEST_EXTRA=regress_dump_test
> $ meson test --suite setup && meson test pg_upgrade/002_pg_upgrade
> ... snip ...
> 1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
> 72.38s 44 subtests passed
>
>
> Ok: 1
> Expected Fail: 0
> Fail: 0
> Unexpected Pass: 0
> Skipped: 0
> Timeout: 0
>
> Full log written to
> /home/ashutosh/work/units/pg_dump_test/build/dev/meson-logs/testlog.txt
> $ export LC_MONETARY="es_CL.UTF-8"
> $ meson test --suite setup && meson test pg_upgrade/002_pg_upgrade
> ... snip ...
> 1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade ERROR
> 69.18s exit status 4
> >>> with_icu=no LD_LIBRARY_PATH=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install//home/ashutosh/work/units/pg_dump_test/build/dev/lib/x86_64-linux-gnu REGRESS_SHLIB=/home/ashutosh/work/units/pg_dump_test/build/dev/src/test/regress/regress.so PATH=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install//home/ashutosh/work/units/pg_dump_test/build/dev/bin:/home/ashutosh/work/units/pg_dump_test/build/dev/src/bin/pg_upgrade:/home/ashutosh/work/units/pg_dump_test/build/dev/src/bin/pg_upgrade/test:/home/ashutosh/work/units/pg_dump_test/build/dev/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin MALLOC_PERTURB_=30 share_contrib_dir=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install//home/ashutosh/work/units/pg_dump_test/build/dev/share/postgresql/contrib PG_REGRESS=/home/ashutosh/work/units/pg_dump_test/build/dev/src/test/regress/pg_regress top_builddir=/home/ashutosh/work/units/pg_dump_test/build/dev INITDB_TEMPLATE=/home/ashutosh/work/units/pg_dump_test/build/dev/tmp_install/initdb-template /usr/bin/python3 /home/ashutosh/work/units/pg_dump_test/build/dev/../../coderoot/pg/src/tools/testwrap --basedir /home/ashutosh/work/units/pg_dump_test/build/dev --srcdir /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/bin/pg_upgrade --pg-test-extra '' --testgroup pg_upgrade --testname 002_pg_upgrade -- /usr/bin/perl -I /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/test/perl -I /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/bin/pg_upgrade /home/ashutosh/work/units/pg_dump_test/coderoot/pg/src/bin/pg_upgrade/t/002_pg_upgrade.pl
>
>
>
> Ok: 0
> Expected Fail: 0
> Fail: 1
> Unexpected Pass: 0
> Skipped: 0
> Timeout: 0
>
> I see what's happening. If I set LC_MONETARY environment explicitly,
> that's taken by initdb
> $ export LC_MONETARY="es_CL.UTF-8";rm -rf $DataDir; $BinDir/initdb -D
> $DataDir -A trust -N --wal-segsize 1 --allow-group-access --encoding
> UTF-8 --lc-collate C --lc-ctype C --locale-provider builtin
> --builtin-locale C.UTF-8 -k
> The files belonging to this database system will be owned by user "ashutosh".
> This user must also own the server process.
>
> The database cluster will be initialized with this locale configuration:
> locale provider: builtin
> default collation: C.UTF-8
> LC_COLLATE: C
> LC_CTYPE: C
> LC_MESSAGES: en_US.UTF-8
> LC_MONETARY: es_CL.UTF-8
> LC_NUMERIC: en_US.UTF-8
> LC_TIME: en_US.UTF-8
> The default text search configuration will be set to "english".
>
> If I don't set it explicitly, it's taken from default settings
> $ unset LC_MONETARY;rm -rf $DataDir; $BinDir/initdb -D $DataDir -A
> trust -N --wal-segsize 1 --allow-group-access --encoding UTF-8
> --lc-collate C --lc-ctype C --locale-provider builtin --builtin-locale
> C.UTF-8 -k
> The files belonging to this database system will be owned by user "ashutosh".
> This user must also own the server process.
>
> The database cluster will be initialized with this locale configuration:
> locale provider: builtin
> default collation: C.UTF-8
> LC_COLLATE: C
> LC_CTYPE: C
> LC_MESSAGES: en_US.UTF-8
> LC_MONETARY: en_US.UTF-8
> LC_NUMERIC: en_US.UTF-8
> LC_TIME: en_US.UTF-8
> The default text search configuration will be set to "english".
>
> In your case probably your default setting is es_CL.UTF-8 or have set
> LC_MONETARY explicitly in your environment.
>
> I think the fix is to explicitly pass --lc-monetary to the old cluster
> and the restored cluster. 003 patch in the attached patch set does
> that. Please check if it fixes the issue for you.
>
> Additionally we should check that it gets copied to the new cluster as
> well. But I haven't figured out how to get those settings yet. This
> treatment is similar to how --lc-collate and --lc-ctype are treated. I
> am wondering whether we should explicitly pass --lc-messages,
> --lc-time and --lc-numeric as well.
>
> 2d819a08a1cbc11364e36f816b02e33e8dcc030b introduced buildin locale
> provider and added overrides to LC_COLLATE and LC_TYPE. But it did not
> override other LC_, which I think it should have. In pure upgrade
> test, the upgraded node inherits the locale settings of the original
> cluster, so this wasn't apparent. But with pg_dump testing, the
> original and restored databases are independent. Hence I think we have
> to override all LC_* settings by explicitly mentioning --lc-* options
> to initdb. Please let me know what you think about this?
>
> --
> Best Wishes,
> Ashutosh Bapat
--
Best Wishes,
Ashutosh Bapat
Attachments:
[text/x-patch] 0001-Test-pg_dump-restore-of-regression-objects-20250313.patch (16.1K, ../../CAExHW5tOhQi2Fyf5My-YK3uzP8QwVJZQDfC3o-vvAxUUG-CNhg@mail.gmail.com/2-0001-Test-pg_dump-restore-of-regression-objects-20250313.patch)
download | inline diff:
From ec6c178ba0a1a20dc989fee94fdc8d53d531e2e4 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Thu, 27 Jun 2024 10:03:53 +0530
Subject: [PATCH 1/3] Test pg_dump/restore of regression objects
002_pg_upgrade.pl tests pg_upgrade of the regression database left
behind by regression run. Modify it to test dump and restore of the
regression database as well.
Regression database created by regression run contains almost all the
database objects supported by PostgreSQL in various states. Hence the
new testcase covers dump and restore scenarios not covered by individual
dump/restore cases. Till now 002_pg_upgrade only tested dump/restore
through pg_upgrade which only uses binary mode. Many regression tests
mention that they leave objects behind for dump/restore testing but they
are not tested in a non-binary mode. The new testcase closes that
gap.
Testing dump and restore of regression database makes this test run
longer for a relatively smaller benefit. Hence run it only when
explicitly requested by user by specifying "regress_dump_test" in
PG_TEST_EXTRA.
Note For the reviewers:
The new test has uncovered two bugs so far in one year.
1. Introduced by 14e87ffa5c54. Fixed in fd41ba93e4630921a72ed5127cd0d552a8f3f8fc.
2. Introduced by 0413a556990ba628a3de8a0b58be020fd9a14ed0. Reverted in 74563f6b90216180fc13649725179fc119dddeb5.
Author: Ashutosh Bapat
Reviewed by: Michael Pacquire, Daniel Gustafsson, Tom Lane, Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
---
doc/src/sgml/regress.sgml | 12 ++
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 141 ++++++++++++++++-
src/test/perl/Makefile | 2 +
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 167 ++++++++++++++++++++
src/test/perl/meson.build | 1 +
5 files changed, 321 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/AdjustDump.pm
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 0e5e8e8f309..237b974b3ab 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,6 +357,18 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>regress_dump_test</literal></term>
+ <listitem>
+ <para>
+ When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
+ tests dump and restore of regression database left behind by the
+ regression run. Not enabled by default because it is time and resource
+ consuming.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index c00cf68d660..bd8313cee6f 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -12,6 +12,7 @@ use File::Path qw(rmtree);
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::AdjustUpgrade;
+use PostgreSQL::Test::AdjustDump;
use Test::More;
# Can be changed to test the other modes.
@@ -35,8 +36,8 @@ sub generate_db
"created database with ASCII characters from $from_char to $to_char");
}
-# Filter the contents of a dump before its use in a content comparison.
-# This returns the path to the filtered dump.
+# Filter the contents of a dump before its use in a content comparison for
+# upgrade testing. This returns the path to the filtered dump.
sub filter_dump
{
my ($is_old, $old_version, $dump_file) = @_;
@@ -261,6 +262,21 @@ else
}
}
is($rc, 0, 'regression tests pass');
+
+ # Test dump/restore of the objects left behind by regression. Ideally it
+ # should be done in a separate TAP test, but doing it here saves us one full
+ # regression run.
+ #
+ # This step takes several extra seconds and some extra disk space, so
+ # requires an opt-in with the PG_TEST_EXTRA environment variable.
+ #
+ # Do this while the old cluster is running before it is shut down by the
+ # upgrade test.
+ if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
+ {
+ test_regression_dump_restore($oldnode, %node_params);
+ }
}
# Initialize a new node for the upgrade.
@@ -527,4 +543,125 @@ my $dump2_filtered = filter_dump(0, $oldnode->pg_version, $dump2_file);
compare_files($dump1_filtered, $dump2_filtered,
'old and new dumps match after pg_upgrade');
+# Test dump and restore of objects left behind by the regression run.
+#
+# It is expected that regression tests, which create `regression` database, are
+# run on `src_node`, which in turn, is left in running state. The dump from
+# `src_node` is restored on a fresh node created using given `node_params`.
+# Plain dumps from both the nodes are compared to make sure that all the dumped
+# objects are restored faithfully.
+sub test_regression_dump_restore
+{
+ my ($src_node, %node_params) = @_;
+ my $dst_node = PostgreSQL::Test::Cluster->new('dst_node');
+
+ # Make sure that the source and destination nodes have the same version and
+ # do not use custom install paths. In both the cases, the dump files may
+ # require additional adjustments unknown to code here. Do not run this test
+ # in such a case to avoid utilizing the time and resources unnecessarily.
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default installation");
+ return;
+ }
+
+ # Dump the original database for comparison later.
+ my $src_dump =
+ get_dump_for_comparison($src_node, 'regression', 'src_dump', 1);
+
+ # Setup destination database cluster
+ $dst_node->init(%node_params);
+ # Stabilize stats for comparison.
+ $dst_node->append_conf('postgresql.conf', 'autovacuum = off');
+ $dst_node->start;
+
+ for my $format ('plain', 'tar', 'directory', 'custom')
+ {
+ my $dump_file = "$tempdir/regression_dump.$format";
+ my $restored_db = 'regression_' . $format;
+
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ # Create a new database for restoring dump from every format so that it
+ # is available for debugging in case the test fails.
+ $dst_node->command_ok([ 'createdb', $restored_db ],
+ "created destination database '$restored_db'");
+
+ # Restore into destination database.
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [
+ 'psql', '-d', $dst_node->connstr($restored_db),
+ '-f', $dump_file
+ ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '-d',
+ $dst_node->connstr($restored_db), $dump_file
+ ];
+ }
+ $dst_node->command_ok(@restore_command,
+ "restored dump taken in $format format on destination instance");
+
+ my $dst_dump =
+ get_dump_for_comparison($dst_node, $restored_db,
+ 'dest_dump.' . $format, 0);
+
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database (using $format format) match"
+ );
+ }
+}
+
+# Dump database `db` from the given `node` in plain format and adjust it for
+# comparing dumps from the original and the restored database.
+#
+# `file_prefix` is used to create unique names for all dump files so that they
+# remain available for debugging in case the test fails.
+#
+# `adjust_child_columns` is passed to adjust_regress_dumpfile() which actually
+# adjusts the dump output.
+#
+# The name of the file containting adjusted dump is returned.
+sub get_dump_for_comparison
+{
+ my ($node, $db, $file_prefix, $adjust_child_columns) = @_;
+
+ my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
+ my $dump_adjusted = "${dumpfile}_adjusted";
+
+ # Usually we avoid comparing statistics in our tests since it is flaky by
+ # nature. However, if statistics is dumped and restored it is expected to be
+ # restored as it is i.e. the statistics from the original database and that
+ # from the restored database should match. We turn off autovacuum on the
+ # source and the target database to avoid any statistics update during
+ # restore operation. Hence we do not exclude statistics from dump.
+ $node->command_ok(
+ [
+ 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ $dumpfile
+ ],
+ 'dump for comparison succeeded');
+
+ open(my $dh, '>', $dump_adjusted)
+ || die
+ "could not open $dump_adjusted for writing the adjusted dump: $!";
+ print $dh adjust_regress_dumpfile(slurp_file($dumpfile),
+ $adjust_child_columns);
+ close($dh);
+
+ return $dump_adjusted;
+}
+
done_testing();
diff --git a/src/test/perl/Makefile b/src/test/perl/Makefile
index d82fb67540e..def89650ead 100644
--- a/src/test/perl/Makefile
+++ b/src/test/perl/Makefile
@@ -26,6 +26,7 @@ install: all installdirs
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/Cluster.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/BackgroundPsql.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustUpgrade.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustDump.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Version.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
uninstall:
@@ -36,6 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
endif
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..74b9a60cf34
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,167 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::AdjustDump - helper module for dump and restore tests
+
+=head1 SYNOPSIS
+
+ use PostgreSQL::Test::AdjustDump;
+
+ # Adjust contents of dump output file so that dump output from original
+ # regression database and that from the restored regression database match
+ $dump = adjust_regress_dumpfile($dump, $adjust_child_columns);
+
+=head1 DESCRIPTION
+
+C<PostgreSQL::Test::AdjustDump> encapsulates various hacks needed to
+compare the results of dump and restore tests
+
+=cut
+
+package PostgreSQL::Test::AdjustDump;
+
+use strict;
+use warnings FATAL => 'all';
+
+use Exporter 'import';
+use Test::More;
+
+our @EXPORT = qw(
+ adjust_regress_dumpfile
+);
+
+=pod
+
+=head1 ROUTINES
+
+=over
+
+=item $dump = adjust_regress_dumpfile($dump, $adjust_child_columns)
+
+If we take dump of the regression database left behind after running regression
+tests, restore the dump, and take dump of the restored regression database, the
+outputs of both the dumps differ in the following cases. This routine adjusts
+the given dump so that dump outputs from the original and restored database,
+respectively, match.
+
+Case 1: Some regression tests purposefully create child tables in such a way
+that the order of their inherited columns differ from column orders of their
+respective parents. In the restored database, however, the order of their
+inherited columns are same as that of their respective parents. Thus the column
+orders of these child tables in the original database and those in the restored
+database differ, causing difference in the dump outputs. See MergeAttributes()
+and dumpTableSchema() for details. This routine rearranges the column
+declarations in the relevant C<CREATE TABLE... INHERITS> statements in the dump
+file from original database to match those from the restored database. We could,
+instead, adjust the statements in the dump from the restored database to match
+those from original database or adjust both to a canonical order. But we have
+chosen to adjust the statements in the dump from original database for no
+particular reason.
+
+Case 2: When dumping COPY statements the columns are ordered by their attribute
+number by fmtCopyColumnList(). If a column is added to a parent table after a
+child has inherited the parent and the child has its own columns, the attribute
+number of the column changes after restoring the child table. This is because
+when executing the dumped C<CREATE TABLE... INHERITS> statement all the parent
+attributes are created before any child attributes. Thus the order of columns in
+COPY statements dumped from the original and the restored databases,
+respectively, differs. Such tables in regression tests are listed below. It is
+hard to adjust the column order in the COPY statement along with the data. Hence
+we just remove such COPY statements from the dump output.
+
+Additionally the routine adjusts blank and new lines to avoid noise.
+
+Note: Usually we avoid comparing statistics in our tests since it is flaky by
+nature. However, if statistics is dumped and restored it is expected to be
+restored as it is i.e. the statistics from the original database and that from
+the restored database should match. Hence we do not filter statistics from dump,
+if it's dumped.
+
+Arguments:
+
+=over
+
+=item C<dump>: Contents of dump file
+
+=item C<adjust_child_columns>: 1 indicates that the given dump file requires
+adjusting columns in the child tables; usually when the dump is from original
+database. 0 indicates no such adjustment is needed; usually when the dump is
+from restored database.
+
+=back
+
+Returns the adjusted dump text.
+
+=cut
+
+sub adjust_regress_dumpfile
+{
+ my ($dump, $adjust_child_columns) = @_;
+
+ # use Unix newlines
+ $dump =~ s/\r\n/\n/g;
+
+ # Adjust the CREATE TABLE ... INHERITS statements.
+ if ($adjust_child_columns)
+ {
+ my $saved_dump = $dump;
+
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_stored_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_stored_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_virtual_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_virtual_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c1\s\()
+ (\n\s+int_four\sbigint),
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint)/$1$4,$2,$3/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c1 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c2\s\()
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint),
+ (\n\s+int_four\sbigint)/$1$3,$4,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c2 adjustments');
+ }
+
+ # Remove COPY statements with differing column order
+ for my $table (
+ 'public\.b_star', 'public\.c_star',
+ 'public\.cc2', 'public\.d_star',
+ 'public\.e_star', 'public\.f_star',
+ 'public\.renamecolumnanother', 'public\.renamecolumnchild',
+ 'public\.test_type_diff2_c1', 'public\.test_type_diff2_c2',
+ 'public\.test_type_diff_c')
+ {
+ $dump =~ s/^COPY\s$table\s\(.+?^\\\.$//sm;
+ }
+
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
+ return $dump;
+}
+
+=pod
+
+=back
+
+=cut
+
+1;
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
index 58e30f15f9d..492ca571ff8 100644
--- a/src/test/perl/meson.build
+++ b/src/test/perl/meson.build
@@ -14,4 +14,5 @@ install_data(
'PostgreSQL/Test/Cluster.pm',
'PostgreSQL/Test/BackgroundPsql.pm',
'PostgreSQL/Test/AdjustUpgrade.pm',
+ 'PostgreSQL/Test/AdjustDump.pm',
install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
base-commit: 3691edfab97187789b8a1cbb9dce4acf0ecd8f5a
--
2.34.1
[text/x-patch] 0003-set-lc_monetary-explicitly-at-initdb-time-20250313.patch (1.2K, ../../CAExHW5tOhQi2Fyf5My-YK3uzP8QwVJZQDfC3o-vvAxUUG-CNhg@mail.gmail.com/3-0003-set-lc_monetary-explicitly-at-initdb-time-20250313.patch)
download | inline diff:
From 2f41b4371c0f0b8a3535537c4e4f9ddd1118d1ce Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Thu, 13 Mar 2025 16:17:57 +0530
Subject: [PATCH 3/3] set lc_monetary explicitly at initdb time
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 65f4c7d4f2b..51ba79c8589 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -134,6 +134,7 @@ my $original_enc_name;
my $original_provider;
my $original_datcollate = "C";
my $original_datctype = "C";
+my $original_datmonetary = "C";
my $original_datlocale;
if ($oldnode->pg_version >= '17devel')
@@ -163,6 +164,7 @@ my @initdb_params = @custom_opts;
push @initdb_params, ('--encoding', $original_enc_name);
push @initdb_params, ('--lc-collate', $original_datcollate);
push @initdb_params, ('--lc-ctype', $original_datctype);
+push @initdb_params, ('--lc-monetary', $original_datmonetary);
# add --locale-provider, if supported
my %provider_name = ('b' => 'builtin', 'i' => 'icu', 'c' => 'libc');
--
2.34.1
[text/x-patch] 0002-Do-not-dump-statistics-in-the-file-dumped-f-20250313.patch (2.1K, ../../CAExHW5tOhQi2Fyf5My-YK3uzP8QwVJZQDfC3o-vvAxUUG-CNhg@mail.gmail.com/4-0002-Do-not-dump-statistics-in-the-file-dumped-f-20250313.patch)
download | inline diff:
From 75f2b869764d9db3ac5c548636ed5c2be8b47b36 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Tue, 25 Feb 2025 11:42:51 +0530
Subject: [PATCH 2/3] Do not dump statistics in the file dumped for comparison
The dumped and restored statistics of a materialized view may differ as
reported in [1]. Hence do not dump the statistics to avoid differences
in the dump output from the original and restored database.
[1] https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Ashutosh Bapat
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index bd8313cee6f..65f4c7d4f2b 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -641,15 +641,15 @@ sub get_dump_for_comparison
my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
my $dump_adjusted = "${dumpfile}_adjusted";
- # Usually we avoid comparing statistics in our tests since it is flaky by
- # nature. However, if statistics is dumped and restored it is expected to be
- # restored as it is i.e. the statistics from the original database and that
- # from the restored database should match. We turn off autovacuum on the
- # source and the target database to avoid any statistics update during
- # restore operation. Hence we do not exclude statistics from dump.
+ # If statistics is dumped and restored it is expected to be restored as it
+ # is i.e. the statistics from the original database and that from the
+ # restored database should match. We turn off autovacuum on the source and
+ # the target database to avoid any statistics update during restore
+ # operation. But as of now, there are cases when statistics is not being
+ # restored faithfully. Hence for now do not dump statistics.
$node->command_ok(
[
- 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ 'pg_dump', '--no-sync', '--no-statistics', '-d', $node->connstr($db), '-f',
$dumpfile
],
'dump for comparison succeeded');
--
2.34.1
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-13 14:15 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-13 14:15 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Wed, Mar 12, 2025 at 04:41:15PM +0100, Jakub Wartak wrote:
> On Mon, Mar 10, 2025 at 11:14 AM Bertrand Drouvot
> <[email protected]> wrote:
>
> > Thanks for the new version!
>
> v10 is attached with most fixes after review and one new thing
> introduced: pg_numa_available() for run-time decision inside tests
> which was needed after simplifying code a little bit as you wanted.
> I've also fixed -Dlibnuma=disabled as it was not properly implemented.
> There are couple open items (minor/decision things), but most is fixed
> or answered:
Thanks for the new version!
> > === 2
> >
> > +else
> > + as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
> > +fi
> >
> > Maybe we should add the same test (checking for numa.h) for meson?
>
> TBH, I have no idea, libnuma.h may exist but it may not link e.g. when
> cross-compiling 32-bit on 64-bit. Or is this more about keeping sync
> between meson and autoconf?
Yeah, idea was to have both in sync.
> > === 8
> >
> > # create extension pg_buffercache;
> > ERROR: could not load library "/home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so": /home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so: undefined symbol: pg_numa_query_pages
> > CONTEXT: SQL statement "CREATE FUNCTION pg_buffercache_pages()
> > RETURNS SETOF RECORD
> > AS '$libdir/pg_buffercache', 'pg_buffercache_pages'
> > LANGUAGE C PARALLEL SAFE"
> > extension script file "pg_buffercache--1.2.sql", near line 7
> >
> > While that's ok if 0003 is applied. I think that each individual patch should
> > "fully" work individually.
>
> STILL OPEN QUESTION: Not sure I understand: you need to have 0001 +
> 0002 or 0001 + 0003, but here 0002 is complaining about lack of
> pg_numa_query_pages() which is part of 0001 (?) Should I merge those
> patches or keep them separate?
Applying 0001 + 0002 only does produce the issue above. I think that each
sub-patch once applied should pass make check-world, that means:
0001 : should pass
0001 + 0002 : should pass
0001 + 0002 + 0003 : should pass
> > Also I need to think more about how firstUseInBackend is used, for example:
> >
> > ==== 17.1
> >
> > would it be better to define it as a file scope variable? (at least that
> > would avoid to pass it as an extra parameter in some functions).
>
> I have no strong opinion on this, but I have one doubt (for future):
> isn't creating global variables making life harder for upcoming
> multithreading guys ?
That would be "just" one more. I think it's better to use it to avoid the "current"
code using "useless" function parameters.
> > === 17.2
> >
> > what happens if firstUseInBackend is set to false and later on the pages
> > are moved to different NUMA nodes. Then pg_buffercache_numa_pages() is
> > called again by a backend that already had set firstUseInBackend to false,
> > would that provide accurate information?
>
> It is still the correct result. That "touching" (paging-in) is only
> necessary probably to properly resolve PTEs as the fork() does not
> seem to carry them over from parent:
>
> So the above clearly shows that initial touching of shm is required,
> but just once and it stays valid afterwards.
Great, thanks for the demo and the explanation!
> > === 19
> >
>
> Can you please take a look again on this
Sure, will do.
> > === 23
> >
> > + if (s >= 0)
> > + zones[s]++;
> >
> > should we also check that s is below a limit?
>
> STILL OPEN QUESTION: I'm not sure it would give us value to report
> e.g. -2 on per shm entry/per numa node entry, would it? If it would we
> could somehow overallocate that array and remember negative ones too.
I meant to say, ensure that it is below the max node number.
> > === 24
> >
> > Regarding how we make use of pg_numa_get_max_node(), are we sure there is
> > no possible holes? I mean could a system have node 0,1 and 3 but not 2?
>
> I have never seen a hole in numbering as it is established during
> boot, and the only way that could get close to adjusting it could be
> making processor books (CPU and RAM together) offline.
Yeah probably.
> Even with all of that, that still
> shouldn't cause issues for this code I think, because
> `numa_max_node()` says it `returns the >> highest node number <<<
> available on the current system.`
Yeah, I agree, thanks for clearing my doubts on it.
> > Also I don't think I'm a Co-author, I think I'm just a reviewer (even if I
> > did a little in 0001 though)
>
> That was an attempt to say "Thank You",
Thanks a lot! OTOH, I don't want to get credit for something that I did not do ;-)
I'll have a look at v11 soon.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-13 15:38 Bertrand Drouvot <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-13 15:38 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Thu, Mar 13, 2025 at 02:15:14PM +0000, Bertrand Drouvot wrote:
> > > === 19
> > >
> >
> > Can you please take a look again on this
>
> Sure, will do.
> I'll have a look at v11 soon.
About 0001:
=== 1
git am produces:
.git/rebase-apply/patch:378: new blank line at EOF.
+
.git/rebase-apply/patch:411: new blank line at EOF.
+
warning: 2 lines add whitespace errors.
=== 2
+ Returns true if a <acronym>NUMA</acronym> support is available.
What about "Returns true if the server has been compiled with NUMA support"?
=== 3
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ if(pg_numa_init() == -1)
+ PG_RETURN_BOOL(false);
+ PG_RETURN_BOOL(true);
+}
What about PG_RETURN_BOOL(pg_numa_init() != -1)?
Also I wonder if pg_numa.c would not be a better place for it.
=== 4
+{ oid => '5102', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
Not sure that 5102 is a good choice.
src/include/catalog/unused_oids is telling me:
Best practice is to start with a random choice in the range 8000-9999.
Suggested random unused OID: 9685 (23 consecutive OID(s) available starting here)
So maybe use 9685 instead?
=== 5
@@ -12477,3 +12481,4 @@
prosrc => 'gist_stratnum_common' },
]
+
garbage?
=== 6
Run pgindent as it looks like it's finding some things to do in src/backend/storage/ipc/shmem.c
and src/port/pg_numa.c.
=== 7
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
I think that's more usual to:
Size
pg_numa_get_pagesize(void)
{
Size os_page_size = sysconf(_SC_PAGESIZE);
if (huge_pages_status == HUGE_PAGES_ON)
GetHugePageSize(&os_page_size, NULL);
return os_page_size;
}
I think that makes sense to check huge_pages_status as you did.
=== 8
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
I wonder if that would make sense to add a comment mentioning
github.com/numactl/numactl/blob/master/libnuma.c here.
I still need to look at 0002 and 0003.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-14 10:05 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-14 10:05 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Thu, Mar 13, 2025 at 3:15 PM Bertrand Drouvot
<[email protected]> wrote:
Hi,
Thank you very much for the review! I'm answering to both reviews in
one go and the results is attached v12, seems it all should be solved
now:
> > > === 2
> > >
> > > +else
> > > + as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
> > > +fi
> > >
> > > Maybe we should add the same test (checking for numa.h) for meson?
> >
> > TBH, I have no idea, libnuma.h may exist but it may not link e.g. when
> > cross-compiling 32-bit on 64-bit. Or is this more about keeping sync
> > between meson and autoconf?
>
> Yeah, idea was to have both in sync.
Added.
> > > === 8
> > >
> > > # create extension pg_buffercache;
> > > ERROR: could not load library "/home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so": /home/postgres/postgresql/pg_installed/pg18/lib/pg_buffercache.so: undefined symbol: pg_numa_query_pages
> > > CONTEXT: SQL statement "CREATE FUNCTION pg_buffercache_pages()
> > > RETURNS SETOF RECORD
> > > AS '$libdir/pg_buffercache', 'pg_buffercache_pages'
> > > LANGUAGE C PARALLEL SAFE"
> > > extension script file "pg_buffercache--1.2.sql", near line 7
> > >
> > > While that's ok if 0003 is applied. I think that each individual patch should
> > > "fully" work individually.
> >
> > STILL OPEN QUESTION: Not sure I understand: you need to have 0001 +
> > 0002 or 0001 + 0003, but here 0002 is complaining about lack of
> > pg_numa_query_pages() which is part of 0001 (?) Should I merge those
> > patches or keep them separate?
>
> Applying 0001 + 0002 only does produce the issue above. I think that each
> sub-patch once applied should pass make check-world, that means:
>
> 0001 : should pass
> 0001 + 0002 : should pass
> 0001 + 0002 + 0003 : should pass
OK, I've retested v11 for all three of them. It worked fine (I think
in v10 I've moved one function to 0001, but pg_numa_query_pages() as
per error message above was always in the 0001).
> > > Also I need to think more about how firstUseInBackend is used, for example:
> > >
> > > ==== 17.1
> > >
> > > would it be better to define it as a file scope variable? (at least that
> > > would avoid to pass it as an extra parameter in some functions).
> >
> > I have no strong opinion on this, but I have one doubt (for future):
> > isn't creating global variables making life harder for upcoming
> > multithreading guys ?
>
> That would be "just" one more. I think it's better to use it to avoid the "current"
> code using "useless" function parameters.
Done.
> > > === 23
> > >
> > > + if (s >= 0)
> > > + zones[s]++;
> > >
> > > should we also check that s is below a limit?
> >
> > STILL OPEN QUESTION: I'm not sure it would give us value to report
> > e.g. -2 on per shm entry/per numa node entry, would it? If it would we
> > could somehow overallocate that array and remember negative ones too.
>
> I meant to say, ensure that it is below the max node number.
Done, but I doubt the kernel would return a value higher than
numa_max_nodes(), but who knows. Additional defense for this array is
now there.
> SECOND REVIEW//v11-0001 review
>
> === 1
>
> git am produces:
>
> .git/rebase-apply/patch:378: new blank line at EOF.
> +
> .git/rebase-apply/patch:411: new blank line at EOF.
> +
> warning: 2 lines add whitespace errors.
Should be gone, but in at least one case (0003/numa.out) we need to
have empty EOF because otherwise expected tests don't pass (even if
numa.sql doesnt have EOF in numa.sql)
> === 2
>
> + Returns true if a <acronym>NUMA</acronym> support is available.
>
> What about "Returns true if the server has been compiled with NUMA support"?
Done.
> === 3
>
> +Datum
> +pg_numa_available(PG_FUNCTION_ARGS)
> +{
> + if(pg_numa_init() == -1)
> + PG_RETURN_BOOL(false);
> + PG_RETURN_BOOL(true);
> +}
>
> What about PG_RETURN_BOOL(pg_numa_init() != -1)?
>
> Also I wonder if pg_numa.c would not be a better place for it.
Both done.
> === 4
>
> +{ oid => '5102', descr => 'Is NUMA compilation available?',
> + proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
> + proargtypes => '', prosrc => 'pg_numa_available' },
> +
>
> Not sure that 5102 is a good choice.
>
> src/include/catalog/unused_oids is telling me:
>
> Best practice is to start with a random choice in the range 8000-9999.
> Suggested random unused OID: 9685 (23 consecutive OID(s) available starting here)
>
> So maybe use 9685 instead?
It's because i've got 5101 there earlier (for pg_shm_numa_allocs
view), but I've aligned both (5102@0001 and 5101@0003) to 968[56].
> === 5
>
> @@ -12477,3 +12481,4 @@
> prosrc => 'gist_stratnum_common' },
>
> ]
> +
>
> garbage?
Yea, fixed.
> === 6
>
> Run pgindent as it looks like it's finding some things to do in src/backend/storage/ipc/shmem.c
> and src/port/pg_numa.c.
Fixed.
> === 7
>
> +Size
> +pg_numa_get_pagesize(void)
> +{
> + Size os_page_size = sysconf(_SC_PAGESIZE);
> + if (huge_pages_status == HUGE_PAGES_ON)
> + GetHugePageSize(&os_page_size, NULL);
> + return os_page_size;
> +}
>
> I think that's more usual to:
>
> Size
> pg_numa_get_pagesize(void)
> {
> Size os_page_size = sysconf(_SC_PAGESIZE);
>
> if (huge_pages_status == HUGE_PAGES_ON)
> GetHugePageSize(&os_page_size, NULL);
>
> return os_page_size;
> }
>
> I think that makes sense to check huge_pages_status as you did.
Done.
> === 8
>
> +extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
> +extern void numa_error(char *where);
>
> I wonder if that would make sense to add a comment mentioning
> github.com/numactl/numactl/blob/master/libnuma.c here.
Sure thing, added.
-J.
Attachments:
[application/octet-stream] v12-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch (15.9K, ../../CAKZiRmyXPfHMZngT29LCbG2bvkemYBWcKpiqM9JhE4CaiTRjvw@mail.gmail.com/2-v12-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch)
download | inline diff:
From 6068f8b5c4a0eb29d684e8865221801a0c682543 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v12 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 78 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 9 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 266 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..5164083131a 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,79 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> view shows NUMA nodes
+ assigned allocations made from the server's main shared memory segment.
+ This includes both memory allocated by <productname>PostgreSQL</productname>
+ itself and memory allocated by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of NUMA node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf5..cc014a62dc2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..9331a5760f6 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port//pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -89,6 +90,8 @@ slock_t *ShmemLock; /* spinlock for shared memory and LWLock
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+/* To get reliable results for NUMA inquiry we need to "touch pages" once */
+static bool firstUseInBackend = true;
/*
* InitShmemAccess() --- set up basic pointers to shared memory.
@@ -568,3 +571,125 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count,
+ max_zones;
+ Size *zones;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+ max_zones = pg_numa_get_max_node();
+ zones = palloc(sizeof(Size) * (max_zones + 1));
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ memset(zones, 0, sizeof(Size) * (max_zones + 1));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ /* Ensure we are adding only valid index to the array */
+ if (s >= 0 && s <= max_zones)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= max_zones; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 85902903653..55ff305a713 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8496,6 +8496,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '9686', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 954f549555e..d9d62470cdc 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b2..b63c6e0f744 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 37b6d21e1f9..c07a4c7633a 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..fddb21a260a
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,9 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index b81694c24f2..f93d4829702 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
[application/octet-stream] v12-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (27.8K, ../../CAKZiRmyXPfHMZngT29LCbG2bvkemYBWcKpiqM9JhE4CaiTRjvw@mail.gmail.com/3-v12-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From 78607bd84be0b9a448491bcb0a7d3c6b8a042d1c Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v12 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 ++
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 463 +++++++++++++-----
.../sql/pg_buffercache_numa.sql | 20 +
doc/src/sgml/pgbuffercache.sgml | 64 ++-
9 files changed, 493 insertions(+), 134 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..52f63aa258c
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..b27add81f0a 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,12 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +43,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -61,84 +62,255 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * To get reliable results we need to "touch pages" once, see
+ * comments nearby pg_buffercache_numa_prepare_ptrs().
+ */
+static bool firstUseInBackend = true;
+
+/*
+ * Helper routine to map Buffers into addresses that can be
+ * later consumed by pg_numa_query_pages()
+ *
+ * Many buffers can point to the same page (in case of
+ * BLCKSZ < 4kB), but we want to also query just first
+ * address.
+ *
+ * In order to get reliable results we also need to touch
+ * memory pages, so that inquiry about NUMA zone doesn't
+ * return -2.
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk, Size os_page_size,
+ void **os_page_ptrs)
+{
+ size_t blk2page = (size_t)(buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers count start really from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) + (os_page_size * j);
+
+ /* We just need to do it only once in backend */
+ if (firstUseInBackend)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ }
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages.
+ *
+ * We need fcinfo here and we pass it here with PG_FUNCTION_ARGS
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static void
+pg_buffercache_build_tuple(int i, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(i);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[i].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[i].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[i].blocknum = bufHdr->tag.blockNum;
+ fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[i].isdirty = true;
+ else
+ fctx->record[i].isdirty = false;
+
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[i].isvalid = true;
+ else
+ fctx->record[i].isvalid = false;
+
+ fctx->record[i].numa_zone_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_(numa_)pages
+ */
+static Datum
+get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[i].bufferid);
+ nulls[0] = false;
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[i].blocknum == InvalidBlockNumber ||
+ fctx->record[i].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[i].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[i].usagecount);
+ nulls[7] = false;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ nulls[9] = false;
+ }
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+/*
+ * When updating this routine please sync it with below one:
+ * pg_buffercache_numa_pages()
+ */
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +321,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_build_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +331,117 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ int os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * This is for gathering some NUMA statistics. We might be using
+ * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
+ * allocated in various different OS memory pages sizes, so first
+ * we need to understand the OS memory page size before calling
+ * move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
+ os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(int) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably
+ * have bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstUseInBackend)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+
+ /*
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ pg_buffercache_build_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs);
}
- else
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Technically we can get errors too here and pass that to
+ * user. Also we could somehow report single DB block spanning
+ * more than one NUMA zone, but it should be rare.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
}
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstUseInBackend = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..2225b879f58
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,20 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..75978a6eaed 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,13 @@
convenient use.
</para>
+ <para>
+ The similiar <function>pg_buffercache_numa_pages()</function> is a slower
+ variant of the above, but also can provide NUMA node ID for shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +209,59 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are almost identical to the previous
+ <structname>pg_buffercache</structname> view, but this one includes one additional
+ column numa_zone_id as defined in <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ ID (number) of the NUMA node for this particular buffer. NULL if the shared buffer
+ has not been used yet.On systems without NUMA this usually returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ This is clone version of the original pg_buffercache view, however it provides
+ additional <structfield>numa_zone_id</structfield> column. Fetching this
+ information from OS is costly and might take much longer and querying it is not
+ recommended by automated or monitoring systems.
+ </para>
+
+ <para>
+ As NUMA node ID inquiry for each page requires memory pages to be paged-in, first
+ execution of this function can take long time especially on systems with bigint
+ shared_buffers and without huge_pages enabled.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
[application/octet-stream] v12-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (18.8K, ../../CAKZiRmyXPfHMZngT29LCbG2bvkemYBWcKpiqM9JhE4CaiTRjvw@mail.gmail.com/4-v12-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From 0754817d5081f18c9d7edb9c95d412fb4dba7552 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v12 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper. Other platforms can be added later.
This also adds function pg_numa_available() that can be used to check if
the server was linked with NUMA support.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway on i386 it is very memory
limited platform even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 +-
configure | 87 ++++++++++++++
configure.ac | 13 +++
doc/src/sgml/func.sgml | 13 +++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 23 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 46 ++++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 168 ++++++++++++++++++++++++++++
17 files changed, 397 insertions(+), 5 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 5849cbb839a..7010dff7aef 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -519,6 +522,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -835,8 +839,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 1c3810e1a04..113588defdd 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25078,6 +25078,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if the server has been compiled with <acronym>NUMA</acronym> support.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b9..9f56205a1d7 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 13c13748e5d..4106c4b13f5 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,27 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ # via pkg-config
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if not cc.has_header('numa.h', dependencies: libnuma, required: libnumaopt)
+ libnuma = not_found_dep
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3189,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3845,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 508970680d1..4255b47c29a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 890822eaf79..85902903653 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8492,6 +8492,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '9685', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..986152e0942
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,46 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+#include "postgres.h"
+#include "fmgr.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+extern PGDLLIMPORT Datum pg_numa_available(PG_FUNCTION_ARGS);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..7d905ef31f5
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "fmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/*
+ * XXX: not really tested as there is no way to trigger this in our
+ * current usage of libnuma.
+ *
+ * The libnuma built-in code can be seen here:
+ * https://github.com/numactl/numactl/blob/master/libnuma.c
+ *
+ */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here.
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
+
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_BOOL(pg_numa_init() != -1);
+}
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-14 12:08 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-14 12:08 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Fri, Mar 14, 2025 at 11:05:28AM +0100, Jakub Wartak wrote:
> On Thu, Mar 13, 2025 at 3:15 PM Bertrand Drouvot
> <[email protected]> wrote:
>
> Hi,
>
> Thank you very much for the review! I'm answering to both reviews in
> one go and the results is attached v12, seems it all should be solved
> now:
Thanks for v12!
I'll review 0001 and 0003 later, but want to share what I've done for 0002.
I did prepare a patch file (attached as .txt to not disturb the cfbot) to apply
on top of v11 0002 (I just rebased it a bit so that it now applies on top of
v12 0002).
The main changes are:
=== 1
Some doc wording changes.
=== 2
Some comments, pgindent and friends changes.
=== 3
relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
- pinning_backends int4, numa_zone_id int4);
+ pinning_backends int4, zone_id int4);
I changed numa_zone_id to zone_id as we are already in "numa" related functions
and/or views.
=== 4
- CHECK_FOR_INTERRUPTS();
}
+
+ CHECK_FOR_INTERRUPTS();
I think that it's better to put the CFI outside of the if condition, so that it
can be called for every page.
=== 5
-pg_buffercache_build_tuple(int i, BufferCachePagesContext *fctx)
+pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
for clarity.
=== 6
-get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
for clarity.
=== 7
- int os_page_count = 0;
+ uint64 os_page_count = 0;
I think that's better.
=== 8
But then, here, we need to change its format to %lu and and to cast to (unsigned long)
to make the CI CompilerWarnings happy.
That's fine because we don't provide NUMA support for 32 bits anyway.
- elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
- os_page_count, os_page_size, pages_per_blk);
+ elog(DEBUG1, "NUMA: os_page_count=%lu os_page_size=%zu pages_per_blk=%.2f",
+ (unsigned long) os_page_count, os_page_size, pages_per_blk);
=== 9
-static bool firstUseInBackend = true;
+static bool firstNumaTouch = true;
Looks better to me but still not 100% convinced by the name.
=== 10
static BufferCachePagesContext *
-pg_buffercache_init_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
+pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
as PG_FUNCTION_ARGS is usually used for fmgr-compatible function and there is
a lof of examples in the code that make use of "FunctionCallInfo" for non
fmgr-compatible function.
and also:
=== 11
I don't like the fact that we iterate 2 times over NBuffers in
pg_buffercache_numa_pages().
But I'm having having hard time finding a better approach given the fact that
pg_numa_query_pages() needs all the pointers "prepared" before it can be called...
Those 2 loops are probably the best approach, unless someone has a better idea.
=== 12
Upthread you asked "Can you please take a look again on this, is this up to the
project standards?"
Was the question about using pg_buffercache_numa_prepare_ptrs() as an inlined wrapper?
What do you think? The comments, doc and code changes are just proposals and are
fully open to discussion.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
[text/x-diff] on_top_of_v12_0002.patch (16.0K, ../../[email protected]/2-on_top_of_v12_0002.patch)
download | inline diff:
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
index 52f63aa258c..42a693aa4d4 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -28,7 +28,7 @@ CREATE OR REPLACE VIEW pg_buffercache_numa AS
SELECT P.* FROM pg_buffercache_numa_pages() AS P
(bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
- pinning_backends int4, numa_zone_id int4);
+ pinning_backends int4, zone_id int4);
-- Don't want these to be available to public.
REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index b27add81f0a..c5cfa32fa07 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -67,56 +67,59 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-/*
- * To get reliable results we need to "touch pages" once, see
- * comments nearby pg_buffercache_numa_prepare_ptrs().
- */
-static bool firstUseInBackend = true;
+/* Only need to touch memory once per backend process lifetime */
+static bool firstNumaTouch = true;
/*
- * Helper routine to map Buffers into addresses that can be
- * later consumed by pg_numa_query_pages()
+ * Helper routine to map Buffers into addresses that is used by
+ * pg_numa_query_pages().
*
- * Many buffers can point to the same page (in case of
- * BLCKSZ < 4kB), but we want to also query just first
- * address.
+ * When database block size (BLCKSZ) is smaller than the OS page size (4kB),
+ * multiple database buffers will map to the same OS memory page. In this case,
+ * we only need to query the NUMA zone for the first memory address of each
+ * unique OS page rather than for every buffer.
*
- * In order to get reliable results we also need to touch
- * memory pages, so that inquiry about NUMA zone doesn't
- * return -2.
+ * In order to get reliable results we also need to touch memory pages, so that
+ * inquiry about NUMA zone doesn't return -2 (which indicates unmapped/unallocated
+ * pages)
*/
static inline void
-pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk, Size os_page_size,
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk,
+ Size os_page_size,
void **os_page_ptrs)
{
- size_t blk2page = (size_t)(buffer_id * pages_per_blk);
+ size_t blk2page = (size_t) (buffer_id * pages_per_blk);
for (size_t j = 0; j < pages_per_blk; j++)
{
- size_t blk2pageoff = blk2page + j;
+ size_t blk2pageoff = blk2page + j;
+
if (os_page_ptrs[blk2pageoff] == 0)
{
volatile uint64 touch pg_attribute_unused();
- /* NBuffers count start really from 1 */
- os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) + (os_page_size * j);
+ /* NBuffers starts from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) +
+ (os_page_size * j);
- /* We just need to do it only once in backend */
- if (firstUseInBackend)
+ /* Only need to touch memory once per backend process lifetime */
+ if (firstNumaTouch)
pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
- CHECK_FOR_INTERRUPTS();
}
+
+ CHECK_FOR_INTERRUPTS();
}
}
/*
- * Helper routine for pg_buffercache_(numa_)pages.
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
*
- * We need fcinfo here and we pass it here with PG_FUNCTION_ARGS
+ * This is almost identical to pg_buffercache_numa_pages(), but this one performs
+ * memory mapping inquiries to display NUMA zone information for each buffer.
*/
static BufferCachePagesContext *
-pg_buffercache_init_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
+pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
{
BufferCachePagesContext *fctx; /* User function context. */
MemoryContext oldcontext;
@@ -191,64 +194,68 @@ pg_buffercache_init_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
}
/*
- * Helper routine for pg_buffercache_(numa_)pages
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Build buffer cache information for a single buffer.
*/
static void
-pg_buffercache_build_tuple(int i, BufferCachePagesContext *fctx)
+pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
{
BufferDesc *bufHdr;
uint32 buf_state;
- bufHdr = GetBufferDescriptor(i);
+ bufHdr = GetBufferDescriptor(record_id);
/* Lock each buffer header before inspecting. */
buf_state = LockBufHdr(bufHdr);
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+ fctx->record[record_id].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[record_id].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[record_id].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[record_id].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[record_id].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[record_id].blocknum = bufHdr->tag.blockNum;
+ fctx->record[record_id].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[record_id].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
+ fctx->record[record_id].isdirty = true;
else
- fctx->record[i].isdirty = false;
+ fctx->record[record_id].isdirty = false;
/*
* Note if the buffer is valid, and has storage created
*/
if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
+ fctx->record[record_id].isvalid = true;
else
- fctx->record[i].isvalid = false;
+ fctx->record[record_id].isvalid = false;
- fctx->record[i].numa_zone_id = -1;
+ fctx->record[record_id].numa_zone_id = -1;
UnlockBufHdr(bufHdr, buf_state);
}
/*
- * Helper routine for pg_buffercache_(numa_)pages
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Format and return a tuple for a single buffer cache entry.
*/
static Datum
-get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
+get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
{
Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
HeapTuple tuple;
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
+ values[0] = Int32GetDatum(fctx->record[record_id].bufferid);
nulls[0] = false;
/*
* Set all fields except the bufferid to null if the buffer is unused or
* not valid.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ if (fctx->record[record_id].blocknum == InvalidBlockNumber ||
+ fctx->record[record_id].isvalid == false)
{
nulls[1] = true;
nulls[2] = true;
@@ -266,27 +273,27 @@ get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
}
else
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
+ values[1] = ObjectIdGetDatum(fctx->record[record_id].relfilenumber);
nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
+ values[2] = ObjectIdGetDatum(fctx->record[record_id].reltablespace);
nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
+ values[3] = ObjectIdGetDatum(fctx->record[record_id].reldatabase);
nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
+ values[4] = ObjectIdGetDatum(fctx->record[record_id].forknum);
nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
+ values[5] = Int64GetDatum((int64) fctx->record[record_id].blocknum);
nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
+ values[6] = BoolGetDatum(fctx->record[record_id].isdirty);
nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
+ values[7] = Int16GetDatum(fctx->record[record_id].usagecount);
nulls[7] = false;
/*
* unused for v1.0 callers, but the array is always long enough
*/
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
+ values[8] = Int32GetDatum(fctx->record[record_id].pinning_backends);
nulls[8] = false;
- values[9] = Int32GetDatum(fctx->record[i].numa_zone_id);
+ values[9] = Int32GetDatum(fctx->record[record_id].numa_zone_id);
nulls[9] = false;
}
@@ -295,10 +302,6 @@ get_buffercache_tuple(int i, BufferCachePagesContext *fctx)
return HeapTupleGetDatum(tuple);
}
-/*
- * When updating this routine please sync it with below one:
- * pg_buffercache_numa_pages()
- */
Datum
pg_buffercache_pages(PG_FUNCTION_ARGS)
{
@@ -359,39 +362,46 @@ pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
Size os_page_size = 0;
void **os_page_ptrs = NULL;
int *os_pages_status = NULL;
- int os_page_count = 0;
+ uint64 os_page_count = 0;
float pages_per_blk = 0;
funcctx = SRF_FIRSTCALL_INIT();
+
if (pg_numa_init() == -1)
- elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+
fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
- * This is for gathering some NUMA statistics. We might be using
- * various DB block sizes (4kB, 8kB , .. 32kB) that end up being
- * allocated in various different OS memory pages sizes, so first
- * we need to understand the OS memory page size before calling
- * move_pages()
- */
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used,
+ * while the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: - Determine the OS
+ * memory page size - Calculate how many OS pages are used by all
+ * buffer blocks - Calculate how many OS pages are contained within
+ * each database block
+ *
+ * This information is needed before calling move_pages() for NUMA
+ * zone inquiry.
+ */
os_page_size = pg_numa_get_pagesize();
os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
pages_per_blk = (float) BLCKSZ / os_page_size;
- elog(DEBUG1, "NUMA: os_page_count=%d os_page_size=%zu pages_per_blk=%f",
- os_page_count, os_page_size, pages_per_blk);
+ elog(DEBUG1, "NUMA: os_page_count=%lu os_page_size=%zu pages_per_blk=%.2f",
+ (unsigned long) os_page_count, os_page_size, pages_per_blk);
os_page_ptrs = palloc(sizeof(void *) * os_page_count);
- os_pages_status = palloc(sizeof(int) * os_page_count);
+ os_pages_status = palloc(sizeof(uint64) * os_page_count);
memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
/*
- * If we ever get 0xff back from kernel inquiry, then we probably
- * have bug in our buffers to OS page mapping code here
- */
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
- if (firstUseInBackend)
+ if (firstNumaTouch)
elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
/*
@@ -405,7 +415,8 @@ pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
for (i = 0; i < NBuffers; i++)
{
pg_buffercache_build_tuple(i, fctx);
- pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size, os_page_ptrs);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size,
+ os_page_ptrs);
}
if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
@@ -416,10 +427,15 @@ pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
int blk2page = (int) i * pages_per_blk;
/*
- * Technically we can get errors too here and pass that to
- * user. Also we could somehow report single DB block spanning
- * more than one NUMA zone, but it should be rare.
- */
+ * Set the NUMA zone ID for this buffer based on the first OS page
+ * it maps to.
+ *
+ * Note: We could check for errors in os_pages_status and report
+ * them. Also, a single DB block might span multiple NUMA zones if
+ * it crosses OS pages on zone boundaries, but we only record the
+ * zone of the first page. This is a simplification but should be
+ * sufficient for most analyses.
+ */
fctx->record[i].numa_zone_id = os_pages_status[blk2page];
}
}
@@ -439,7 +455,7 @@ pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
}
else
{
- firstUseInBackend = false;
+ firstNumaTouch = false;
SRF_RETURN_DONE(funcctx);
}
}
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 75978a6eaed..4b49bb2974a 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -45,8 +45,9 @@
</para>
<para>
- The similiar <function>pg_buffercache_numa_pages()</function> is a slower
- variant of the above, but also can provide NUMA node ID for shared buffer entry.
+ The <function>pg_buffercache_numa_pages()</function> provides the same information
+ as <function>pg_buffercache_pages()</function> but is slower because it also
+ provides the <acronym>NUMA</acronym> node ID per shared buffer entry.
The <structname>pg_buffercache_numa</structname> view wraps the function for
convenient use.
</para>
@@ -213,13 +214,14 @@
<title>The <structname>pg_buffercache_numa</structname> View</title>
<para>
- The definitions of the columns exposed are almost identical to the previous
- <structname>pg_buffercache</structname> view, but this one includes one additional
- column numa_zone_id as defined in <xref linkend="pgbuffercache-numa-columns"/>.
+ The definitions of the columns exposed are identical to the
+ <structname>pg_buffercache</structname> view, except that this one includes
+ one additional <structfield>zone_id</structfield> column as defined in
+ <xref linkend="pgbuffercache-numa-columns"/>.
</para>
<table id="pgbuffercache-numa-columns">
- <title><structname>pg_buffercache_numa</structname> Columns</title>
+ <title><structname>pg_buffercache_numa</structname> Extra column</title>
<tgroup cols="1">
<thead>
<row>
@@ -235,11 +237,12 @@
<tbody>
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>numa_zone_id</structfield> <type>integer</type>
+ <structfield>zone_id</structfield> <type>integer</type>
</para>
<para>
- ID (number) of the NUMA node for this particular buffer. NULL if the shared buffer
- has not been used yet.On systems without NUMA this usually returns 0.
+ <acronym>NUMA</acronym> node ID. NULL if the shared buffer
+ has not been used yet. On systems without <acronym>NUMA</acronym> support
+ this returns 0.
</para></entry>
</row>
@@ -248,16 +251,10 @@
</table>
<para>
- This is clone version of the original pg_buffercache view, however it provides
- additional <structfield>numa_zone_id</structfield> column. Fetching this
- information from OS is costly and might take much longer and querying it is not
- recommended by automated or monitoring systems.
- </para>
-
- <para>
- As NUMA node ID inquiry for each page requires memory pages to be paged-in, first
- execution of this function can take long time especially on systems with bigint
- shared_buffers and without huge_pages enabled.
+ As <acronym>NUMA</acronym> node ID inquiry for each page requires memory pages
+ to be paged-in, the first execution of this function can take a noticeable
+ amount of time. In all the cases (first execution or not), retrieving this
+ information is costly and querying the view at a high frequency is not recommended.
</para>
</sect2>
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-17 07:28 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-17 07:28 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Fri, Mar 14, 2025 at 1:08 PM Bertrand Drouvot
<[email protected]> wrote:
> On Fri, Mar 14, 2025 at 11:05:28AM +0100, Jakub Wartak wrote:
> > On Thu, Mar 13, 2025 at 3:15 PM Bertrand Drouvot
> > <[email protected]> wrote:
> >
> > Hi,
> >
> > Thank you very much for the review! I'm answering to both reviews in
> > one go and the results is attached v12, seems it all should be solved
> > now:
>
> Thanks for v12!
>
> I'll review 0001 and 0003 later, but want to share what I've done for 0002.
>
> I did prepare a patch file (attached as .txt to not disturb the cfbot) to apply
> on top of v11 0002 (I just rebased it a bit so that it now applies on top of
> v12 0002).
Hey Bertrand,
all LGTM (good ideas), so here's v13 attached with applied all of that
(rebased, tested). BTW: I'm sending to make cfbot as it still tried to
apply that .patch (on my side it .patch, not .txt)
> === 9
>
> -static bool firstUseInBackend = true;
> +static bool firstNumaTouch = true;
>
> Looks better to me but still not 100% convinced by the name.
IMHO, Yes, it looks much better.
> === 10
>
> static BufferCachePagesContext *
> -pg_buffercache_init_entries(FuncCallContext *funcctx, PG_FUNCTION_ARGS)
> +pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
>
> as PG_FUNCTION_ARGS is usually used for fmgr-compatible function and there is
> a lof of examples in the code that make use of "FunctionCallInfo" for non
> fmgr-compatible function.
Cool, thanks.
> and also:
>
> === 11
>
> I don't like the fact that we iterate 2 times over NBuffers in
> pg_buffercache_numa_pages().
>
> But I'm having having hard time finding a better approach given the fact that
> pg_numa_query_pages() needs all the pointers "prepared" before it can be called...
>
> Those 2 loops are probably the best approach, unless someone has a better idea.
IMHO, it doesn't hurt and I've also not been able to think of any better idea.
> === 12
>
> Upthread you asked "Can you please take a look again on this, is this up to the
> project standards?"
>
> Was the question about using pg_buffercache_numa_prepare_ptrs() as an inlined wrapper?
Yes, this was for an earlier doubt regarding question "19" about
reviewing the code after removal of `query_numa` variable. This is the
same code for 2 loops, IMHO it is good now.
> What do you think? The comments, doc and code changes are just proposals and are
> fully open to discussion.
They are great, thank You!
-J.
Attachments:
[application/x-patch] v13-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (18.8K, ../../CAKZiRmxrnyGN9TAc_pBL5HCEgTG+NpK01DzaheTXB07zb4_80w@mail.gmail.com/2-v13-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From 1a55056446fff06e0441d8d05a9e84832dbdc821 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v13 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper. Other platforms can be added later.
This also adds function pg_numa_available() that can be used to check if
the server was linked with NUMA support.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway on i386 it is very memory
limited platform even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 +-
configure | 87 ++++++++++++++
configure.ac | 13 +++
doc/src/sgml/func.sgml | 13 +++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 23 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 46 ++++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 168 ++++++++++++++++++++++++++++
17 files changed, 397 insertions(+), 5 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 5849cbb839a..7010dff7aef 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -519,6 +522,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -835,8 +839,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 1c3810e1a04..113588defdd 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25078,6 +25078,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if the server has been compiled with <acronym>NUMA</acronym> support.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b9..9f56205a1d7 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 13c13748e5d..4106c4b13f5 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,27 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ # via pkg-config
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if not cc.has_header('numa.h', dependencies: libnuma, required: libnumaopt)
+ libnuma = not_found_dep
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3189,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3845,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 9c0b10ad4dc..c5e8ce06c97 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -563,7 +563,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 890822eaf79..85902903653 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8492,6 +8492,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '9685', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..986152e0942
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,46 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+#include "postgres.h"
+#include "fmgr.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+extern PGDLLIMPORT Datum pg_numa_available(PG_FUNCTION_ARGS);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..7d905ef31f5
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "fmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/*
+ * XXX: not really tested as there is no way to trigger this in our
+ * current usage of libnuma.
+ *
+ * The libnuma built-in code can be seen here:
+ * https://github.com/numactl/numactl/blob/master/libnuma.c
+ *
+ */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here.
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
+
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_BOOL(pg_numa_init() != -1);
+}
--
2.39.5
[application/x-patch] v13-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch (15.9K, ../../CAKZiRmxrnyGN9TAc_pBL5HCEgTG+NpK01DzaheTXB07zb4_80w@mail.gmail.com/3-v13-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-zones.patch)
download | inline diff:
From 3110606f68cc40e02b8ab4670c66089be4e2e305 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v13 3/3] Add pg_shmem_numa_allocations to show NUMA zones for
shared memory allocations
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 78 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 125 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 9 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 266 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..5164083131a 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,79 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> view shows NUMA nodes
+ assigned allocations made from the server's main shared memory segment.
+ This includes both memory allocated by <productname>PostgreSQL</productname>
+ itself and memory allocated by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_zone_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of NUMA node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf5..cc014a62dc2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..9331a5760f6 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port//pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -89,6 +90,8 @@ slock_t *ShmemLock; /* spinlock for shared memory and LWLock
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+/* To get reliable results for NUMA inquiry we need to "touch pages" once */
+static bool firstUseInBackend = true;
/*
* InitShmemAccess() --- set up basic pointers to shared memory.
@@ -568,3 +571,125 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA zones for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ int shm_total_page_count,
+ shm_ent_page_count,
+ max_zones;
+ Size *zones;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
+ return (Datum) 0;
+ }
+ max_zones = pg_numa_get_max_node();
+ zones = palloc(sizeof(Size) * (max_zones + 1));
+
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
+
+ if (firstUseInBackend)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstUseInBackend)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ memset(zones, 0, sizeof(Size) * (max_zones + 1));
+ /* Count number of NUMA zones used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ /* Ensure we are adding only valid index to the array */
+ if (s >= 0 && s <= max_zones)
+ zones[s]++;
+ }
+
+ for (i = 0; i <= max_zones; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(zones[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstUseInBackend = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 85902903653..55ff305a713 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8496,6 +8496,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '9686', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_zone_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 954f549555e..d9d62470cdc 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b2..b63c6e0f744 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_zone_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_zone_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 37b6d21e1f9..c07a4c7633a 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..fddb21a260a
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,9 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index b81694c24f2..f93d4829702 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
[application/x-patch] v13-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (28.6K, ../../CAKZiRmxrnyGN9TAc_pBL5HCEgTG+NpK01DzaheTXB07zb4_80w@mail.gmail.com/4-v13-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From 661c36d0a98e572ad0d3d47174f273f5fa3943c4 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v13 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA zone for indvidual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 +
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 479 +++++++++++++-----
.../sql/pg_buffercache_numa.sql | 20 +
doc/src/sgml/pgbuffercache.sgml | 61 ++-
9 files changed, 506 insertions(+), 134 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..42a693aa4d4
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, zone_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..c5cfa32fa07 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,12 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +43,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_zone_id;
} BufferCachePagesRec;
@@ -61,84 +62,258 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/* Only need to touch memory once per backend process lifetime */
+static bool firstNumaTouch = true;
+
+/*
+ * Helper routine to map Buffers into addresses that is used by
+ * pg_numa_query_pages().
+ *
+ * When database block size (BLCKSZ) is smaller than the OS page size (4kB),
+ * multiple database buffers will map to the same OS memory page. In this case,
+ * we only need to query the NUMA zone for the first memory address of each
+ * unique OS page rather than for every buffer.
+ *
+ * In order to get reliable results we also need to touch memory pages, so that
+ * inquiry about NUMA zone doesn't return -2 (which indicates unmapped/unallocated
+ * pages)
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk,
+ Size os_page_size,
+ void **os_page_ptrs)
+{
+ size_t blk2page = (size_t) (buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers starts from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) +
+ (os_page_size * j);
+
+ /* Only need to touch memory once per backend process lifetime */
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ }
+
+ CHECK_FOR_INTERRUPTS();
+ }
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * This is almost identical to pg_buffercache_numa_pages(), but this one performs
+ * memory mapping inquiries to display NUMA zone information for each buffer.
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_zone_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Build buffer cache information for a single buffer.
+ */
+static void
+pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(record_id);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[record_id].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[record_id].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[record_id].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[record_id].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[record_id].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[record_id].blocknum = bufHdr->tag.blockNum;
+ fctx->record[record_id].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[record_id].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[record_id].isdirty = true;
+ else
+ fctx->record[record_id].isdirty = false;
+
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[record_id].isvalid = true;
+ else
+ fctx->record[record_id].isvalid = false;
+
+ fctx->record[record_id].numa_zone_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Format and return a tuple for a single buffer cache entry.
+ */
+static Datum
+get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[record_id].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[record_id].blocknum == InvalidBlockNumber ||
+ fctx->record[record_id].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[record_id].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[record_id].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[record_id].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[record_id].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[record_id].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[record_id].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[record_id].usagecount);
+ nulls[7] = false;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ values[8] = Int32GetDatum(fctx->record[record_id].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[record_id].numa_zone_id);
+ nulls[9] = false;
+ }
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +324,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_build_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +334,130 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ uint64 os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used,
+ * while the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: - Determine the OS
+ * memory page size - Calculate how many OS pages are used by all
+ * buffer blocks - Calculate how many OS pages are contained within
+ * each database block
+ *
+ * This information is needed before calling move_pages() for NUMA
+ * zone inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%lu os_page_size=%zu pages_per_blk=%.2f",
+ (unsigned long) os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(uint64) * os_page_count);
+ memset(os_page_ptrs, 0, sizeof(void *) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+
+ /*
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ pg_buffercache_build_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size,
+ os_page_ptrs);
}
- else
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Set the NUMA zone ID for this buffer based on the first OS page
+ * it maps to.
+ *
+ * Note: We could check for errors in os_pages_status and report
+ * them. Also, a single DB block might span multiple NUMA zones if
+ * it crosses OS pages on zone boundaries, but we only record the
+ * zone of the first page. This is a simplification but should be
+ * sufficient for most analyses.
+ */
+ fctx->record[i].numa_zone_id = os_pages_status[blk2page];
}
+ }
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ funcctx = SRF_PERCALL_SETUP();
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstNumaTouch = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..2225b879f58
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,20 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..4b49bb2974a 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,14 @@
convenient use.
</para>
+ <para>
+ The <function>pg_buffercache_numa_pages()</function> provides the same information
+ as <function>pg_buffercache_pages()</function> but is slower because it also
+ provides the <acronym>NUMA</acronym> node ID per shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +210,55 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are identical to the
+ <structname>pg_buffercache</structname> view, except that this one includes
+ one additional <structfield>zone_id</structfield> column as defined in
+ <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Extra column</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>zone_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ <acronym>NUMA</acronym> node ID. NULL if the shared buffer
+ has not been used yet. On systems without <acronym>NUMA</acronym> support
+ this returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ As <acronym>NUMA</acronym> node ID inquiry for each page requires memory pages
+ to be paged-in, the first execution of this function can take a noticeable
+ amount of time. In all the cases (first execution or not), retrieving this
+ information is costly and querying the view at a high frequency is not recommended.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-17 16:11 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-17 16:11 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Mon, Mar 17, 2025 at 08:28:46AM +0100, Jakub Wartak wrote:
> On Fri, Mar 14, 2025 at 1:08 PM Bertrand Drouvot
> <[email protected]> wrote:
> >
> > I did prepare a patch file (attached as .txt to not disturb the cfbot) to apply
> > on top of v11 0002 (I just rebased it a bit so that it now applies on top of
> > v12 0002).
>
> Hey Bertrand,
>
> all LGTM (good ideas), so here's v13 attached with applied all of that
> (rebased, tested).
Thanks for v13!
Looking at 0003:
=== 1
+ <entry>NUMA mappings for shared memory allocations</entry>
s/NUMA mappings/NUMA node mappings/ maybe?
=== 2
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> view shows NUMA nodes
+ assigned allocations made from the server's main shared memory segment.
What about?
"
shows how shared memory allocations in the server's main shared memory segment
are distributed across NUMA nodes" ?
=== 3
+ <structfield>numa_zone_id</structfield> <type>int4</type>
s/numa_zone_id/zone_id? to be consistent with pg_buffercache_numa introduced in
0002.
BTW, I wonder if "node_id" would be better (to match the descriptions...).
If so, would also need to be done in 0002.
=== 4
+ ID of NUMA node
<acronym>NUMA</acronym> node ID? (to be consistent with 0002).
=== 5
+static bool firstUseInBackend = true;
Let's use firstNumaTouch to be consistent with 0002.
=== 6
+ elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
There is 2 ";" + I think that we should used the same wording as in
pg_buffercache_numa_pages().
=== 7
What about using ERROR instead? (like in pg_buffercache_numa_pages())
=== 8
+ /*
+ * This is for gathering some NUMA statistics. We might be using various
+ * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
+ * various different OS memory pages sizes, so first we need to understand
+ * the OS memory page size before calling move_pages()
+ */
+ os_page_size = pg_numa_get_pagesize();
Maybe use the same comment as the one in pg_buffercache_numa_pages() before calling
pg_numa_get_pagesize()?
=== 9
+ max_zones = pg_numa_get_max_node();
I think we are mixing "zone" and "node". I think we should standardize on one
and use it everywhere (code and doc for both 0002 and 0003). I'm tempted to
vote for node, but zone is fine too if you prefer.
=== 10
+ /*
+ * Preallocate memory all at once without going into details which shared
+ * memory segment is the biggest (technically min s_b can be as low as
+ * 16xBLCKSZ)
+ */
What about?
"
Allocate memory for page pointers and status based on total shared memory size.
This simplified approach allocates enough space for all pages in shared memory
rather than calculating the exact requirements for each segment.
" instead?
=== 11
+ int shm_total_page_count,
+ shm_ent_page_count,
I think those 2 should be uint64.
=== 12
+ /*
+ * XXX: We are ignoring in NUMA version reporting of the following regions
+ * (compare to pg_get_shmem_allocations() case): 1. output shared memory
+ * allocated but not counted via the shmem index 2. output as-of-yet
+ * unused shared memory
+ */
why XXX?
what about?
"
We are ignoring the following memory regions (as compared to
pg_get_shmem_allocations())....
=== 13
+ page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
+ memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
maybe we could use palloc0() here?
=== 14
and I realize that we could probably use it in 0002 for os_page_ptrs.
=== 15
I think there is still some multi-lines comments that are missing a period. I
probably also missed some in 0002 during the previous review. I think that's
worth another check.
=== 16
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA zone doesn't return -2.
+ */
maybe use the same wording as in 0002?
=== 17
The logic in 0003 looks ok to me. I don't like the 2 loops on shm_ent_page_count
but (as for 0002) it looks like we can not avoid it (or at least I don't see
a way to avoid it).
I'll still review the whole set of patches 0001, 0002 and 0003 once 0003 is
updated.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-18 10:19 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-18 10:19 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Mon, Mar 17, 2025 at 5:11 PM Bertrand Drouvot
<[email protected]> wrote:
> Thanks for v13!
Rebased and fixes inside in the attached v14 (it passes CI too):
> Looking at 0003:
>
> === 1
>
> + <entry>NUMA mappings for shared memory allocations</entry>
>
> s/NUMA mappings/NUMA node mappings/ maybe?
Done.
> === 2
>
> + <para>
> + The <structname>pg_shmem_numa_allocations</structname> view shows NUMA nodes
> + assigned allocations made from the server's main shared memory segment.
>
> What about?
>
> "
> shows how shared memory allocations in the server's main shared memory segment
> are distributed across NUMA nodes" ?
Done.
> === 3
>
> + <structfield>numa_zone_id</structfield> <type>int4</type>
>
> s/numa_zone_id/zone_id? to be consistent with pg_buffercache_numa introduced in
> 0002.
>
> BTW, I wonder if "node_id" would be better (to match the descriptions...).
> If so, would also need to be done in 0002.
Somewhat duplicate, please see answer for #9
> === 4
>
> + ID of NUMA node
>
> <acronym>NUMA</acronym> node ID? (to be consistent with 0002).
>
> === 5
>
> +static bool firstUseInBackend = true;
>
> Let's use firstNumaTouch to be consistent with 0002.
Done.
> === 6
>
> + elog(NOTICE, "libnuma initialization failed or NUMA is not supported on this platform, some NUMA data might be unavailable.");;
>
> There is 2 ";" + I think that we should used the same wording as in
> pg_buffercache_numa_pages().
>
> === 7
>
> What about using ERROR instead? (like in pg_buffercache_numa_pages())
Both are synced now.
> === 8
>
> + /*
> + * This is for gathering some NUMA statistics. We might be using various
> + * DB block sizes (4kB, 8kB , .. 32kB) that end up being allocated in
> + * various different OS memory pages sizes, so first we need to understand
> + * the OS memory page size before calling move_pages()
> + */
> + os_page_size = pg_numa_get_pagesize();
>
> Maybe use the same comment as the one in pg_buffercache_numa_pages() before calling
> pg_numa_get_pagesize()?
Done, improved style of the comment there and synced pg_buffercache
one to shmem.c one.
> === 9
>
> + max_zones = pg_numa_get_max_node();
>
> I think we are mixing "zone" and "node". I think we should standardize on one
> and use it everywhere (code and doc for both 0002 and 0003). I'm tempted to
> vote for node, but zone is fine too if you prefer.
Given that numa(7) does not use "zone" keyword at all and both
/proc/zoneinfo and /proc/pagetypeinfo shows that NUMA nodes are split
into zones, we can conclude that "zone" is simply a subdivision within
a NUMA node's memory (internal kernel thing). Examples are ZONE_DMA,
ZONE_NORMAL, ZONE_HIGHMEM. We are fetching just node id info (without
internal information about zones), therefore we should stay away from
using "zone" within the patch at all, as we are just fetching NUMA
node info. My bad, it's a terminology error on my side from start -
I've probably saw "zone" info in some command output back then when we
had that workshop and started using it and somehow it propagated
through the patchset up to this day... I've adjusted it all and
settled on "numa_node_id" column name.
> === 10
>
> + /*
> + * Preallocate memory all at once without going into details which shared
> + * memory segment is the biggest (technically min s_b can be as low as
> + * 16xBLCKSZ)
> + */
>
> What about?
>
> "
> Allocate memory for page pointers and status based on total shared memory size.
> This simplified approach allocates enough space for all pages in shared memory
> rather than calculating the exact requirements for each segment.
> " instead?
Done.
> === 11
>
> + int shm_total_page_count,
> + shm_ent_page_count,
>
> I think those 2 should be uint64.
Right...
> === 12
>
> + /*
> + * XXX: We are ignoring in NUMA version reporting of the following regions
> + * (compare to pg_get_shmem_allocations() case): 1. output shared memory
> + * allocated but not counted via the shmem index 2. output as-of-yet
> + * unused shared memory
> + */
>
> why XXX?
>
> what about?
>
> "
> We are ignoring the following memory regions (as compared to
> pg_get_shmem_allocations())....
Fixed , it was apparently leftover of when I was thinking if we should
still report it.
> === 13
>
> + page_ptrs = palloc(sizeof(void *) * shm_total_page_count);
> + memset(page_ptrs, 0, sizeof(void *) * shm_total_page_count);
>
> maybe we could use palloc0() here?
Of course!++
> === 14
>
> and I realize that we could probably use it in 0002 for os_page_ptrs.
Of course!++
> === 15
>
> I think there is still some multi-lines comments that are missing a period. I
> probably also missed some in 0002 during the previous review. I think that's
> worth another check.
Please do such a check, I've tried pgident on all .c files, but I'm
apparently blind to such issues. BTW if patch has anything left that
causes pgident to fix, that is not picked by CI but it is picked by
buildfarm??
> === 16
>
> + * In order to get reliable results we also need to touch memory
> + * pages so that inquiry about NUMA zone doesn't return -2.
> + */
>
> maybe use the same wording as in 0002?
But 0002 used:
"In order to get reliable results we also need to touch memory pages, so that
inquiry about NUMA zone doesn't return -2 (which indicates
unmapped/unallocated
pages)"
or are you looking at something different?
> === 17
>
> The logic in 0003 looks ok to me. I don't like the 2 loops on shm_ent_page_count
> but (as for 0002) it looks like we can not avoid it (or at least I don't see
> a way to avoid it).
Hm, it's literally debug code. Why would we care so much if it is 2
loops rather than 1? (as stated earlier we need to pack ptrs and then
analyze it)
> I'll still review the whole set of patches 0001, 0002 and 0003 once 0003 is
> updated.
Cool, thanks in advance.
-J.
Attachments:
[application/octet-stream] v14-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (18.8K, ../../CAKZiRmzGTMbCfEcqHXhQuqc-Pb7P4iFW95JNXXjnpPtr5g_6rw@mail.gmail.com/2-v14-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From 75f4af43e74320058af914fe0ccc1b07f2e61cd6 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v14 1/3] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper. Other platforms can be added later.
This also adds function pg_numa_available() that can be used to check if
the server was linked with NUMA support.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway on i386 it is very memory
limited platform even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 +-
configure | 87 ++++++++++++++
configure.ac | 13 +++
doc/src/sgml/func.sgml | 13 +++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 23 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 46 ++++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 168 ++++++++++++++++++++++++++++
17 files changed, 397 insertions(+), 5 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 5849cbb839a..7010dff7aef 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -519,6 +522,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -835,8 +839,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 93fddd69981..23c33dd9971 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 1c3810e1a04..113588defdd 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25078,6 +25078,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if the server has been compiled with <acronym>NUMA</acronym> support.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b9..9f56205a1d7 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 13c13748e5d..4106c4b13f5 100644
--- a/meson.build
+++ b/meson.build
@@ -949,6 +949,27 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ # via pkg-config
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if not cc.has_header('numa.h', dependencies: libnuma, required: libnumaopt)
+ libnuma = not_found_dep
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3168,6 +3189,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3823,6 +3845,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 3b620bac5ac..0bd4b2d7d32 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -191,6 +191,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 0d3ebf06a95..37cda877f57 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -564,7 +564,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 890822eaf79..85902903653 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8492,6 +8492,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '9685', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..986152e0942
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,46 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+#include "postgres.h"
+#include "fmgr.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+extern PGDLLIMPORT Datum pg_numa_available(PG_FUNCTION_ARGS);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..7d905ef31f5
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "fmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/*
+ * XXX: not really tested as there is no way to trigger this in our
+ * current usage of libnuma.
+ *
+ * The libnuma built-in code can be seen here:
+ * https://github.com/numactl/numactl/blob/master/libnuma.c
+ *
+ */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here.
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
+
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_BOOL(pg_numa_init() != -1);
+}
--
2.39.5
[application/octet-stream] v14-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch (16.1K, ../../CAKZiRmzGTMbCfEcqHXhQuqc-Pb7P4iFW95JNXXjnpPtr5g_6rw@mail.gmail.com/3-v14-0003-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch)
download | inline diff:
From ddde9c9eb099b81892bdfe243d68870402eb8f4f Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v14 3/3] Add pg_shmem_numa_allocations to show NUMA memory
node for shared memory allocations.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 79 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 129 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 9 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 271 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..310ee861dd8 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA node mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,80 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> shows how shared
+ memory allocations in the server's main shared memory segment are distributed
+ across NUMA nodes. This includes both memory allocated by
+ <productname>PostgreSQL</productname> itself and memory allocated
+ by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_node_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of <acronym>NUMA</acronym> node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA memory node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf5..cc014a62dc2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..9094d39ba3c 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port//pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -89,6 +90,8 @@ slock_t *ShmemLock; /* spinlock for shared memory and LWLock
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+/* To get reliable results for NUMA inquiry we need to "touch pages" once */
+static bool firstNumaTouch = true;
/*
* InitShmemAccess() --- set up basic pointers to shared memory.
@@ -568,3 +571,129 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA memory nodes for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ uint64 shm_total_page_count,
+ shm_ent_page_count,
+ max_nodes;
+ Size *nodes;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+ return (Datum) 0;
+ }
+ max_nodes = pg_numa_get_max_node();
+ nodes = palloc(sizeof(Size) * (max_nodes + 1));
+
+ /*
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used, while
+ * the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: - Determine the OS memory
+ * page size - Calculate how many OS pages are used by all buffer blocks -
+ * Calculate how many OS pages are contained within each database block
+ *
+ * This information is needed before calling move_pages() for NUMA memory
+ * node inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Allocate memory for page pointers and status based on total shared
+ * memory size. This simplified approach allocates enough space for all
+ * pages in shared memory rather than calculating the exact requirements
+ * for each segment.
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc0(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages so that inquiry about NUMA memory node doesn't return -2.
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ memset(nodes, 0, sizeof(Size) * (max_nodes + 1));
+ /* Count number of NUMA nodes used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ /* Ensure we are adding only valid index to the array */
+ if (s >= 0 && s <= max_nodes)
+ nodes[s]++;
+ }
+
+ for (i = 0; i <= max_nodes; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(nodes[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /*
+ * We are ignoring the following memory regions (as compared to
+ * pg_get_shmem_allocations()): 1. output shared memory allocated but not
+ * counted via the shmem index 2. output as-of-yet unused shared memory
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstNumaTouch = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 85902903653..f7e7c90a886 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8496,6 +8496,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '9686', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,numa_node_id,numa_size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 954f549555e..d9d62470cdc 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b2..0a0a989fcf6 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ numa_node_id,
+ numa_size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, numa_node_id, numa_size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0a35f2f8f6a..0f38caa0d24 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..fddb21a260a
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,9 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index b81694c24f2..f93d4829702 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
[application/octet-stream] v14-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (28.6K, ../../CAKZiRmzGTMbCfEcqHXhQuqc-Pb7P4iFW95JNXXjnpPtr5g_6rw@mail.gmail.com/4-v14-0002-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From 1684b2a2e8d89ec94e1669e8707d6ff6ebaff737 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 11:17:28 +0100
Subject: [PATCH v14 2/3] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA memory node for individual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 +
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 42 ++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 478 +++++++++++++-----
.../sql/pg_buffercache_numa.sql | 20 +
doc/src/sgml/pgbuffercache.sgml | 61 ++-
9 files changed, 505 insertions(+), 134 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..4b5e864cb79
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, numa_node_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..27762db24ba 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,12 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -43,6 +43,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_node_id;
} BufferCachePagesRec;
@@ -61,84 +62,258 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/* Only need to touch memory once per backend process lifetime */
+static bool firstNumaTouch = true;
+
+/*
+ * Helper routine to map Buffers into addresses that is used by
+ * pg_numa_query_pages().
+ *
+ * When database block size (BLCKSZ) is smaller than the OS page size (4kB),
+ * multiple database buffers will map to the same OS memory page. In this case,
+ * we only need to query the NUMA node for the first memory address of each
+ * unique OS page rather than for every buffer.
+ *
+ * In order to get reliable results we also need to touch memory pages, so that
+ * inquiry about NUMA memory node doesn't return -2 (which indicates
+ * unmapped/unallocated pages)
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk,
+ Size os_page_size,
+ void **os_page_ptrs)
+{
+ size_t blk2page = (size_t) (buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers starts from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) +
+ (os_page_size * j);
+
+ /* Only need to touch memory once per backend process lifetime */
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ }
+
+ CHECK_FOR_INTERRUPTS();
+ }
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * This is almost identical to pg_buffercache_numa_pages(), but this one performs
+ * memory mapping inquiries to display NUMA node information for each buffer.
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. We unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "numa_node_id",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /*
+ * Return to original context when allocating transient memory
+ */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Build buffer cache information for a single buffer.
+ */
+static void
+pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(record_id);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[record_id].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[record_id].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[record_id].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[record_id].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[record_id].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[record_id].blocknum = bufHdr->tag.blockNum;
+ fctx->record[record_id].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[record_id].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[record_id].isdirty = true;
+ else
+ fctx->record[record_id].isdirty = false;
+
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[record_id].isvalid = true;
+ else
+ fctx->record[record_id].isvalid = false;
+
+ fctx->record[record_id].numa_node_id = -1;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Format and return a tuple for a single buffer cache entry.
+ */
+static Datum
+get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[record_id].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[record_id].blocknum == InvalidBlockNumber ||
+ fctx->record[record_id].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
+
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
+ nulls[8] = true;
+ nulls[9] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[record_id].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[record_id].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[record_id].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[record_id].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[record_id].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[record_id].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[record_id].usagecount);
+ nulls[7] = false;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ values[8] = Int32GetDatum(fctx->record[record_id].pinning_backends);
+ nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[record_id].numa_node_id);
+ nulls[9] = false;
+ }
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +324,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_build_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +334,129 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ uint64 os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used,
+ * while the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: - Determine the OS
+ * memory page size - Calculate how many OS pages are used by all
+ * buffer blocks - Calculate how many OS pages are contained within
+ * each database block
+ *
+ * This information is needed before calling move_pages() for NUMA
+ * node id inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%lu os_page_size=%zu pages_per_blk=%.2f",
+ (unsigned long) os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc0(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(uint64) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+
+ /*
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
*/
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
+ for (i = 0; i < NBuffers; i++)
{
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
+ pg_buffercache_build_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size,
+ os_page_ptrs);
}
- else
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
{
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Set the NUMA node id for this buffer based on the first OS page
+ * it maps to.
+ *
+ * Note: We could check for errors in os_pages_status and report
+ * them. Also, a single DB block might span multiple NUMA nodes if
+ * it crosses OS pages on node boundaries, but we only record the
+ * node of the first page. This is a simplification but should be
+ * sufficient for most analyses.
+ */
+ fctx->record[i].numa_node_id = os_pages_status[blk2page];
}
+ }
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ funcctx = SRF_PERCALL_SETUP();
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
+ firstNumaTouch = false;
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..2225b879f58
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,20 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..8216b7cd93b 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,14 @@
convenient use.
</para>
+ <para>
+ The <function>pg_buffercache_numa_pages()</function> provides the same information
+ as <function>pg_buffercache_pages()</function> but is slower because it also
+ provides the <acronym>NUMA</acronym> node ID per shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +210,55 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are identical to the
+ <structname>pg_buffercache</structname> view, except that this one includes
+ one additional <structfield>numa_node_id</structfield> column as defined in
+ <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Extra column</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_node_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ <acronym>NUMA</acronym> node ID. NULL if the shared buffer
+ has not been used yet. On systems without <acronym>NUMA</acronym> support
+ this returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ As <acronym>NUMA</acronym> node ID inquiry for each page requires memory pages
+ to be paged-in, the first execution of this function can take a noticeable
+ amount of time. In all the cases (first execution or not), retrieving this
+ information is costly and querying the view at a high frequency is not recommended.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-18 14:29 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-18 14:29 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
Hi,
On Tue, Mar 18, 2025 at 11:19:32AM +0100, Jakub Wartak wrote:
> On Mon, Mar 17, 2025 at 5:11 PM Bertrand Drouvot
> <[email protected]> wrote:
>
> > Thanks for v13!
>
> Rebased and fixes inside in the attached v14 (it passes CI too):
Thanks!
> > === 9
> >
> > + max_zones = pg_numa_get_max_node();
> >
> > I think we are mixing "zone" and "node". I think we should standardize on one
> > and use it everywhere (code and doc for both 0002 and 0003). I'm tempted to
> > vote for node, but zone is fine too if you prefer.
>
> Given that numa(7) does not use "zone" keyword at all and both
> /proc/zoneinfo and /proc/pagetypeinfo shows that NUMA nodes are split
> into zones, we can conclude that "zone" is simply a subdivision within
> a NUMA node's memory (internal kernel thing). Examples are ZONE_DMA,
> ZONE_NORMAL, ZONE_HIGHMEM. We are fetching just node id info (without
> internal information about zones), therefore we should stay away from
> using "zone" within the patch at all, as we are just fetching NUMA
> node info. My bad, it's a terminology error on my side from start -
> I've probably saw "zone" info in some command output back then when we
> had that workshop and started using it and somehow it propagated
> through the patchset up to this day...
Thanks for the explanation.
> I've adjusted it all and settled on "numa_node_id" column name.
Yeah, I can see, things like:
+ <para>
+ The definitions of the columns exposed are identical to the
+ <structname>pg_buffercache</structname> view, except that this one includes
+ one additional <structfield>numa_node_id</structfield> column as defined in
+ <xref linkend="pgbuffercache-numa-columns"/>.
and like:
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>numa_size</structfield> <type>int4</type>
+ </para>
I think that you re-introduced the "numa_" in the column(s) name that we get
rid (or agreed to) of previously.
I think that we can get rid of the "numa_" stuff in column(s) name as
the column(s) are part of "numa" relation views/output anyway.
I think "node_id", "size" as column(s) name should be enough.
Or maybe that re-adding "numa_" was intentional?
> > === 15
> >
> > I think there is still some multi-lines comments that are missing a period. I
> > probably also missed some in 0002 during the previous review. I think that's
> > worth another check.
>
> Please do such a check,
Found much more:
+ * In order to get reliable results we also need to touch memory pages, so that
+ * inquiry about NUMA memory node doesn't return -2 (which indicates
+ * unmapped/unallocated pages)
is missing the period
and
+ /*
+ * Switch context when allocating stuff to be used in later calls
+ */
should be as before, meaning on current HEAD:
/* Switch context when allocating stuff to be used in later calls */
and
+ /*
+ * Return to original context when allocating transient memory
+ */
should be as before, meaning on current HEAD:
/* Return to original context when allocating transient memory */
and
+ /*
+ * Note if the buffer is valid, and has storage created
+ */
should be as before, meaning on current HEAD:
/* Note if the buffer is valid, and has storage created */
and
+ /*
+ * unused for v1.0 callers, but the array is always long enough
+ */
should be as before, meaning on current HEAD:
/* unused for v1.0 callers, but the array is always long enough */
and
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings
is missing the period
and
+ * To correctly map between them, we need to: - Determine the OS
+ * memory page size - Calculate how many OS pages are used by all
+ * buffer blocks - Calculate how many OS pages are contained within
+ * each database block
is missing the period (2 times as this comment appears in 0002 and 0003)
and
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here
is missing the period (2 times as this comment appears in 0002 and 0003)
and
+ /*
+ * We are ignoring the following memory regions (as compared to
+ * pg_get_shmem_allocations()): 1. output shared memory allocated but not
+ * counted via the shmem index 2. output as-of-yet unused shared memory
is missing the period
> I've tried pgident on all .c files, but I'm
> apparently blind to such issues.
I don't think pgident would report missing period.
> BTW if patch has anything left that
> causes pgident to fix, that is not picked by CI but it is picked by
> buildfarm??
I think it has to be done manually before each commit and that this is anyway
done at least once per release cycle.
> > === 16
> >
> > + * In order to get reliable results we also need to touch memory
> > + * pages so that inquiry about NUMA zone doesn't return -2.
> > + */
> >
> > maybe use the same wording as in 0002?
>
> But 0002 used:
>
> "In order to get reliable results we also need to touch memory pages, so that
> inquiry about NUMA zone doesn't return -2 (which indicates
> unmapped/unallocated
> pages)"
>
> or are you looking at something different?
Nope, I meant to say that it could make sense to have the exact same comment.
> > === 17
> >
> > The logic in 0003 looks ok to me. I don't like the 2 loops on shm_ent_page_count
> > but (as for 0002) it looks like we can not avoid it (or at least I don't see
> > a way to avoid it).
>
> Hm, it's literally debug code. Why would we care so much if it is 2
> loops rather than 1? (as stated earlier we need to pack ptrs and then
> analyze it)
Yeah, but if we could just loop one time I'm pretty sure we'd have done that.
> > I'll still review the whole set of patches 0001, 0002 and 0003 once 0003 is
> > updated.
> :w
> Cool, thanks in advance.
0001 looks in a good shape from my point of view.
For 0002:
=== 1
I wonder if pg_buffercache_init_entries() and pg_buffercache_build_tuple() could
deserve their own patch. That would ease the review for the "real" numa stuff.
Maybe something like:
0001 as it is
0002 introduces (and uses) pg_buffercache_init_entries() and
pg_buffercache_build_tuple()
0003 current 0002 attached minus 0002 above
We did it that way in c2a50ac678e and ff7c40d7fd6 for example.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-19 09:06 Jakub Wartak <[email protected]>
parent: Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-19 09:06 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers
On Tue, Mar 18, 2025 at 3:29 PM Bertrand Drouvot
<[email protected]> wrote:
Hi! v15 attached, rebased, CI-tested, all fixed incorporated.
> > I've adjusted it all and settled on "numa_node_id" column name.
>
[...]
> I think that we can get rid of the "numa_" stuff in column(s) name as
> the column(s) are part of "numa" relation views/output anyway.
[...]
Done, you are probably right (it was done to keep consistency between
those two views probably), I'm just not that strongly attached to the
naming things.
> > Please do such a check,
>
> Found much more:
>
[.. 9 issues with missing dots at the end of sentences in comments +
fixes to comment structure in relation to HEAD..]
All fixed.
> > BTW if patch has anything left that
> > causes pgident to fix, that is not picked by CI but it is picked by
> > buildfarm??
>
> I think it has to be done manually before each commit and that this is anyway
> done at least once per release cycle.
OK, thanks for clarification.
[..]
> >
> > But 0002 used:
> >
> > "In order to get reliable results we also need to touch memory pages, so that
> > inquiry about NUMA zone doesn't return -2 (which indicates
> > unmapped/unallocated
> > pages)"
> >
> > or are you looking at something different?
>
> Nope, I meant to say that it could make sense to have the exact same comment.
Synced those two.
[..]
>
> 0001 looks in a good shape from my point of view.
Cool!
> For 0002:
>
> === 1
>
> I wonder if pg_buffercache_init_entries() and pg_buffercache_build_tuple() could
> deserve their own patch. That would ease the review for the "real" numa stuff.
>
Done, 0001+0002 alone passes the meson test.
-J.
Attachments:
[application/octet-stream] v15-0003-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (17.5K, ../../CAKZiRmyjWhuDRNSEHbCxiGqA-adSUB=DrSEVNfOQSLZ0eDNiAQ@mail.gmail.com/2-v15-0003-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From ee17287bcab2178bfd473a7043ece3b54f498817 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Wed, 19 Mar 2025 09:34:56 +0100
Subject: [PATCH v15 3/4] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA memory node for individual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 +++
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 42 +++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 172 +++++++++++++++++-
.../sql/pg_buffercache_numa.sql | 20 ++
doc/src/sgml/pgbuffercache.sgml | 61 ++++++-
9 files changed, 329 insertions(+), 4 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..0f4b2eaf444
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, node_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index f342005fd96..35b019206f5 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,11 +11,12 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -42,6 +43,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_node_id;
} BufferCachePagesRec;
@@ -60,10 +62,56 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+/* Only need to touch memory once per backend process lifetime */
+static bool firstNumaTouch = true;
+
+/*
+ * Helper routine to map Buffers into addresses that is used by
+ * pg_numa_query_pages().
+ *
+ * When database block size (BLCKSZ) is smaller than the OS page size (4kB),
+ * multiple database buffers will map to the same OS memory page. In this case,
+ * we only need to query the NUMA node for the first memory address of each
+ * unique OS page rather than for every buffer.
+ *
+ * In order to get reliable results we also need to touch memory pages, so that
+ * inquiry about NUMA memory node doesn't return -2 (which indicates
+ * unmapped/unallocated pages).
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk,
+ Size os_page_size,
+ void **os_page_ptrs)
+{
+ size_t blk2page = (size_t) (buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers starts from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) +
+ (os_page_size * j);
+
+ /* Only need to touch memory once per backend process lifetime */
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ }
+
+ CHECK_FOR_INTERRUPTS();
+ }
+}
+
/*
* Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
*
@@ -121,6 +169,9 @@ pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "node_id",
+ INT4OID, -1, 0);
fctx->tupdesc = BlessTupleDesc(tupledesc);
@@ -173,6 +224,8 @@ pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
else
fctx->record[record_id].isvalid = false;
+ fctx->record[record_id].numa_node_id = -1;
+
UnlockBufHdr(bufHdr, buf_state);
}
@@ -208,6 +261,7 @@ get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
/* unused for v1.0 callers, but the array is always long enough */
nulls[8] = true;
+ nulls[9] = true;
}
else
{
@@ -231,6 +285,8 @@ get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
*/
values[8] = Int32GetDatum(fctx->record[record_id].pinning_backends);
nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[record_id].numa_node_id);
+ nulls[9] = false;
}
/* Build and return the tuple. */
@@ -282,6 +338,120 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
}
}
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings.
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ uint64 os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
+
+ /*
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used,
+ * while the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: 1. Determine the OS
+ * memory page size 2. Calculate how many OS pages are used by all
+ * buffer blocks 3. Calculate how many OS pages are contained within
+ * each database block.
+ *
+ * This information is needed before calling move_pages() for NUMA
+ * node id inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%lu os_page_size=%zu pages_per_blk=%.2f",
+ (unsigned long) os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc0(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(uint64) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here.
+ *
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+
+ /*
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
+ */
+ for (i = 0; i < NBuffers; i++)
+ {
+ pg_buffercache_build_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size,
+ os_page_ptrs);
+ }
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Set the NUMA node id for this buffer based on the first OS page
+ * it maps to.
+ *
+ * Note: We could check for errors in os_pages_status and report
+ * them. Also, a single DB block might span multiple NUMA nodes if
+ * it crosses OS pages on node boundaries, but we only record the
+ * node of the first page. This is a simplification but should be
+ * sufficient for most analyses.
+ */
+ fctx->record[i].numa_node_id = os_pages_status[blk2page];
+ }
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
+
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ firstNumaTouch = false;
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
Datum
pg_buffercache_summary(PG_FUNCTION_ARGS)
{
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..2225b879f58
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,20 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..086e0062a17 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,14 @@
convenient use.
</para>
+ <para>
+ The <function>pg_buffercache_numa_pages()</function> provides the same information
+ as <function>pg_buffercache_pages()</function> but is slower because it also
+ provides the <acronym>NUMA</acronym> node ID per shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +210,55 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are identical to the
+ <structname>pg_buffercache</structname> view, except that this one includes
+ one additional <structfield>node_id</structfield> column as defined in
+ <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Extra column</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>node_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ <acronym>NUMA</acronym> node ID. NULL if the shared buffer
+ has not been used yet. On systems without <acronym>NUMA</acronym> support
+ this returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ As <acronym>NUMA</acronym> node ID inquiry for each page requires memory pages
+ to be paged-in, the first execution of this function can take a noticeable
+ amount of time. In all the cases (first execution or not), retrieving this
+ information is costly and querying the view at a high frequency is not recommended.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
[application/octet-stream] v15-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (18.8K, ../../CAKZiRmyjWhuDRNSEHbCxiGqA-adSUB=DrSEVNfOQSLZ0eDNiAQ@mail.gmail.com/3-v15-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From 244d6e9c95adf8857219bbfefefe76c07addde66 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v15 1/4] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper. Other platforms can be added later.
This also adds function pg_numa_available() that can be used to check if
the server was linked with NUMA support.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway on i386 it is very memory
limited platform even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 +-
configure | 87 ++++++++++++++
configure.ac | 13 +++
doc/src/sgml/func.sgml | 13 +++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 23 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 46 ++++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 168 ++++++++++++++++++++++++++++
17 files changed, 397 insertions(+), 5 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 5849cbb839a..7010dff7aef 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
\
${LINUX_CONFIGURE_FEATURES} \
\
@@ -519,6 +522,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -835,8 +839,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 559f535f5cd..0931331f627 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
with_readline
with_systemd
@@ -868,6 +869,7 @@ with_libedit_preferred
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1581,6 +1583,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9140,6 +9143,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12378,6 +12408,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index b6d02f5ecc7..1a394dfc077 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1041,6 +1041,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 2ab5661602c..d7b33c67ec6 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25078,6 +25078,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if the server has been compiled with <acronym>NUMA</acronym> support.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e076cefa3b9..9f56205a1d7 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-libxml">
<term><option>--with-libxml</option></term>
<listitem>
@@ -2611,6 +2621,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index b6daa5b7040..b4cb6929bfc 100644
--- a/meson.build
+++ b/meson.build
@@ -943,6 +943,27 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ # via pkg-config
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if not cc.has_header('numa.h', dependencies: libnuma, required: libnumaopt)
+ libnuma = not_found_dep
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: libxml
@@ -3162,6 +3183,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
libxml,
lz4,
pam,
@@ -3817,6 +3839,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/meson_options.txt b/meson_options.txt
index 702c4517145..adaadb5faf1 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('libxml', type: 'feature', value: 'auto',
description: 'XML support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8fe9d61e82a..7ff45cf86e7 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -196,6 +196,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index cc8f2b1230a..ae5452d9539 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -566,7 +566,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 890822eaf79..85902903653 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8492,6 +8492,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '9685', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index db6454090d2..8894f800607 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -672,6 +672,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to 1 to build with XML support. (--with-libxml) */
#undef USE_LIBXML
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..986152e0942
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,46 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+#include "postgres.h"
+#include "fmgr.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+extern PGDLLIMPORT Datum pg_numa_available(PG_FUNCTION_ARGS);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 60e13d50235..f786c191605 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -199,6 +199,8 @@ pgxs_empty = [
'PTHREAD_CFLAGS', 'PTHREAD_LIBS',
'ICU_LIBS',
+
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS'
]
if host_system == 'windows' and cc.get_argument_syntax() != 'msvc'
@@ -230,6 +232,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'libxml': libxml,
'libxslt': libxslt,
'llvm': llvm,
diff --git a/src/port/Makefile b/src/port/Makefile
index 4c224319512..a68a29d5414 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -44,6 +44,7 @@ OBJS = \
noblock.o \
path.o \
pg_bitutils.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 7fcfa728d43..7ffbd4d88d2 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -7,6 +7,7 @@ pgport_sources = [
'noblock.c',
'path.c',
'pg_bitutils.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..7d905ef31f5
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "fmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/*
+ * XXX: not really tested as there is no way to trigger this in our
+ * current usage of libnuma.
+ *
+ * The libnuma built-in code can be seen here:
+ * https://github.com/numactl/numactl/blob/master/libnuma.c
+ *
+ */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here.
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
+
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_BOOL(pg_numa_init() != -1);
+}
--
2.39.5
[application/octet-stream] v15-0002-This-extracts-code-from-contrib-pg_buffercache-s.patch (13.6K, ../../CAKZiRmyjWhuDRNSEHbCxiGqA-adSUB=DrSEVNfOQSLZ0eDNiAQ@mail.gmail.com/4-v15-0002-This-extracts-code-from-contrib-pg_buffercache-s.patch)
download | inline diff:
From fbb7f669d18dfcc730e64287ab662965d03c460a Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Wed, 19 Mar 2025 09:34:56 +0100
Subject: [PATCH v15 2/4] This extracts code from contrib/pg_buffercache's
primary function to separate functions.
This commit adds pg_buffercache_init_entries(), pg_buffercache_build_tuple()
and get_buffercache_tuple() that help fill result tuplestores based on the
buffercache contents. This will be used in a follow-up commit that implements
NUMA observability in pg_buffercache.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/pg_buffercache_pages.c | 317 ++++++++++--------
1 file changed, 178 insertions(+), 139 deletions(-)
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3ae0a018e10..f342005fd96 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -14,7 +14,6 @@
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
#define NUM_BUFFERCACHE_PAGES_ELEM 9
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
@@ -65,80 +64,192 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * This is almost identical to pg_buffercache_numa_pages(), but this one performs
+ * memory mapping inquiries to display NUMA node information for each buffer.
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /* Switch context when allocating stuff to be used in later calls */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. ee unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /* Return to original context when allocating transient memory */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Build buffer cache information for a single buffer.
+ */
+static void
+pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(record_id);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[record_id].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[record_id].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[record_id].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[record_id].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[record_id].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[record_id].blocknum = bufHdr->tag.blockNum;
+ fctx->record[record_id].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[record_id].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[record_id].isdirty = true;
+ else
+ fctx->record[record_id].isdirty = false;
+
+ /* Note if the buffer is valid, and has storage created */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[record_id].isvalid = true;
+ else
+ fctx->record[record_id].isvalid = false;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Format and return a tuple for a single buffer cache entry.
+ */
+static Datum
+get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[record_id].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[record_id].blocknum == InvalidBlockNumber ||
+ fctx->record[record_id].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
+
+ /* unused for v1.0 callers, but the array is always long enough */
+ nulls[8] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[record_id].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[record_id].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[record_id].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[record_id].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[record_id].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[record_id].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[record_id].usagecount);
+ nulls[7] = false;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ values[8] = Int32GetDatum(fctx->record[record_id].pinning_backends);
+ nulls[8] = false;
+ }
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -149,36 +260,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_build_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -188,59 +270,16 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
-
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
-
- /*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
- */
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
- {
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
- }
- else
- {
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
- }
-
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
--
2.39.5
[application/octet-stream] v15-0004-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch (16.2K, ../../CAKZiRmyjWhuDRNSEHbCxiGqA-adSUB=DrSEVNfOQSLZ0eDNiAQ@mail.gmail.com/5-v15-0004-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch)
download | inline diff:
From f19a8151a9b436b2ab7c5d8b5ae1afe41eb49fb9 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v15 4/4] Add pg_shmem_numa_allocations to show NUMA memory
node for shared memory allocations.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 79 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 131 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 9 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 273 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..e83711f9578 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA node mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,80 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> shows how shared
+ memory allocations in the server's main shared memory segment are distributed
+ across NUMA nodes. This includes both memory allocated by
+ <productname>PostgreSQL</productname> itself and memory allocated
+ by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>node_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of <acronym>NUMA</acronym> node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA memory node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index a4d2cfdcaf5..cc014a62dc2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..a011603d318 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port//pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -89,6 +90,8 @@ slock_t *ShmemLock; /* spinlock for shared memory and LWLock
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+/* To get reliable results for NUMA inquiry we need to "touch pages" once */
+static bool firstNumaTouch = true;
/*
* InitShmemAccess() --- set up basic pointers to shared memory.
@@ -568,3 +571,131 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA memory nodes for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ uint64 shm_total_page_count,
+ shm_ent_page_count,
+ max_nodes;
+ Size *nodes;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+ return (Datum) 0;
+ }
+ max_nodes = pg_numa_get_max_node();
+ nodes = palloc(sizeof(Size) * (max_nodes + 1));
+
+ /*
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used, while
+ * the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: 1. Determine the OS memory
+ * page size 2. Calculate how many OS pages are used by all buffer blocks
+ * 3. Calculate how many OS pages are contained within each database
+ * block.
+ *
+ * This information is needed before calling move_pages() for NUMA memory
+ * node inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Allocate memory for page pointers and status based on total shared
+ * memory size. This simplified approach allocates enough space for all
+ * pages in shared memory rather than calculating the exact requirements
+ * for each segment.
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc0(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here.
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages, so that inquiry about NUMA memory node doesn't return -2
+ * (which indicates unmapped/unallocated pages).
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ memset(nodes, 0, sizeof(Size) * (max_nodes + 1));
+ /* Count number of NUMA nodes used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ /* Ensure we are adding only valid index to the array */
+ if (s >= 0 && s <= max_nodes)
+ nodes[s]++;
+ }
+
+ for (i = 0; i <= max_nodes; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(nodes[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /*
+ * We are ignoring the following memory regions (as compared to
+ * pg_get_shmem_allocations()): 1. output shared memory allocated but not
+ * counted via the shmem index 2. output as-of-yet unused shared memory.
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstNumaTouch = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 85902903653..66d753733d5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8496,6 +8496,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '9686', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,node_id,size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 954f549555e..d9d62470cdc 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 62f69ac20b2..f9b57e2a33f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ node_id,
+ size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, node_id, size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0a35f2f8f6a..0f38caa0d24 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..fddb21a260a
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,9 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index b81694c24f2..f93d4829702 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-19 11:43 Ashutosh Bapat <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-19 11:43 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 13, 2025 at 6:10 PM Ashutosh Bapat
<[email protected]> wrote:
> >
> > I think the fix is to explicitly pass --lc-monetary to the old cluster
> > and the restored cluster. 003 patch in the attached patch set does
> > that. Please check if it fixes the issue for you.
> >
> > Additionally we should check that it gets copied to the new cluster as
> > well. But I haven't figured out how to get those settings yet. This
> > treatment is similar to how --lc-collate and --lc-ctype are treated. I
> > am wondering whether we should explicitly pass --lc-messages,
> > --lc-time and --lc-numeric as well.
> >
> > 2d819a08a1cbc11364e36f816b02e33e8dcc030b introduced buildin locale
> > provider and added overrides to LC_COLLATE and LC_TYPE. But it did not
> > override other LC_, which I think it should have. In pure upgrade
> > test, the upgraded node inherits the locale settings of the original
> > cluster, so this wasn't apparent. But with pg_dump testing, the
> > original and restored databases are independent. Hence I think we have
> > to override all LC_* settings by explicitly mentioning --lc-* options
> > to initdb. Please let me know what you think about this?
> >
Investigated this further. The problem is that the pg_regress run
creates regression database with specific properties but the restored
database does not have those properties. That led me to a better
solution. Additionally it's local to the new test. Use --create when
dumping and restoring the regression database. This way the database
properties or "configuration variable settings (as pg_dump
documentation calls them) are copied to the restored database as well.
Those properties include LC_MONETARY. Additionally now the test covers
--create option as well.
PFA patches.
--
Best Wishes,
Ashutosh Bapat
Attachments:
[text/x-patch] 0002-Do-not-dump-statistics-in-the-file-dumped-f-20250319.patch (2.1K, ../../CAExHW5tLVXNVSYwWaq9k8DuYNLZGAVqNzkyZTUCUGQ4OtbD3tQ@mail.gmail.com/2-0002-Do-not-dump-statistics-in-the-file-dumped-f-20250319.patch)
download | inline diff:
From 886e241e304a23bb31b5e59f12149741dfff2b14 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Tue, 25 Feb 2025 11:42:51 +0530
Subject: [PATCH 2/2] Do not dump statistics in the file dumped for comparison
The dumped and restored statistics of a materialized view may differ as
reported in [1]. Hence do not dump the statistics to avoid differences
in the dump output from the original and restored database.
[1] https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Ashutosh Bapat
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index d08eea6693f..f931fef2307 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -656,15 +656,15 @@ sub get_dump_for_comparison
my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
my $dump_adjusted = "${dumpfile}_adjusted";
- # Usually we avoid comparing statistics in our tests since it is flaky by
- # nature. However, if statistics is dumped and restored it is expected to be
- # restored as it is i.e. the statistics from the original database and that
- # from the restored database should match. We turn off autovacuum on the
- # source and the target database to avoid any statistics update during
- # restore operation. Hence we do not exclude statistics from dump.
+ # If statistics is dumped and restored it is expected to be restored as it
+ # is i.e. the statistics from the original database and that from the
+ # restored database should match. We turn off autovacuum on the source and
+ # the target database to avoid any statistics update during restore
+ # operation. But as of now, there are cases when statistics is not being
+ # restored faithfully. Hence for now do not dump statistics.
$node->command_ok(
[
- 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ 'pg_dump', '--no-sync', '--no-statistics', '-d', $node->connstr($db), '-f',
$dumpfile
],
'dump for comparison succeeded');
--
2.34.1
[text/x-patch] 0001-Test-pg_dump-restore-of-regression-objects-20250319.patch (16.5K, ../../CAExHW5tLVXNVSYwWaq9k8DuYNLZGAVqNzkyZTUCUGQ4OtbD3tQ@mail.gmail.com/3-0001-Test-pg_dump-restore-of-regression-objects-20250319.patch)
download | inline diff:
From 1723050dadb89f3187fef19c994d8c866ee5a788 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Thu, 27 Jun 2024 10:03:53 +0530
Subject: [PATCH 1/2] Test pg_dump/restore of regression objects
002_pg_upgrade.pl tests pg_upgrade of the regression database left
behind by regression run. Modify it to test dump and restore of the
regression database as well.
Regression database created by regression run contains almost all the
database objects supported by PostgreSQL in various states. Hence the
new testcase covers dump and restore scenarios not covered by individual
dump/restore cases. Till now 002_pg_upgrade only tested dump/restore
through pg_upgrade which only uses binary mode. Many regression tests
mention that they leave objects behind for dump/restore testing but they
are not tested in a non-binary mode. The new testcase closes that
gap.
Testing dump and restore of regression database makes this test run
longer for a relatively smaller benefit. Hence run it only when
explicitly requested by user by specifying "regress_dump_test" in
PG_TEST_EXTRA.
Note For the reviewers:
The new test has uncovered many bugs so far in one year.
1. Introduced by 14e87ffa5c54. Fixed in fd41ba93e4630921a72ed5127cd0d552a8f3f8fc.
2. Introduced by 0413a556990ba628a3de8a0b58be020fd9a14ed0. Reverted in 74563f6b90216180fc13649725179fc119dddeb5.
3. Fixed by d611f8b1587b8f30caa7c0da99ae5d28e914d54f
3. Being discussed on hackers at https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Author: Ashutosh Bapat
Reviewed by: Michael Pacquire, Daniel Gustafsson, Tom Lane, Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
---
doc/src/sgml/regress.sgml | 12 ++
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 144 ++++++++++++++++-
src/test/perl/Makefile | 2 +
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 167 ++++++++++++++++++++
src/test/perl/meson.build | 1 +
5 files changed, 324 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/AdjustDump.pm
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 0e5e8e8f309..237b974b3ab 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,6 +357,18 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>regress_dump_test</literal></term>
+ <listitem>
+ <para>
+ When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
+ tests dump and restore of regression database left behind by the
+ regression run. Not enabled by default because it is time and resource
+ consuming.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 00051b85035..d08eea6693f 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -12,6 +12,7 @@ use File::Path qw(rmtree);
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::AdjustUpgrade;
+use PostgreSQL::Test::AdjustDump;
use Test::More;
# Can be changed to test the other modes.
@@ -35,8 +36,8 @@ sub generate_db
"created database with ASCII characters from $from_char to $to_char");
}
-# Filter the contents of a dump before its use in a content comparison.
-# This returns the path to the filtered dump.
+# Filter the contents of a dump before its use in a content comparison for
+# upgrade testing. This returns the path to the filtered dump.
sub filter_dump
{
my ($is_old, $old_version, $dump_file) = @_;
@@ -262,6 +263,21 @@ else
}
}
is($rc, 0, 'regression tests pass');
+
+ # Test dump/restore of the objects left behind by regression. Ideally it
+ # should be done in a separate TAP test, but doing it here saves us one full
+ # regression run.
+ #
+ # This step takes several extra seconds and some extra disk space, so
+ # requires an opt-in with the PG_TEST_EXTRA environment variable.
+ #
+ # Do this while the old cluster is running before it is shut down by the
+ # upgrade test.
+ if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
+ {
+ test_regression_dump_restore($oldnode, %node_params);
+ }
}
# Initialize a new node for the upgrade.
@@ -539,4 +555,128 @@ my $dump2_filtered = filter_dump(0, $oldnode->pg_version, $dump2_file);
compare_files($dump1_filtered, $dump2_filtered,
'old and new dumps match after pg_upgrade');
+# Test dump and restore of objects left behind by the regression run.
+#
+# It is expected that regression tests, which create `regression` database, are
+# run on `src_node`, which in turn, is left in running state. A fresh node is
+# created using given `node_params`, which are expected to be the same ones used
+# to create `src_node`, so as to avoid any differences in the databases.
+#
+# Plain dumps from both the nodes are compared to make sure that all the dumped
+# objects are restored faithfully.
+sub test_regression_dump_restore
+{
+ my ($src_node, %node_params) = @_;
+ my $dst_node = PostgreSQL::Test::Cluster->new('dst_node');
+
+ # Make sure that the source and destination nodes have the same version and
+ # do not use custom install paths. In both the cases, the dump files may
+ # require additional adjustments unknown to code here. Do not run this test
+ # in such a case to avoid utilizing the time and resources unnecessarily.
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default installation");
+ return;
+ }
+
+ # Dump the original database for comparison later.
+ my $src_dump =
+ get_dump_for_comparison($src_node, 'regression', 'src_dump', 1);
+
+ # Setup destination database cluster
+ $dst_node->init(%node_params);
+ # Stabilize stats for comparison.
+ $dst_node->append_conf('postgresql.conf', 'autovacuum = off');
+ $dst_node->start;
+
+ # Test all formats one by one.
+ for my $format ('plain', 'tar', 'directory', 'custom')
+ {
+ my $dump_file = "$tempdir/regression_dump.$format";
+ my $restored_db = 'regression_' . $format;
+
+ # Use --create in dump and restore commands so that the restored
+ # database has the same configurable variable settings as the original
+ # database and the plain dumps taken for comparsion do not differ
+ # because of locale changes. Additionally this provides test coverage
+ # for --create option.
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [ 'psql', '-d', 'postgres', '-f', $dump_file ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '--create',
+ '-d', 'postgres', $dump_file
+ ];
+ }
+ $dst_node->command_ok(@restore_command,
+ "restored dump taken in $format format on destination instance");
+
+ my $dst_dump =
+ get_dump_for_comparison($dst_node, 'regression',
+ 'dest_dump.' . $format, 0);
+
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database (using $format format) match"
+ );
+
+ # Rename the restored database so that it is available for debugging in
+ # case the test fails.
+ $dst_node->safe_psql('postgres', "ALTER DATABASE regression RENAME TO $restored_db");
+ }
+}
+
+# Dump database `db` from the given `node` in plain format and adjust it for
+# comparing dumps from the original and the restored database.
+#
+# `file_prefix` is used to create unique names for all dump files so that they
+# remain available for debugging in case the test fails.
+#
+# `adjust_child_columns` is passed to adjust_regress_dumpfile() which actually
+# adjusts the dump output.
+#
+# The name of the file containting adjusted dump is returned.
+sub get_dump_for_comparison
+{
+ my ($node, $db, $file_prefix, $adjust_child_columns) = @_;
+
+ my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
+ my $dump_adjusted = "${dumpfile}_adjusted";
+
+ # Usually we avoid comparing statistics in our tests since it is flaky by
+ # nature. However, if statistics is dumped and restored it is expected to be
+ # restored as it is i.e. the statistics from the original database and that
+ # from the restored database should match. We turn off autovacuum on the
+ # source and the target database to avoid any statistics update during
+ # restore operation. Hence we do not exclude statistics from dump.
+ $node->command_ok(
+ [
+ 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ $dumpfile
+ ],
+ 'dump for comparison succeeded');
+
+ open(my $dh, '>', $dump_adjusted)
+ || die
+ "could not open $dump_adjusted for writing the adjusted dump: $!";
+ print $dh adjust_regress_dumpfile(slurp_file($dumpfile),
+ $adjust_child_columns);
+ close($dh);
+
+ return $dump_adjusted;
+}
+
done_testing();
diff --git a/src/test/perl/Makefile b/src/test/perl/Makefile
index d82fb67540e..def89650ead 100644
--- a/src/test/perl/Makefile
+++ b/src/test/perl/Makefile
@@ -26,6 +26,7 @@ install: all installdirs
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/Cluster.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/BackgroundPsql.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustUpgrade.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustDump.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Version.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
uninstall:
@@ -36,6 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
endif
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..74b9a60cf34
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,167 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::AdjustDump - helper module for dump and restore tests
+
+=head1 SYNOPSIS
+
+ use PostgreSQL::Test::AdjustDump;
+
+ # Adjust contents of dump output file so that dump output from original
+ # regression database and that from the restored regression database match
+ $dump = adjust_regress_dumpfile($dump, $adjust_child_columns);
+
+=head1 DESCRIPTION
+
+C<PostgreSQL::Test::AdjustDump> encapsulates various hacks needed to
+compare the results of dump and restore tests
+
+=cut
+
+package PostgreSQL::Test::AdjustDump;
+
+use strict;
+use warnings FATAL => 'all';
+
+use Exporter 'import';
+use Test::More;
+
+our @EXPORT = qw(
+ adjust_regress_dumpfile
+);
+
+=pod
+
+=head1 ROUTINES
+
+=over
+
+=item $dump = adjust_regress_dumpfile($dump, $adjust_child_columns)
+
+If we take dump of the regression database left behind after running regression
+tests, restore the dump, and take dump of the restored regression database, the
+outputs of both the dumps differ in the following cases. This routine adjusts
+the given dump so that dump outputs from the original and restored database,
+respectively, match.
+
+Case 1: Some regression tests purposefully create child tables in such a way
+that the order of their inherited columns differ from column orders of their
+respective parents. In the restored database, however, the order of their
+inherited columns are same as that of their respective parents. Thus the column
+orders of these child tables in the original database and those in the restored
+database differ, causing difference in the dump outputs. See MergeAttributes()
+and dumpTableSchema() for details. This routine rearranges the column
+declarations in the relevant C<CREATE TABLE... INHERITS> statements in the dump
+file from original database to match those from the restored database. We could,
+instead, adjust the statements in the dump from the restored database to match
+those from original database or adjust both to a canonical order. But we have
+chosen to adjust the statements in the dump from original database for no
+particular reason.
+
+Case 2: When dumping COPY statements the columns are ordered by their attribute
+number by fmtCopyColumnList(). If a column is added to a parent table after a
+child has inherited the parent and the child has its own columns, the attribute
+number of the column changes after restoring the child table. This is because
+when executing the dumped C<CREATE TABLE... INHERITS> statement all the parent
+attributes are created before any child attributes. Thus the order of columns in
+COPY statements dumped from the original and the restored databases,
+respectively, differs. Such tables in regression tests are listed below. It is
+hard to adjust the column order in the COPY statement along with the data. Hence
+we just remove such COPY statements from the dump output.
+
+Additionally the routine adjusts blank and new lines to avoid noise.
+
+Note: Usually we avoid comparing statistics in our tests since it is flaky by
+nature. However, if statistics is dumped and restored it is expected to be
+restored as it is i.e. the statistics from the original database and that from
+the restored database should match. Hence we do not filter statistics from dump,
+if it's dumped.
+
+Arguments:
+
+=over
+
+=item C<dump>: Contents of dump file
+
+=item C<adjust_child_columns>: 1 indicates that the given dump file requires
+adjusting columns in the child tables; usually when the dump is from original
+database. 0 indicates no such adjustment is needed; usually when the dump is
+from restored database.
+
+=back
+
+Returns the adjusted dump text.
+
+=cut
+
+sub adjust_regress_dumpfile
+{
+ my ($dump, $adjust_child_columns) = @_;
+
+ # use Unix newlines
+ $dump =~ s/\r\n/\n/g;
+
+ # Adjust the CREATE TABLE ... INHERITS statements.
+ if ($adjust_child_columns)
+ {
+ my $saved_dump = $dump;
+
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_stored_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_stored_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_virtual_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_virtual_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c1\s\()
+ (\n\s+int_four\sbigint),
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint)/$1$4,$2,$3/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c1 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c2\s\()
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint),
+ (\n\s+int_four\sbigint)/$1$3,$4,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c2 adjustments');
+ }
+
+ # Remove COPY statements with differing column order
+ for my $table (
+ 'public\.b_star', 'public\.c_star',
+ 'public\.cc2', 'public\.d_star',
+ 'public\.e_star', 'public\.f_star',
+ 'public\.renamecolumnanother', 'public\.renamecolumnchild',
+ 'public\.test_type_diff2_c1', 'public\.test_type_diff2_c2',
+ 'public\.test_type_diff_c')
+ {
+ $dump =~ s/^COPY\s$table\s\(.+?^\\\.$//sm;
+ }
+
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
+ return $dump;
+}
+
+=pod
+
+=back
+
+=cut
+
+1;
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
index 58e30f15f9d..492ca571ff8 100644
--- a/src/test/perl/meson.build
+++ b/src/test/perl/meson.build
@@ -14,4 +14,5 @@ install_data(
'PostgreSQL/Test/Cluster.pm',
'PostgreSQL/Test/BackgroundPsql.pm',
'PostgreSQL/Test/AdjustUpgrade.pm',
+ 'PostgreSQL/Test/AdjustDump.pm',
install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
base-commit: 190dc27998d5b7b4c36e12bebe62f7176f4b4507
--
2.34.1
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-20 15:06 vignesh C <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: vignesh C @ 2025-03-20 15:06 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, 19 Mar 2025 at 17:13, Ashutosh Bapat
<[email protected]> wrote:
>
> On Thu, Mar 13, 2025 at 6:10 PM Ashutosh Bapat
> <[email protected]> wrote:
> > >
> > > I think the fix is to explicitly pass --lc-monetary to the old cluster
> > > and the restored cluster. 003 patch in the attached patch set does
> > > that. Please check if it fixes the issue for you.
> > >
> > > Additionally we should check that it gets copied to the new cluster as
> > > well. But I haven't figured out how to get those settings yet. This
> > > treatment is similar to how --lc-collate and --lc-ctype are treated. I
> > > am wondering whether we should explicitly pass --lc-messages,
> > > --lc-time and --lc-numeric as well.
> > >
> > > 2d819a08a1cbc11364e36f816b02e33e8dcc030b introduced buildin locale
> > > provider and added overrides to LC_COLLATE and LC_TYPE. But it did not
> > > override other LC_, which I think it should have. In pure upgrade
> > > test, the upgraded node inherits the locale settings of the original
> > > cluster, so this wasn't apparent. But with pg_dump testing, the
> > > original and restored databases are independent. Hence I think we have
> > > to override all LC_* settings by explicitly mentioning --lc-* options
> > > to initdb. Please let me know what you think about this?
> > >
>
> Investigated this further. The problem is that the pg_regress run
> creates regression database with specific properties but the restored
> database does not have those properties. That led me to a better
> solution. Additionally it's local to the new test. Use --create when
> dumping and restoring the regression database. This way the database
> properties or "configuration variable settings (as pg_dump
> documentation calls them) are copied to the restored database as well.
> Those properties include LC_MONETARY. Additionally now the test covers
> --create option as well.
>
> PFA patches.
Will it help the execution time if we use --jobs in case of pg_dump
and pg_restore wherever supported:
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [ 'psql', '-d', 'postgres',
'-f', $dump_file ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '--create',
+ '-d', 'postgres', $dump_file
+ ];
+ }
Should the copyright be only 2025 in this case:
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm
b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..74b9a60cf34
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,167 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
Regards,
Vignesh
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-20 16:39 Alvaro Herrera <[email protected]>
parent: vignesh C <[email protected]>
1 sibling, 2 replies; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-20 16:39 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-20, vignesh C wrote:
> Will it help the execution time if we use --jobs in case of pg_dump
> and pg_restore wherever supported:
As I said in another thread, I think we should enable this test to run
without requiring any PG_TEST_EXTRA, because otherwise the only way to
know about problems is to commit a patch and wait for buildfarm to run
it. Furthermore, I think running all 4 dump format modes is a waste of
time; there isn't any extra coverage by running this test in additional
formats.
Putting those two thoughts together with yours about running with -j,
I propose that what we should do is make this test use -Fc with no
compression (to avoid wasting CPU on that) and use a lowish -j value for
both pg_dump and pg_restore, probably 2, or 3 at most. (Not more,
because this is likely to run in parallel with other tests anyway.)
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"No renuncies a nada. No te aferres a nada."
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 11:51 Ashutosh Bapat <[email protected]>
parent: vignesh C <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-21 11:51 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 20, 2025 at 8:37 PM vignesh C <[email protected]> wrote:
>
> Will it help the execution time if we use --jobs in case of pg_dump
> and pg_restore wherever supported:
> + $src_node->command_ok(
> + [
> + 'pg_dump', "-F$format", '--no-sync',
> + '-d', $src_node->connstr('regression'),
> + '--create', '-f', $dump_file
> + ],
> + "pg_dump on source instance in $format format");
> +
> + my @restore_command;
> + if ($format eq 'plain')
> + {
> + # Restore dump in "plain" format using `psql`.
> + @restore_command = [ 'psql', '-d', 'postgres',
> '-f', $dump_file ];
> + }
> + else
> + {
> + @restore_command = [
> + 'pg_restore', '--create',
> + '-d', 'postgres', $dump_file
> + ];
> + }
Will reply to this separately along with reply to Alvaro's comments.
>
> Should the copyright be only 2025 in this case:
> diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm
> b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
> new file mode 100644
> index 00000000000..74b9a60cf34
> --- /dev/null
> +++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
> @@ -0,0 +1,167 @@
> +
> +# Copyright (c) 2024-2025, PostgreSQL Global Development Group
The patch was posted in 2024 to this mailing list. So we better
protect the copyright since then. I remember a hackers discussion
where a senior member of the community mentioned that there's not harm
in mentioning longer copyright periods than being stricter about it. I
couldn't find the discussion though.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 12:34 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-21 12:34 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-21, Ashutosh Bapat wrote:
> On Thu, Mar 20, 2025 at 8:37 PM vignesh C <[email protected]> wrote:
> > Should the copyright be only 2025 in this case:
> The patch was posted in 2024 to this mailing list. So we better
> protect the copyright since then. I remember a hackers discussion
> where a senior member of the community mentioned that there's not harm
> in mentioning longer copyright periods than being stricter about it. I
> couldn't find the discussion though.
On the other hand, my impression is that we do update copyright years to
current year, when committing new files of patches that have been around
for long.
And there's always
https://liferay.dev/blogs/-/blogs/how-and-why-to-properly-write-copyright-statements-in-your-code
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Las cosas son buenas o malas segun las hace nuestra opinión" (Lisias)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 12:41 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-21 12:41 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 21, 2025 at 6:04 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-21, Ashutosh Bapat wrote:
>
> > On Thu, Mar 20, 2025 at 8:37 PM vignesh C <[email protected]> wrote:
>
> > > Should the copyright be only 2025 in this case:
>
> > The patch was posted in 2024 to this mailing list. So we better
> > protect the copyright since then. I remember a hackers discussion
> > where a senior member of the community mentioned that there's not harm
> > in mentioning longer copyright periods than being stricter about it. I
> > couldn't find the discussion though.
>
> On the other hand, my impression is that we do update copyright years to
> current year, when committing new files of patches that have been around
> for long.
>
> And there's always
> https://liferay.dev/blogs/-/blogs/how-and-why-to-properly-write-copyright-statements-in-your-code
Right. So shouldn't the copyright notice be 2024-2025 and not just
only 2025? - Next year it will be changed to 2024-2026.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 12:45 vignesh C <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: vignesh C @ 2025-03-21 12:45 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, 20 Mar 2025 at 22:09, Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-20, vignesh C wrote:
>
> > Will it help the execution time if we use --jobs in case of pg_dump
> > and pg_restore wherever supported:
>
> As I said in another thread, I think we should enable this test to run
> without requiring any PG_TEST_EXTRA, because otherwise the only way to
> know about problems is to commit a patch and wait for buildfarm to run
> it. Furthermore, I think running all 4 dump format modes is a waste of
> time; there isn't any extra coverage by running this test in additional
> formats.
+1 for running it in only one of the formats.
Regards,
Vignesh
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 13:09 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-21 13:09 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 20, 2025 at 10:09 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-20, vignesh C wrote:
>
> > Will it help the execution time if we use --jobs in case of pg_dump
> > and pg_restore wherever supported:
>
> As I said in another thread, I think we should enable this test to run
> without requiring any PG_TEST_EXTRA, because otherwise the only way to
> know about problems is to commit a patch and wait for buildfarm to run
> it. Furthermore, I think running all 4 dump format modes is a waste of
> time; there isn't any extra coverage by running this test in additional
> formats.
>
> Putting those two thoughts together with yours about running with -j,
> I propose that what we should do is make this test use -Fc with no
> compression (to avoid wasting CPU on that) and use a lowish -j value for
> both pg_dump and pg_restore, probably 2, or 3 at most. (Not more,
> because this is likely to run in parallel with other tests anyway.)
-Fc and -j are not allowed. -j is only allowed for directory format.
$ pg_dump -Fc -j2
pg_dump: error: parallel backup only supported by the directory format
Using just directory format, on my laptop with dev build (because
that's what most developers will use when running the tests)
$ meson test -C $BuildDir pg_upgrade/002_pg_upgrade | grep 002_pg_upgrade
without dump/restore test
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
33.51s 19 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
34.22s 19 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
34.64s 19 subtests passed
without -j, extra ~9 seconds
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
43.33s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
43.25s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
43.10s 28 subtests passed
with -j2, extra 7.5 seconds
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
42.77s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
41.67s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
41.88s 28 subtests passed
with -j3, extra 7 seconds
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
40.77s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
41.05s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
41.28s 28 subtests passed
Between -j2 and -j3 there's not much difference so we could use -j2.
But it still takes 7.5 extra seconds which almost 20% extra time. Do
you think that will be acceptable? I saw somewhere Andres mentioning
that he runs this test quite frequently. Please note that I would very
much like this test to be run by default, but Tom Lane has expressed a
concern about adding even that much time [1] to run the test and
mentioned that he would like the test to be opt-in.
When I started writing the test one year before, people raised
concerns about how useful the test would be. Within a year it has
shown 4 bugs. I have similar feeling about the formats - it's doubtful
now but will prove useful soon especially with the work happening on
dump formats in nearby threads. If we run the test by default, we
could run directory with -j by default and leave other formats as
opt-in OR just forget those formats for now. But If we are going to
make it opt-in, testing all formats gives the extra coverage.
About the format coverage, consensus so far is me and Daniel are for
including all formats when running test as opt-in. Alvaro and Vignesh
are for just one format. We need a tie-breaker or someone amongst us
needs to change their vote :D.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 14:39 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-21 14:39 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
I passed PROVE_FLAGS="--timer -v" to get the timings and run under
--format=directory.
Without new test:
ok 23400 ms ( 0.00 usr 0.00 sys + 2.84 cusr 1.53 csys = 4.37 CPU)
ok 23409 ms ( 0.00 usr 0.01 sys + 2.81 cusr 1.53 csys = 4.35 CPU)
With new test, under --format=directory:
-j2 (parallel, default gzip compression)
ok 27517 ms ( 0.00 usr 0.00 sys + 3.92 cusr 1.86 csys = 5.78 CPU)
ok 27772 ms ( 0.01 usr 0.00 sys + 3.96 cusr 1.86 csys = 5.83 CPU)
ok 27654 ms ( 0.00 usr 0.00 sys + 3.81 cusr 1.94 csys = 5.75 CPU)
ok 27663 ms ( 0.00 usr 0.00 sys + 4.11 cusr 1.71 csys = 5.82 CPU)
-j2 --compress=0
ok 27710 ms ( 0.00 usr 0.00 sys + 3.79 cusr 1.86 csys = 5.65 CPU)
ok 27567 ms ( 0.01 usr 0.00 sys + 3.67 cusr 1.96 csys = 5.64 CPU)
ok 27582 ms ( 0.00 usr 0.00 sys + 3.60 cusr 1.90 csys = 5.50 CPU)
ok 27519 ms ( 0.01 usr 0.00 sys + 3.71 cusr 1.80 csys = 5.52 CPU)
-j2 --compress=zstd
ok 27240 ms ( 0.01 usr 0.00 sys + 3.65 cusr 2.10 csys = 5.76 CPU)
ok 27301 ms ( 0.01 usr 0.00 sys + 3.77 cusr 1.97 csys = 5.75 CPU)
-j2 --compress=zstd:1
ok 27695 ms ( 0.01 usr 0.00 sys + 3.66 cusr 2.05 csys = 5.72 CPU)
ok 27671 ms ( 0.01 usr 0.00 sys + 3.76 cusr 1.95 csys = 5.72 CPU)
--compress=zstd:1 (no parallelism)
ok 28417 ms ( 0.01 usr 0.00 sys + 3.90 cusr 1.75 csys = 5.66 CPU)
ok 28388 ms ( 0.00 usr 0.00 sys + 3.74 cusr 1.81 csys = 5.55 CPU)
--compress=zstd (no parallelism)
ok 28310 ms ( 0.00 usr 0.01 sys + 3.81 cusr 1.83 csys = 5.65 CPU)
ok 28277 ms ( 0.01 usr 0.00 sys + 3.71 cusr 1.87 csys = 5.59 CPU)
So apparently, zstd if available is a bit better than gzip and
parallelism is better than no. But the differences are small -- half a
second or so. The total increase in runtime in the best case is about
four seconds. In all cases I used the same parallelism in pg_restore
than pg_dump; not sure if that could cause a difference.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 16:03 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-21 16:03 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 21, 2025 at 8:13 PM Alvaro Herrera <[email protected]> wrote:
>
> I passed PROVE_FLAGS="--timer -v" to get the timings and run under
> --format=directory.
>
> Without new test:
> ok 23400 ms ( 0.00 usr 0.00 sys + 2.84 cusr 1.53 csys = 4.37 CPU)
> ok 23409 ms ( 0.00 usr 0.01 sys + 2.81 cusr 1.53 csys = 4.35 CPU)
>
>
> With new test, under --format=directory:
> -j2 (parallel, default gzip compression)
> ok 27517 ms ( 0.00 usr 0.00 sys + 3.92 cusr 1.86 csys = 5.78 CPU)
> ok 27772 ms ( 0.01 usr 0.00 sys + 3.96 cusr 1.86 csys = 5.83 CPU)
> ok 27654 ms ( 0.00 usr 0.00 sys + 3.81 cusr 1.94 csys = 5.75 CPU)
> ok 27663 ms ( 0.00 usr 0.00 sys + 4.11 cusr 1.71 csys = 5.82 CPU)
>
> -j2 --compress=0
> ok 27710 ms ( 0.00 usr 0.00 sys + 3.79 cusr 1.86 csys = 5.65 CPU)
> ok 27567 ms ( 0.01 usr 0.00 sys + 3.67 cusr 1.96 csys = 5.64 CPU)
> ok 27582 ms ( 0.00 usr 0.00 sys + 3.60 cusr 1.90 csys = 5.50 CPU)
> ok 27519 ms ( 0.01 usr 0.00 sys + 3.71 cusr 1.80 csys = 5.52 CPU)
>
> -j2 --compress=zstd
> ok 27240 ms ( 0.01 usr 0.00 sys + 3.65 cusr 2.10 csys = 5.76 CPU)
> ok 27301 ms ( 0.01 usr 0.00 sys + 3.77 cusr 1.97 csys = 5.75 CPU)
>
> -j2 --compress=zstd:1
> ok 27695 ms ( 0.01 usr 0.00 sys + 3.66 cusr 2.05 csys = 5.72 CPU)
> ok 27671 ms ( 0.01 usr 0.00 sys + 3.76 cusr 1.95 csys = 5.72 CPU)
>
> --compress=zstd:1 (no parallelism)
> ok 28417 ms ( 0.01 usr 0.00 sys + 3.90 cusr 1.75 csys = 5.66 CPU)
> ok 28388 ms ( 0.00 usr 0.00 sys + 3.74 cusr 1.81 csys = 5.55 CPU)
>
> --compress=zstd (no parallelism)
> ok 28310 ms ( 0.00 usr 0.01 sys + 3.81 cusr 1.83 csys = 5.65 CPU)
> ok 28277 ms ( 0.01 usr 0.00 sys + 3.71 cusr 1.87 csys = 5.59 CPU)
>
>
> So apparently, zstd if available is a bit better than gzip and
> parallelism is better than no. But the differences are small -- half a
> second or so. The total increase in runtime in the best case is about
> four seconds. In all cases I used the same parallelism in pg_restore
> than pg_dump; not sure if that could cause a difference.
I used the same parallelism in pg_restore and pg_dump too. And your
numbers seem to be similar to mine; slightly less than 20% slowdown.
But is that slowdown acceptable? From the earlier discussions, it
seems the answer is No. Haven't heard otherwise.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-21 18:07 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-21 18:07 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-21, Ashutosh Bapat wrote:
> I used the same parallelism in pg_restore and pg_dump too. And your
> numbers seem to be similar to mine; slightly less than 20% slowdown.
> But is that slowdown acceptable? From the earlier discussions, it
> seems the answer is No. Haven't heard otherwise.
I don't think we need to see slowdown this in relative terms, the way we
would discuss a change in the executor. This is not a change that
would affect user-level stuff in any way. We need to see it in absolute
terms: in machines similar to mine, the pg_upgrade test would go from
taking 23s to taking 27s. This is 4s slower, but this isn't an increase
in total test runtime, because decently run test suites run multiple
tests in parallel. This is the same that Peter said in [1]. The total
test runtime change might not be *that* large. I'll take a few numbers
and report back.
[1] https://postgr.es/m/[email protected]
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"I love the Postgres community. It's all about doing things _properly_. :-)"
(David Garamond)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-24 09:54 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-24 09:54 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 21, 2025 at 11:38 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-21, Ashutosh Bapat wrote:
>
> > I used the same parallelism in pg_restore and pg_dump too. And your
> > numbers seem to be similar to mine; slightly less than 20% slowdown.
> > But is that slowdown acceptable? From the earlier discussions, it
> > seems the answer is No. Haven't heard otherwise.
>
> I don't think we need to see slowdown this in relative terms, the way we
> would discuss a change in the executor. This is not a change that
> would affect user-level stuff in any way. We need to see it in absolute
> terms: in machines similar to mine, the pg_upgrade test would go from
> taking 23s to taking 27s. This is 4s slower, but this isn't an increase
> in total test runtime, because decently run test suites run multiple
> tests in parallel. This is the same that Peter said in [1]. The total
> test runtime change might not be *that* large. I'll take a few numbers
> and report back.
Using -j2 in pg_dump and -j3 in pg_restore does not improve timing
much on my laptop. I have used -j2 for both pg_dump and restore
instead of -j3 so as to avoid using more cores when tests are run in
parallel.
Further to reduce run time, I tried -1/--single-transaction but that's
not allowed with --create. I also tried --transaction-size=1000 but
that doesn't affect the run time of the test. Next I thought of using
standard output and input instead of files but it doesn't help since
1. directory format cannot use those and it's the only format allowing
parallelism, 2. that's slower than using files with --no-sync. Didn't
find any other way which can help us reduce the test time.
Please note that the dumps taken for comparison cannot use -j since
they are required to be in "plain" format so that text manipulation
comparison works on them.
One concern I have with directory format is the dumped database is not
readable. This might make investigating a but identified the test a
bit more complex. But I guess, in such a case investigator can either
use the dumps taken for comparison or change the code to use plain
format for investigation. So it's a price we pay for making test
faster.
Here's next patchset:
0001 - it's the same 0001 patch as previous one, includes the test
with all formats and also the PG_TEST_EXTRA option
0002 - removes PG_TEST_EXTRA and also tests only one format
--directory with -j2 with default compression. It should be merged
into 0001 before committing. This is a separate patch for now in case
we decide to go back to 0001.
0003 - same as 0002 in the previous patch set. It excludes statistics
from comparison, otherwise the test will fail because of bug reported
at [1]. Ideally we shouldn't commit this patch so as to test
statistics dump and restore, but in case we need the test to pass till
the bug is fixed, we should merge this patch to 0001 before
committing.
[1] https://www.postgresql.org/message-id/[email protected]...
--
Best Wishes,
Ashutosh Bapat
Attachments:
[text/x-patch] 0001-Test-pg_dump-restore-of-regression-objects-20250324.patch (16.5K, ../../CAExHW5vw_KaZrjWSNJx-QHF12D4KCmV=AAii3Zh3RHmY43gesw@mail.gmail.com/2-0001-Test-pg_dump-restore-of-regression-objects-20250324.patch)
download | inline diff:
From fcfd0d25ecd374d55970817b4d3ea2aecdd58251 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Thu, 27 Jun 2024 10:03:53 +0530
Subject: [PATCH 1/3] Test pg_dump/restore of regression objects
002_pg_upgrade.pl tests pg_upgrade of the regression database left
behind by regression run. Modify it to test dump and restore of the
regression database as well.
Regression database created by regression run contains almost all the
database objects supported by PostgreSQL in various states. Hence the
new testcase covers dump and restore scenarios not covered by individual
dump/restore cases. Till now 002_pg_upgrade only tested dump/restore
through pg_upgrade which only uses binary mode. Many regression tests
mention that they leave objects behind for dump/restore testing but they
are not tested in a non-binary mode. The new testcase closes that
gap.
Testing dump and restore of regression database makes this test run
longer for a relatively smaller benefit. Hence run it only when
explicitly requested by user by specifying "regress_dump_test" in
PG_TEST_EXTRA.
Note For the reviewers:
The new test has uncovered many bugs so far in one year.
1. Introduced by 14e87ffa5c54. Fixed in fd41ba93e4630921a72ed5127cd0d552a8f3f8fc.
2. Introduced by 0413a556990ba628a3de8a0b58be020fd9a14ed0. Reverted in 74563f6b90216180fc13649725179fc119dddeb5.
3. Fixed by d611f8b1587b8f30caa7c0da99ae5d28e914d54f
3. Being discussed on hackers at https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Author: Ashutosh Bapat
Reviewed by: Michael Pacquire, Daniel Gustafsson, Tom Lane, Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
---
doc/src/sgml/regress.sgml | 12 ++
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 144 ++++++++++++++++-
src/test/perl/Makefile | 2 +
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 167 ++++++++++++++++++++
src/test/perl/meson.build | 1 +
5 files changed, 324 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/AdjustDump.pm
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 0e5e8e8f309..237b974b3ab 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,6 +357,18 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>regress_dump_test</literal></term>
+ <listitem>
+ <para>
+ When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
+ tests dump and restore of regression database left behind by the
+ regression run. Not enabled by default because it is time and resource
+ consuming.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 00051b85035..d08eea6693f 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -12,6 +12,7 @@ use File::Path qw(rmtree);
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::AdjustUpgrade;
+use PostgreSQL::Test::AdjustDump;
use Test::More;
# Can be changed to test the other modes.
@@ -35,8 +36,8 @@ sub generate_db
"created database with ASCII characters from $from_char to $to_char");
}
-# Filter the contents of a dump before its use in a content comparison.
-# This returns the path to the filtered dump.
+# Filter the contents of a dump before its use in a content comparison for
+# upgrade testing. This returns the path to the filtered dump.
sub filter_dump
{
my ($is_old, $old_version, $dump_file) = @_;
@@ -262,6 +263,21 @@ else
}
}
is($rc, 0, 'regression tests pass');
+
+ # Test dump/restore of the objects left behind by regression. Ideally it
+ # should be done in a separate TAP test, but doing it here saves us one full
+ # regression run.
+ #
+ # This step takes several extra seconds and some extra disk space, so
+ # requires an opt-in with the PG_TEST_EXTRA environment variable.
+ #
+ # Do this while the old cluster is running before it is shut down by the
+ # upgrade test.
+ if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
+ {
+ test_regression_dump_restore($oldnode, %node_params);
+ }
}
# Initialize a new node for the upgrade.
@@ -539,4 +555,128 @@ my $dump2_filtered = filter_dump(0, $oldnode->pg_version, $dump2_file);
compare_files($dump1_filtered, $dump2_filtered,
'old and new dumps match after pg_upgrade');
+# Test dump and restore of objects left behind by the regression run.
+#
+# It is expected that regression tests, which create `regression` database, are
+# run on `src_node`, which in turn, is left in running state. A fresh node is
+# created using given `node_params`, which are expected to be the same ones used
+# to create `src_node`, so as to avoid any differences in the databases.
+#
+# Plain dumps from both the nodes are compared to make sure that all the dumped
+# objects are restored faithfully.
+sub test_regression_dump_restore
+{
+ my ($src_node, %node_params) = @_;
+ my $dst_node = PostgreSQL::Test::Cluster->new('dst_node');
+
+ # Make sure that the source and destination nodes have the same version and
+ # do not use custom install paths. In both the cases, the dump files may
+ # require additional adjustments unknown to code here. Do not run this test
+ # in such a case to avoid utilizing the time and resources unnecessarily.
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default installation");
+ return;
+ }
+
+ # Dump the original database for comparison later.
+ my $src_dump =
+ get_dump_for_comparison($src_node, 'regression', 'src_dump', 1);
+
+ # Setup destination database cluster
+ $dst_node->init(%node_params);
+ # Stabilize stats for comparison.
+ $dst_node->append_conf('postgresql.conf', 'autovacuum = off');
+ $dst_node->start;
+
+ # Test all formats one by one.
+ for my $format ('plain', 'tar', 'directory', 'custom')
+ {
+ my $dump_file = "$tempdir/regression_dump.$format";
+ my $restored_db = 'regression_' . $format;
+
+ # Use --create in dump and restore commands so that the restored
+ # database has the same configurable variable settings as the original
+ # database and the plain dumps taken for comparsion do not differ
+ # because of locale changes. Additionally this provides test coverage
+ # for --create option.
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [ 'psql', '-d', 'postgres', '-f', $dump_file ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '--create',
+ '-d', 'postgres', $dump_file
+ ];
+ }
+ $dst_node->command_ok(@restore_command,
+ "restored dump taken in $format format on destination instance");
+
+ my $dst_dump =
+ get_dump_for_comparison($dst_node, 'regression',
+ 'dest_dump.' . $format, 0);
+
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database (using $format format) match"
+ );
+
+ # Rename the restored database so that it is available for debugging in
+ # case the test fails.
+ $dst_node->safe_psql('postgres', "ALTER DATABASE regression RENAME TO $restored_db");
+ }
+}
+
+# Dump database `db` from the given `node` in plain format and adjust it for
+# comparing dumps from the original and the restored database.
+#
+# `file_prefix` is used to create unique names for all dump files so that they
+# remain available for debugging in case the test fails.
+#
+# `adjust_child_columns` is passed to adjust_regress_dumpfile() which actually
+# adjusts the dump output.
+#
+# The name of the file containting adjusted dump is returned.
+sub get_dump_for_comparison
+{
+ my ($node, $db, $file_prefix, $adjust_child_columns) = @_;
+
+ my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
+ my $dump_adjusted = "${dumpfile}_adjusted";
+
+ # Usually we avoid comparing statistics in our tests since it is flaky by
+ # nature. However, if statistics is dumped and restored it is expected to be
+ # restored as it is i.e. the statistics from the original database and that
+ # from the restored database should match. We turn off autovacuum on the
+ # source and the target database to avoid any statistics update during
+ # restore operation. Hence we do not exclude statistics from dump.
+ $node->command_ok(
+ [
+ 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ $dumpfile
+ ],
+ 'dump for comparison succeeded');
+
+ open(my $dh, '>', $dump_adjusted)
+ || die
+ "could not open $dump_adjusted for writing the adjusted dump: $!";
+ print $dh adjust_regress_dumpfile(slurp_file($dumpfile),
+ $adjust_child_columns);
+ close($dh);
+
+ return $dump_adjusted;
+}
+
done_testing();
diff --git a/src/test/perl/Makefile b/src/test/perl/Makefile
index d82fb67540e..def89650ead 100644
--- a/src/test/perl/Makefile
+++ b/src/test/perl/Makefile
@@ -26,6 +26,7 @@ install: all installdirs
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/Cluster.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/BackgroundPsql.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustUpgrade.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustDump.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Version.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
uninstall:
@@ -36,6 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
endif
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..74b9a60cf34
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,167 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::AdjustDump - helper module for dump and restore tests
+
+=head1 SYNOPSIS
+
+ use PostgreSQL::Test::AdjustDump;
+
+ # Adjust contents of dump output file so that dump output from original
+ # regression database and that from the restored regression database match
+ $dump = adjust_regress_dumpfile($dump, $adjust_child_columns);
+
+=head1 DESCRIPTION
+
+C<PostgreSQL::Test::AdjustDump> encapsulates various hacks needed to
+compare the results of dump and restore tests
+
+=cut
+
+package PostgreSQL::Test::AdjustDump;
+
+use strict;
+use warnings FATAL => 'all';
+
+use Exporter 'import';
+use Test::More;
+
+our @EXPORT = qw(
+ adjust_regress_dumpfile
+);
+
+=pod
+
+=head1 ROUTINES
+
+=over
+
+=item $dump = adjust_regress_dumpfile($dump, $adjust_child_columns)
+
+If we take dump of the regression database left behind after running regression
+tests, restore the dump, and take dump of the restored regression database, the
+outputs of both the dumps differ in the following cases. This routine adjusts
+the given dump so that dump outputs from the original and restored database,
+respectively, match.
+
+Case 1: Some regression tests purposefully create child tables in such a way
+that the order of their inherited columns differ from column orders of their
+respective parents. In the restored database, however, the order of their
+inherited columns are same as that of their respective parents. Thus the column
+orders of these child tables in the original database and those in the restored
+database differ, causing difference in the dump outputs. See MergeAttributes()
+and dumpTableSchema() for details. This routine rearranges the column
+declarations in the relevant C<CREATE TABLE... INHERITS> statements in the dump
+file from original database to match those from the restored database. We could,
+instead, adjust the statements in the dump from the restored database to match
+those from original database or adjust both to a canonical order. But we have
+chosen to adjust the statements in the dump from original database for no
+particular reason.
+
+Case 2: When dumping COPY statements the columns are ordered by their attribute
+number by fmtCopyColumnList(). If a column is added to a parent table after a
+child has inherited the parent and the child has its own columns, the attribute
+number of the column changes after restoring the child table. This is because
+when executing the dumped C<CREATE TABLE... INHERITS> statement all the parent
+attributes are created before any child attributes. Thus the order of columns in
+COPY statements dumped from the original and the restored databases,
+respectively, differs. Such tables in regression tests are listed below. It is
+hard to adjust the column order in the COPY statement along with the data. Hence
+we just remove such COPY statements from the dump output.
+
+Additionally the routine adjusts blank and new lines to avoid noise.
+
+Note: Usually we avoid comparing statistics in our tests since it is flaky by
+nature. However, if statistics is dumped and restored it is expected to be
+restored as it is i.e. the statistics from the original database and that from
+the restored database should match. Hence we do not filter statistics from dump,
+if it's dumped.
+
+Arguments:
+
+=over
+
+=item C<dump>: Contents of dump file
+
+=item C<adjust_child_columns>: 1 indicates that the given dump file requires
+adjusting columns in the child tables; usually when the dump is from original
+database. 0 indicates no such adjustment is needed; usually when the dump is
+from restored database.
+
+=back
+
+Returns the adjusted dump text.
+
+=cut
+
+sub adjust_regress_dumpfile
+{
+ my ($dump, $adjust_child_columns) = @_;
+
+ # use Unix newlines
+ $dump =~ s/\r\n/\n/g;
+
+ # Adjust the CREATE TABLE ... INHERITS statements.
+ if ($adjust_child_columns)
+ {
+ my $saved_dump = $dump;
+
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_stored_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_stored_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_virtual_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_virtual_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c1\s\()
+ (\n\s+int_four\sbigint),
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint)/$1$4,$2,$3/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c1 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c2\s\()
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint),
+ (\n\s+int_four\sbigint)/$1$3,$4,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c2 adjustments');
+ }
+
+ # Remove COPY statements with differing column order
+ for my $table (
+ 'public\.b_star', 'public\.c_star',
+ 'public\.cc2', 'public\.d_star',
+ 'public\.e_star', 'public\.f_star',
+ 'public\.renamecolumnanother', 'public\.renamecolumnchild',
+ 'public\.test_type_diff2_c1', 'public\.test_type_diff2_c2',
+ 'public\.test_type_diff_c')
+ {
+ $dump =~ s/^COPY\s$table\s\(.+?^\\\.$//sm;
+ }
+
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
+ return $dump;
+}
+
+=pod
+
+=back
+
+=cut
+
+1;
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
index 58e30f15f9d..492ca571ff8 100644
--- a/src/test/perl/meson.build
+++ b/src/test/perl/meson.build
@@ -14,4 +14,5 @@ install_data(
'PostgreSQL/Test/Cluster.pm',
'PostgreSQL/Test/BackgroundPsql.pm',
'PostgreSQL/Test/AdjustUpgrade.pm',
+ 'PostgreSQL/Test/AdjustDump.pm',
install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
base-commit: 73eba5004a06a744b6b8570e42432b9e9f75997b
--
2.34.1
[text/x-patch] 0003-Do-not-dump-statistics-in-the-file-dumped-f-20250324.patch (2.1K, ../../CAExHW5vw_KaZrjWSNJx-QHF12D4KCmV=AAii3Zh3RHmY43gesw@mail.gmail.com/3-0003-Do-not-dump-statistics-in-the-file-dumped-f-20250324.patch)
download | inline diff:
From 435c659489b34a803675abb65144fab6f0550432 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Tue, 25 Feb 2025 11:42:51 +0530
Subject: [PATCH 3/3] Do not dump statistics in the file dumped for comparison
The dumped and restored statistics of a materialized view may differ as
reported in [1]. Hence do not dump the statistics to avoid differences
in the dump output from the original and restored database.
[1] https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Ashutosh Bapat
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index cbd9831bf9e..abe93a49258 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -630,15 +630,15 @@ sub get_dump_for_comparison
my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
my $dump_adjusted = "${dumpfile}_adjusted";
- # Usually we avoid comparing statistics in our tests since it is flaky by
- # nature. However, if statistics is dumped and restored it is expected to be
- # restored as it is i.e. the statistics from the original database and that
- # from the restored database should match. We turn off autovacuum on the
- # source and the target database to avoid any statistics update during
- # restore operation. Hence we do not exclude statistics from dump.
+ # If statistics is dumped and restored it is expected to be restored as it
+ # is i.e. the statistics from the original database and that from the
+ # restored database should match. We turn off autovacuum on the source and
+ # the target database to avoid any statistics update during restore
+ # operation. But as of now, there are cases when statistics is not being
+ # restored faithfully. Hence for now do not dump statistics.
$node->command_ok(
[
- 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ 'pg_dump', '--no-sync', '--no-statistics', '-d', $node->connstr($db), '-f',
$dumpfile
],
'dump for comparison succeeded');
--
2.34.1
[text/x-patch] 0002-Use-only-one-format-and-make-the-test-run-d-20250324.patch (5.4K, ../../CAExHW5vw_KaZrjWSNJx-QHF12D4KCmV=AAii3Zh3RHmY43gesw@mail.gmail.com/4-0002-Use-only-one-format-and-make-the-test-run-d-20250324.patch)
download | inline diff:
From f26f88364a196dc9589ca451cb54f5e514e3422e Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 24 Mar 2025 11:21:12 +0530
Subject: [PATCH 2/3] Use only one format and make the test run default
According to Alvaro (and I agree with him), the test should be run by
default. Otherwise we get to know about a bug only after buildfarm
animal where it's enabled reports a failure. Further testing only one
format may suffice; since all the formats have shown the same bugs till
now.
If we use --directory format we can use -j which reduces the time taken
by dump/restore test by about 12%.
This patch removes PG_TEST_EXTRA option as well as runs the test only in
directory format with parallelism enabled.
Note for committer: If we decide to accept this change, it should be
merged with the previous commit.
---
doc/src/sgml/regress.sgml | 12 ----
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 76 +++++++++-----------------
2 files changed, 25 insertions(+), 63 deletions(-)
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 237b974b3ab..0e5e8e8f309 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,18 +357,6 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
-
- <varlistentry>
- <term><literal>regress_dump_test</literal></term>
- <listitem>
- <para>
- When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
- tests dump and restore of regression database left behind by the
- regression run. Not enabled by default because it is time and resource
- consuming.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index d08eea6693f..cbd9831bf9e 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -268,16 +268,9 @@ else
# should be done in a separate TAP test, but doing it here saves us one full
# regression run.
#
- # This step takes several extra seconds and some extra disk space, so
- # requires an opt-in with the PG_TEST_EXTRA environment variable.
- #
# Do this while the old cluster is running before it is shut down by the
# upgrade test.
- if ( $ENV{PG_TEST_EXTRA}
- && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
- {
- test_regression_dump_restore($oldnode, %node_params);
- }
+ test_regression_dump_restore($oldnode, %node_params);
}
# Initialize a new node for the upgrade.
@@ -590,53 +583,34 @@ sub test_regression_dump_restore
$dst_node->append_conf('postgresql.conf', 'autovacuum = off');
$dst_node->start;
- # Test all formats one by one.
- for my $format ('plain', 'tar', 'directory', 'custom')
- {
- my $dump_file = "$tempdir/regression_dump.$format";
- my $restored_db = 'regression_' . $format;
-
- # Use --create in dump and restore commands so that the restored
- # database has the same configurable variable settings as the original
- # database and the plain dumps taken for comparsion do not differ
- # because of locale changes. Additionally this provides test coverage
- # for --create option.
- $src_node->command_ok(
- [
- 'pg_dump', "-F$format", '--no-sync',
- '-d', $src_node->connstr('regression'),
- '--create', '-f', $dump_file
- ],
- "pg_dump on source instance in $format format");
+ my $dump_file = "$tempdir/regression.dump";
- my @restore_command;
- if ($format eq 'plain')
- {
- # Restore dump in "plain" format using `psql`.
- @restore_command = [ 'psql', '-d', 'postgres', '-f', $dump_file ];
- }
- else
- {
- @restore_command = [
- 'pg_restore', '--create',
- '-d', 'postgres', $dump_file
- ];
- }
- $dst_node->command_ok(@restore_command,
- "restored dump taken in $format format on destination instance");
+ # Use --create in dump and restore commands so that the restored database
+ # has the same configurable variable settings as the original database so
+ # that the plain dumps taken from both the database taken for comparisong do
+ # not differ because of locale changes. Additionally this provides test
+ # coverage for --create option.
+ #
+ # We use directory format which allows dumping and restoring in parallel to
+ # reduce the test's run time.
+ $src_node->command_ok(
+ [
+ 'pg_dump', '-Fd', '-j2', '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance succeeded");
- my $dst_dump =
- get_dump_for_comparison($dst_node, 'regression',
- 'dest_dump.' . $format, 0);
+ $dst_node->command_ok(
+ [ 'pg_restore', '--create', '-j2', '-d', 'postgres', $dump_file ],
+ "restored dump to destination instance");
- compare_files($src_dump, $dst_dump,
- "dump outputs from original and restored regression database (using $format format) match"
- );
+ my $dst_dump = get_dump_for_comparison($dst_node, 'regression',
+ 'dest_dump', 0);
- # Rename the restored database so that it is available for debugging in
- # case the test fails.
- $dst_node->safe_psql('postgres', "ALTER DATABASE regression RENAME TO $restored_db");
- }
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database match"
+ );
}
# Dump database `db` from the given `node` in plain format and adjust it for
--
2.34.1
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-24 09:59 Daniel Gustafsson <[email protected]>
parent: Ashutosh Bapat <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: Daniel Gustafsson @ 2025-03-24 09:59 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; vignesh C <[email protected]>; Michael Paquier <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
> On 24 Mar 2025, at 10:54, Ashutosh Bapat <[email protected]> wrote:
> 0003 - same as 0002 in the previous patch set. It excludes statistics
> from comparison, otherwise the test will fail because of bug reported
> at [1]. Ideally we shouldn't commit this patch so as to test
> statistics dump and restore, but in case we need the test to pass till
> the bug is fixed, we should merge this patch to 0001 before
> committing.
If the reported bug isn't fixed before feature freeze I think we should commit
this regardless as it has clearly shown value by finding bugs (though perhaps
under PG_TEST_EXTRA or in some disconnected till the bug is fixed to limit the
blast-radius in the buildfarm).
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-24 12:14 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-24 12:14 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-24, Ashutosh Bapat wrote:
> One concern I have with directory format is the dumped database is not
> readable. This might make investigating a but identified the test a
> bit more complex.
Oh, it's readable all right. You just need to use `pg_restore -f-` to
read it. No big deal.
So I ran this a few times:
/usr/bin/time make -j8 -Otarget -C /pgsql/build/master check-world -s PROVE_FLAGS="-c -j6" > /dev/null
commenting out the call to test_regression_dump_restore() to test how
much additional runtime does the new test incur.
With test:
136.95user 116.56system 1:13.23elapsed 346%CPU (0avgtext+0avgdata 250704maxresident)k
4928inputs+55333008outputs (114major+14784937minor)pagefaults 0swaps
138.11user 117.43system 1:15.54elapsed 338%CPU (0avgtext+0avgdata 278592maxresident)k
48inputs+55333464outputs (80major+14794494minor)pagefaults 0swaps
137.05user 113.13system 1:08.19elapsed 366%CPU (0avgtext+0avgdata 279272maxresident)k
48inputs+55330064outputs (83major+14758028minor)pagefaults 0swaps
without the new test:
135.46user 114.55system 1:14.69elapsed 334%CPU (0avgtext+0avgdata 145372maxresident)k
32inputs+55155256outputs (105major+14737549minor)pagefaults 0swaps
135.48user 114.57system 1:09.60elapsed 359%CPU (0avgtext+0avgdata 148224maxresident)k
16inputs+55155432outputs (95major+14749502minor)pagefaults 0swaps
133.76user 113.26system 1:14.92elapsed 329%CPU (0avgtext+0avgdata 148064maxresident)k
48inputs+55154952outputs (84major+14749531minor)pagefaults 0swaps
134.06user 113.83system 1:16.09elapsed 325%CPU (0avgtext+0avgdata 145940maxresident)k
32inputs+55155032outputs (83major+14738602minor)pagefaults 0swaps
The increase in duration here is less than a second.
My conclusion with these numbers is that it's not worth hiding this test
in PG_TEST_EXTRA. If we really wanted to save some total test runtime,
it might be better to write a regress schedule file for
027_stream_regress.pl which only takes the test that emit useful WAL,
rather than all tests.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"The ability of users to misuse tools is, of course, legendary" (David Steele)
https://postgr.es/m/[email protected]
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-25 10:39 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-25 10:39 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-24, Ashutosh Bapat wrote:
>
> > One concern I have with directory format is the dumped database is not
> > readable. This might make investigating a but identified the test a
> > bit more complex.
>
> Oh, it's readable all right. You just need to use `pg_restore -f-` to
> read it. No big deal.
>
>
> So I ran this a few times:
> /usr/bin/time make -j8 -Otarget -C /pgsql/build/master check-world -s PROVE_FLAGS="-c -j6" > /dev/null
>
> commenting out the call to test_regression_dump_restore() to test how
> much additional runtime does the new test incur.
>
> With test:
>
> 136.95user 116.56system 1:13.23elapsed 346%CPU (0avgtext+0avgdata 250704maxresident)k
> 4928inputs+55333008outputs (114major+14784937minor)pagefaults 0swaps
>
> 138.11user 117.43system 1:15.54elapsed 338%CPU (0avgtext+0avgdata 278592maxresident)k
> 48inputs+55333464outputs (80major+14794494minor)pagefaults 0swaps
>
> 137.05user 113.13system 1:08.19elapsed 366%CPU (0avgtext+0avgdata 279272maxresident)k
> 48inputs+55330064outputs (83major+14758028minor)pagefaults 0swaps
>
> without the new test:
>
> 135.46user 114.55system 1:14.69elapsed 334%CPU (0avgtext+0avgdata 145372maxresident)k
> 32inputs+55155256outputs (105major+14737549minor)pagefaults 0swaps
>
> 135.48user 114.57system 1:09.60elapsed 359%CPU (0avgtext+0avgdata 148224maxresident)k
> 16inputs+55155432outputs (95major+14749502minor)pagefaults 0swaps
>
> 133.76user 113.26system 1:14.92elapsed 329%CPU (0avgtext+0avgdata 148064maxresident)k
> 48inputs+55154952outputs (84major+14749531minor)pagefaults 0swaps
>
> 134.06user 113.83system 1:16.09elapsed 325%CPU (0avgtext+0avgdata 145940maxresident)k
> 32inputs+55155032outputs (83major+14738602minor)pagefaults 0swaps
>
> The increase in duration here is less than a second.
>
>
> My conclusion with these numbers is that it's not worth hiding this test
> in PG_TEST_EXTRA.
Thanks for the conclusion.
On Mon, Mar 24, 2025 at 3:29 PM Daniel Gustafsson <[email protected]> wrote:
>
> > On 24 Mar 2025, at 10:54, Ashutosh Bapat <[email protected]> wrote:
>
> > 0003 - same as 0002 in the previous patch set. It excludes statistics
> > from comparison, otherwise the test will fail because of bug reported
> > at [1]. Ideally we shouldn't commit this patch so as to test
> > statistics dump and restore, but in case we need the test to pass till
> > the bug is fixed, we should merge this patch to 0001 before
> > committing.
>
> If the reported bug isn't fixed before feature freeze I think we should commit
> this regardless as it has clearly shown value by finding bugs (though perhaps
> under PG_TEST_EXTRA or in some disconnected till the bug is fixed to limit the
> blast-radius in the buildfarm).
Combining Alvaro's and Daniel's recommendations, I think we should
squash all the three of my patches while committing the test if the
bug is not fixed by then. Otherwise we should squash first two patches
and commit it. Just attaching the patches again for reference.
> If we really wanted to save some total test runtime,
> it might be better to write a regress schedule file for
> 027_stream_regress.pl which only takes the test that emit useful WAL,
> rather than all tests.
That's out of scope for this patch, but it seems like an idea worth exploring.
--
Best Wishes,
Ashutosh Bapat
Attachments:
[text/x-patch] 0002-Use-only-one-format-and-make-the-test-run-d-20250324.patch (5.4K, ../../CAExHW5usOKfi-Q1jSi5F50H1mMykAsCayKWEXsES7QKtmwdxtA@mail.gmail.com/2-0002-Use-only-one-format-and-make-the-test-run-d-20250324.patch)
download | inline diff:
From f26f88364a196dc9589ca451cb54f5e514e3422e Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 24 Mar 2025 11:21:12 +0530
Subject: [PATCH 2/3] Use only one format and make the test run default
According to Alvaro (and I agree with him), the test should be run by
default. Otherwise we get to know about a bug only after buildfarm
animal where it's enabled reports a failure. Further testing only one
format may suffice; since all the formats have shown the same bugs till
now.
If we use --directory format we can use -j which reduces the time taken
by dump/restore test by about 12%.
This patch removes PG_TEST_EXTRA option as well as runs the test only in
directory format with parallelism enabled.
Note for committer: If we decide to accept this change, it should be
merged with the previous commit.
---
doc/src/sgml/regress.sgml | 12 ----
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 76 +++++++++-----------------
2 files changed, 25 insertions(+), 63 deletions(-)
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 237b974b3ab..0e5e8e8f309 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,18 +357,6 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
-
- <varlistentry>
- <term><literal>regress_dump_test</literal></term>
- <listitem>
- <para>
- When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
- tests dump and restore of regression database left behind by the
- regression run. Not enabled by default because it is time and resource
- consuming.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index d08eea6693f..cbd9831bf9e 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -268,16 +268,9 @@ else
# should be done in a separate TAP test, but doing it here saves us one full
# regression run.
#
- # This step takes several extra seconds and some extra disk space, so
- # requires an opt-in with the PG_TEST_EXTRA environment variable.
- #
# Do this while the old cluster is running before it is shut down by the
# upgrade test.
- if ( $ENV{PG_TEST_EXTRA}
- && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
- {
- test_regression_dump_restore($oldnode, %node_params);
- }
+ test_regression_dump_restore($oldnode, %node_params);
}
# Initialize a new node for the upgrade.
@@ -590,53 +583,34 @@ sub test_regression_dump_restore
$dst_node->append_conf('postgresql.conf', 'autovacuum = off');
$dst_node->start;
- # Test all formats one by one.
- for my $format ('plain', 'tar', 'directory', 'custom')
- {
- my $dump_file = "$tempdir/regression_dump.$format";
- my $restored_db = 'regression_' . $format;
-
- # Use --create in dump and restore commands so that the restored
- # database has the same configurable variable settings as the original
- # database and the plain dumps taken for comparsion do not differ
- # because of locale changes. Additionally this provides test coverage
- # for --create option.
- $src_node->command_ok(
- [
- 'pg_dump', "-F$format", '--no-sync',
- '-d', $src_node->connstr('regression'),
- '--create', '-f', $dump_file
- ],
- "pg_dump on source instance in $format format");
+ my $dump_file = "$tempdir/regression.dump";
- my @restore_command;
- if ($format eq 'plain')
- {
- # Restore dump in "plain" format using `psql`.
- @restore_command = [ 'psql', '-d', 'postgres', '-f', $dump_file ];
- }
- else
- {
- @restore_command = [
- 'pg_restore', '--create',
- '-d', 'postgres', $dump_file
- ];
- }
- $dst_node->command_ok(@restore_command,
- "restored dump taken in $format format on destination instance");
+ # Use --create in dump and restore commands so that the restored database
+ # has the same configurable variable settings as the original database so
+ # that the plain dumps taken from both the database taken for comparisong do
+ # not differ because of locale changes. Additionally this provides test
+ # coverage for --create option.
+ #
+ # We use directory format which allows dumping and restoring in parallel to
+ # reduce the test's run time.
+ $src_node->command_ok(
+ [
+ 'pg_dump', '-Fd', '-j2', '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance succeeded");
- my $dst_dump =
- get_dump_for_comparison($dst_node, 'regression',
- 'dest_dump.' . $format, 0);
+ $dst_node->command_ok(
+ [ 'pg_restore', '--create', '-j2', '-d', 'postgres', $dump_file ],
+ "restored dump to destination instance");
- compare_files($src_dump, $dst_dump,
- "dump outputs from original and restored regression database (using $format format) match"
- );
+ my $dst_dump = get_dump_for_comparison($dst_node, 'regression',
+ 'dest_dump', 0);
- # Rename the restored database so that it is available for debugging in
- # case the test fails.
- $dst_node->safe_psql('postgres', "ALTER DATABASE regression RENAME TO $restored_db");
- }
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database match"
+ );
}
# Dump database `db` from the given `node` in plain format and adjust it for
--
2.34.1
[text/x-patch] 0001-Test-pg_dump-restore-of-regression-objects-20250324.patch (16.5K, ../../CAExHW5usOKfi-Q1jSi5F50H1mMykAsCayKWEXsES7QKtmwdxtA@mail.gmail.com/3-0001-Test-pg_dump-restore-of-regression-objects-20250324.patch)
download | inline diff:
From fcfd0d25ecd374d55970817b4d3ea2aecdd58251 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Thu, 27 Jun 2024 10:03:53 +0530
Subject: [PATCH 1/3] Test pg_dump/restore of regression objects
002_pg_upgrade.pl tests pg_upgrade of the regression database left
behind by regression run. Modify it to test dump and restore of the
regression database as well.
Regression database created by regression run contains almost all the
database objects supported by PostgreSQL in various states. Hence the
new testcase covers dump and restore scenarios not covered by individual
dump/restore cases. Till now 002_pg_upgrade only tested dump/restore
through pg_upgrade which only uses binary mode. Many regression tests
mention that they leave objects behind for dump/restore testing but they
are not tested in a non-binary mode. The new testcase closes that
gap.
Testing dump and restore of regression database makes this test run
longer for a relatively smaller benefit. Hence run it only when
explicitly requested by user by specifying "regress_dump_test" in
PG_TEST_EXTRA.
Note For the reviewers:
The new test has uncovered many bugs so far in one year.
1. Introduced by 14e87ffa5c54. Fixed in fd41ba93e4630921a72ed5127cd0d552a8f3f8fc.
2. Introduced by 0413a556990ba628a3de8a0b58be020fd9a14ed0. Reverted in 74563f6b90216180fc13649725179fc119dddeb5.
3. Fixed by d611f8b1587b8f30caa7c0da99ae5d28e914d54f
3. Being discussed on hackers at https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Author: Ashutosh Bapat
Reviewed by: Michael Pacquire, Daniel Gustafsson, Tom Lane, Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
---
doc/src/sgml/regress.sgml | 12 ++
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 144 ++++++++++++++++-
src/test/perl/Makefile | 2 +
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 167 ++++++++++++++++++++
src/test/perl/meson.build | 1 +
5 files changed, 324 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/AdjustDump.pm
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 0e5e8e8f309..237b974b3ab 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,6 +357,18 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>regress_dump_test</literal></term>
+ <listitem>
+ <para>
+ When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
+ tests dump and restore of regression database left behind by the
+ regression run. Not enabled by default because it is time and resource
+ consuming.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 00051b85035..d08eea6693f 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -12,6 +12,7 @@ use File::Path qw(rmtree);
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::AdjustUpgrade;
+use PostgreSQL::Test::AdjustDump;
use Test::More;
# Can be changed to test the other modes.
@@ -35,8 +36,8 @@ sub generate_db
"created database with ASCII characters from $from_char to $to_char");
}
-# Filter the contents of a dump before its use in a content comparison.
-# This returns the path to the filtered dump.
+# Filter the contents of a dump before its use in a content comparison for
+# upgrade testing. This returns the path to the filtered dump.
sub filter_dump
{
my ($is_old, $old_version, $dump_file) = @_;
@@ -262,6 +263,21 @@ else
}
}
is($rc, 0, 'regression tests pass');
+
+ # Test dump/restore of the objects left behind by regression. Ideally it
+ # should be done in a separate TAP test, but doing it here saves us one full
+ # regression run.
+ #
+ # This step takes several extra seconds and some extra disk space, so
+ # requires an opt-in with the PG_TEST_EXTRA environment variable.
+ #
+ # Do this while the old cluster is running before it is shut down by the
+ # upgrade test.
+ if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
+ {
+ test_regression_dump_restore($oldnode, %node_params);
+ }
}
# Initialize a new node for the upgrade.
@@ -539,4 +555,128 @@ my $dump2_filtered = filter_dump(0, $oldnode->pg_version, $dump2_file);
compare_files($dump1_filtered, $dump2_filtered,
'old and new dumps match after pg_upgrade');
+# Test dump and restore of objects left behind by the regression run.
+#
+# It is expected that regression tests, which create `regression` database, are
+# run on `src_node`, which in turn, is left in running state. A fresh node is
+# created using given `node_params`, which are expected to be the same ones used
+# to create `src_node`, so as to avoid any differences in the databases.
+#
+# Plain dumps from both the nodes are compared to make sure that all the dumped
+# objects are restored faithfully.
+sub test_regression_dump_restore
+{
+ my ($src_node, %node_params) = @_;
+ my $dst_node = PostgreSQL::Test::Cluster->new('dst_node');
+
+ # Make sure that the source and destination nodes have the same version and
+ # do not use custom install paths. In both the cases, the dump files may
+ # require additional adjustments unknown to code here. Do not run this test
+ # in such a case to avoid utilizing the time and resources unnecessarily.
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default installation");
+ return;
+ }
+
+ # Dump the original database for comparison later.
+ my $src_dump =
+ get_dump_for_comparison($src_node, 'regression', 'src_dump', 1);
+
+ # Setup destination database cluster
+ $dst_node->init(%node_params);
+ # Stabilize stats for comparison.
+ $dst_node->append_conf('postgresql.conf', 'autovacuum = off');
+ $dst_node->start;
+
+ # Test all formats one by one.
+ for my $format ('plain', 'tar', 'directory', 'custom')
+ {
+ my $dump_file = "$tempdir/regression_dump.$format";
+ my $restored_db = 'regression_' . $format;
+
+ # Use --create in dump and restore commands so that the restored
+ # database has the same configurable variable settings as the original
+ # database and the plain dumps taken for comparsion do not differ
+ # because of locale changes. Additionally this provides test coverage
+ # for --create option.
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [ 'psql', '-d', 'postgres', '-f', $dump_file ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '--create',
+ '-d', 'postgres', $dump_file
+ ];
+ }
+ $dst_node->command_ok(@restore_command,
+ "restored dump taken in $format format on destination instance");
+
+ my $dst_dump =
+ get_dump_for_comparison($dst_node, 'regression',
+ 'dest_dump.' . $format, 0);
+
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database (using $format format) match"
+ );
+
+ # Rename the restored database so that it is available for debugging in
+ # case the test fails.
+ $dst_node->safe_psql('postgres', "ALTER DATABASE regression RENAME TO $restored_db");
+ }
+}
+
+# Dump database `db` from the given `node` in plain format and adjust it for
+# comparing dumps from the original and the restored database.
+#
+# `file_prefix` is used to create unique names for all dump files so that they
+# remain available for debugging in case the test fails.
+#
+# `adjust_child_columns` is passed to adjust_regress_dumpfile() which actually
+# adjusts the dump output.
+#
+# The name of the file containting adjusted dump is returned.
+sub get_dump_for_comparison
+{
+ my ($node, $db, $file_prefix, $adjust_child_columns) = @_;
+
+ my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
+ my $dump_adjusted = "${dumpfile}_adjusted";
+
+ # Usually we avoid comparing statistics in our tests since it is flaky by
+ # nature. However, if statistics is dumped and restored it is expected to be
+ # restored as it is i.e. the statistics from the original database and that
+ # from the restored database should match. We turn off autovacuum on the
+ # source and the target database to avoid any statistics update during
+ # restore operation. Hence we do not exclude statistics from dump.
+ $node->command_ok(
+ [
+ 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ $dumpfile
+ ],
+ 'dump for comparison succeeded');
+
+ open(my $dh, '>', $dump_adjusted)
+ || die
+ "could not open $dump_adjusted for writing the adjusted dump: $!";
+ print $dh adjust_regress_dumpfile(slurp_file($dumpfile),
+ $adjust_child_columns);
+ close($dh);
+
+ return $dump_adjusted;
+}
+
done_testing();
diff --git a/src/test/perl/Makefile b/src/test/perl/Makefile
index d82fb67540e..def89650ead 100644
--- a/src/test/perl/Makefile
+++ b/src/test/perl/Makefile
@@ -26,6 +26,7 @@ install: all installdirs
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/Cluster.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/BackgroundPsql.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustUpgrade.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustDump.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Version.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
uninstall:
@@ -36,6 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
endif
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..74b9a60cf34
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,167 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::AdjustDump - helper module for dump and restore tests
+
+=head1 SYNOPSIS
+
+ use PostgreSQL::Test::AdjustDump;
+
+ # Adjust contents of dump output file so that dump output from original
+ # regression database and that from the restored regression database match
+ $dump = adjust_regress_dumpfile($dump, $adjust_child_columns);
+
+=head1 DESCRIPTION
+
+C<PostgreSQL::Test::AdjustDump> encapsulates various hacks needed to
+compare the results of dump and restore tests
+
+=cut
+
+package PostgreSQL::Test::AdjustDump;
+
+use strict;
+use warnings FATAL => 'all';
+
+use Exporter 'import';
+use Test::More;
+
+our @EXPORT = qw(
+ adjust_regress_dumpfile
+);
+
+=pod
+
+=head1 ROUTINES
+
+=over
+
+=item $dump = adjust_regress_dumpfile($dump, $adjust_child_columns)
+
+If we take dump of the regression database left behind after running regression
+tests, restore the dump, and take dump of the restored regression database, the
+outputs of both the dumps differ in the following cases. This routine adjusts
+the given dump so that dump outputs from the original and restored database,
+respectively, match.
+
+Case 1: Some regression tests purposefully create child tables in such a way
+that the order of their inherited columns differ from column orders of their
+respective parents. In the restored database, however, the order of their
+inherited columns are same as that of their respective parents. Thus the column
+orders of these child tables in the original database and those in the restored
+database differ, causing difference in the dump outputs. See MergeAttributes()
+and dumpTableSchema() for details. This routine rearranges the column
+declarations in the relevant C<CREATE TABLE... INHERITS> statements in the dump
+file from original database to match those from the restored database. We could,
+instead, adjust the statements in the dump from the restored database to match
+those from original database or adjust both to a canonical order. But we have
+chosen to adjust the statements in the dump from original database for no
+particular reason.
+
+Case 2: When dumping COPY statements the columns are ordered by their attribute
+number by fmtCopyColumnList(). If a column is added to a parent table after a
+child has inherited the parent and the child has its own columns, the attribute
+number of the column changes after restoring the child table. This is because
+when executing the dumped C<CREATE TABLE... INHERITS> statement all the parent
+attributes are created before any child attributes. Thus the order of columns in
+COPY statements dumped from the original and the restored databases,
+respectively, differs. Such tables in regression tests are listed below. It is
+hard to adjust the column order in the COPY statement along with the data. Hence
+we just remove such COPY statements from the dump output.
+
+Additionally the routine adjusts blank and new lines to avoid noise.
+
+Note: Usually we avoid comparing statistics in our tests since it is flaky by
+nature. However, if statistics is dumped and restored it is expected to be
+restored as it is i.e. the statistics from the original database and that from
+the restored database should match. Hence we do not filter statistics from dump,
+if it's dumped.
+
+Arguments:
+
+=over
+
+=item C<dump>: Contents of dump file
+
+=item C<adjust_child_columns>: 1 indicates that the given dump file requires
+adjusting columns in the child tables; usually when the dump is from original
+database. 0 indicates no such adjustment is needed; usually when the dump is
+from restored database.
+
+=back
+
+Returns the adjusted dump text.
+
+=cut
+
+sub adjust_regress_dumpfile
+{
+ my ($dump, $adjust_child_columns) = @_;
+
+ # use Unix newlines
+ $dump =~ s/\r\n/\n/g;
+
+ # Adjust the CREATE TABLE ... INHERITS statements.
+ if ($adjust_child_columns)
+ {
+ my $saved_dump = $dump;
+
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_stored_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_stored_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_virtual_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_virtual_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c1\s\()
+ (\n\s+int_four\sbigint),
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint)/$1$4,$2,$3/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c1 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c2\s\()
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint),
+ (\n\s+int_four\sbigint)/$1$3,$4,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c2 adjustments');
+ }
+
+ # Remove COPY statements with differing column order
+ for my $table (
+ 'public\.b_star', 'public\.c_star',
+ 'public\.cc2', 'public\.d_star',
+ 'public\.e_star', 'public\.f_star',
+ 'public\.renamecolumnanother', 'public\.renamecolumnchild',
+ 'public\.test_type_diff2_c1', 'public\.test_type_diff2_c2',
+ 'public\.test_type_diff_c')
+ {
+ $dump =~ s/^COPY\s$table\s\(.+?^\\\.$//sm;
+ }
+
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
+ return $dump;
+}
+
+=pod
+
+=back
+
+=cut
+
+1;
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
index 58e30f15f9d..492ca571ff8 100644
--- a/src/test/perl/meson.build
+++ b/src/test/perl/meson.build
@@ -14,4 +14,5 @@ install_data(
'PostgreSQL/Test/Cluster.pm',
'PostgreSQL/Test/BackgroundPsql.pm',
'PostgreSQL/Test/AdjustUpgrade.pm',
+ 'PostgreSQL/Test/AdjustDump.pm',
install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
base-commit: 73eba5004a06a744b6b8570e42432b9e9f75997b
--
2.34.1
[text/x-patch] 0003-Do-not-dump-statistics-in-the-file-dumped-f-20250324.patch (2.1K, ../../CAExHW5usOKfi-Q1jSi5F50H1mMykAsCayKWEXsES7QKtmwdxtA@mail.gmail.com/4-0003-Do-not-dump-statistics-in-the-file-dumped-f-20250324.patch)
download | inline diff:
From 435c659489b34a803675abb65144fab6f0550432 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Tue, 25 Feb 2025 11:42:51 +0530
Subject: [PATCH 3/3] Do not dump statistics in the file dumped for comparison
The dumped and restored statistics of a materialized view may differ as
reported in [1]. Hence do not dump the statistics to avoid differences
in the dump output from the original and restored database.
[1] https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Ashutosh Bapat
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index cbd9831bf9e..abe93a49258 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -630,15 +630,15 @@ sub get_dump_for_comparison
my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
my $dump_adjusted = "${dumpfile}_adjusted";
- # Usually we avoid comparing statistics in our tests since it is flaky by
- # nature. However, if statistics is dumped and restored it is expected to be
- # restored as it is i.e. the statistics from the original database and that
- # from the restored database should match. We turn off autovacuum on the
- # source and the target database to avoid any statistics update during
- # restore operation. Hence we do not exclude statistics from dump.
+ # If statistics is dumped and restored it is expected to be restored as it
+ # is i.e. the statistics from the original database and that from the
+ # restored database should match. We turn off autovacuum on the source and
+ # the target database to avoid any statistics update during restore
+ # operation. But as of now, there are cases when statistics is not being
+ # restored faithfully. Hence for now do not dump statistics.
$node->command_ok(
[
- 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ 'pg_dump', '--no-sync', '--no-statistics', '-d', $node->connstr($db), '-f',
$dumpfile
],
'dump for comparison succeeded');
--
2.34.1
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-27 11:30 Nazir Bilal Yavuz <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Nazir Bilal Yavuz @ 2025-03-27 11:30 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
Hi,
Thank you for working on this!
On Wed, 19 Mar 2025 at 12:06, Jakub Wartak
<[email protected]> wrote:
>
> On Tue, Mar 18, 2025 at 3:29 PM Bertrand Drouvot
> <[email protected]> wrote:
>
> Hi! v15 attached, rebased, CI-tested, all fixed incorporated.
This needs to be rebased after 8eadd5c73c.
--
Regards,
Nazir Bilal Yavuz
Microsoft
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-27 12:31 vignesh C <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: vignesh C @ 2025-03-27 12:31 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, 25 Mar 2025 at 16:09, Ashutosh Bapat
<[email protected]> wrote:
>
> On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera <[email protected]> wrote:
> >
> > On 2025-Mar-24, Ashutosh Bapat wrote:
> >
> > > One concern I have with directory format is the dumped database is not
> > > readable. This might make investigating a but identified the test a
> > > bit more complex.
> >
> > Oh, it's readable all right. You just need to use `pg_restore -f-` to
> > read it. No big deal.
> >
> >
> > So I ran this a few times:
> > /usr/bin/time make -j8 -Otarget -C /pgsql/build/master check-world -s PROVE_FLAGS="-c -j6" > /dev/null
> >
> > commenting out the call to test_regression_dump_restore() to test how
> > much additional runtime does the new test incur.
> >
> > With test:
> >
> > 136.95user 116.56system 1:13.23elapsed 346%CPU (0avgtext+0avgdata 250704maxresident)k
> > 4928inputs+55333008outputs (114major+14784937minor)pagefaults 0swaps
> >
> > 138.11user 117.43system 1:15.54elapsed 338%CPU (0avgtext+0avgdata 278592maxresident)k
> > 48inputs+55333464outputs (80major+14794494minor)pagefaults 0swaps
> >
> > 137.05user 113.13system 1:08.19elapsed 366%CPU (0avgtext+0avgdata 279272maxresident)k
> > 48inputs+55330064outputs (83major+14758028minor)pagefaults 0swaps
> >
> > without the new test:
> >
> > 135.46user 114.55system 1:14.69elapsed 334%CPU (0avgtext+0avgdata 145372maxresident)k
> > 32inputs+55155256outputs (105major+14737549minor)pagefaults 0swaps
> >
> > 135.48user 114.57system 1:09.60elapsed 359%CPU (0avgtext+0avgdata 148224maxresident)k
> > 16inputs+55155432outputs (95major+14749502minor)pagefaults 0swaps
> >
> > 133.76user 113.26system 1:14.92elapsed 329%CPU (0avgtext+0avgdata 148064maxresident)k
> > 48inputs+55154952outputs (84major+14749531minor)pagefaults 0swaps
> >
> > 134.06user 113.83system 1:16.09elapsed 325%CPU (0avgtext+0avgdata 145940maxresident)k
> > 32inputs+55155032outputs (83major+14738602minor)pagefaults 0swaps
> >
> > The increase in duration here is less than a second.
> >
> >
> > My conclusion with these numbers is that it's not worth hiding this test
> > in PG_TEST_EXTRA.
>
> Thanks for the conclusion.
>
> On Mon, Mar 24, 2025 at 3:29 PM Daniel Gustafsson <[email protected]> wrote:
> >
> > > On 24 Mar 2025, at 10:54, Ashutosh Bapat <[email protected]> wrote:
> >
> > > 0003 - same as 0002 in the previous patch set. It excludes statistics
> > > from comparison, otherwise the test will fail because of bug reported
> > > at [1]. Ideally we shouldn't commit this patch so as to test
> > > statistics dump and restore, but in case we need the test to pass till
> > > the bug is fixed, we should merge this patch to 0001 before
> > > committing.
> >
> > If the reported bug isn't fixed before feature freeze I think we should commit
> > this regardless as it has clearly shown value by finding bugs (though perhaps
> > under PG_TEST_EXTRA or in some disconnected till the bug is fixed to limit the
> > blast-radius in the buildfarm).
>
> Combining Alvaro's and Daniel's recommendations, I think we should
> squash all the three of my patches while committing the test if the
> bug is not fixed by then. Otherwise we should squash first two patches
> and commit it. Just attaching the patches again for reference.
Couple of minor thoughts:
1) I felt this error message is not conveying the error message correctly:
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default
installation");
+ return;
+ }
how about something like below:
fail("source and destination nodes must have the same PostgreSQL
version and default installation paths");
2) Should "`" be ' or " here, we generally use "`" to enclose commands:
+# It is expected that regression tests, which create `regression` database, are
+# run on `src_node`, which in turn, is left in running state. A fresh node is
+# created using given `node_params`, which are expected to be the
same ones used
+# to create `src_node`, so as to avoid any differences in the databases.
There are few other instances similarly in the file.
Regards,
Vignesh
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-27 13:02 Jakub Wartak <[email protected]>
parent: Nazir Bilal Yavuz <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: Jakub Wartak @ 2025-03-27 13:02 UTC (permalink / raw)
To: Nazir Bilal Yavuz <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
On Thu, Mar 27, 2025 at 12:31 PM Nazir Bilal Yavuz <[email protected]> wrote:
>
> Hi,
>
> Thank you for working on this!
>
> On Wed, 19 Mar 2025 at 12:06, Jakub Wartak
> <[email protected]> wrote:
> >
> > On Tue, Mar 18, 2025 at 3:29 PM Bertrand Drouvot
> > <[email protected]> wrote:
> >
> > Hi! v15 attached, rebased, CI-tested, all fixed incorporated.
>
> This needs to be rebased after 8eadd5c73c.
Hi Nazir, thanks for spotting! I've not connected the dots with AIO
going in and my libnuma dependency blowing up... Attached is rebased
v16 that passed my CI run.
-J.
Attachments:
[application/octet-stream] v16-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (18.9K, ../../CAKZiRmzQpQzgooRSMwXDj+zRzz3xu_aGSBsJKFwH9yPxaxeJ=A@mail.gmail.com/2-v16-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From d091528cd990ea84980441bf82064d5b884c6786 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v16 1/4] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper. Other platforms can be added later.
This also adds function pg_numa_available() that can be used to check if
the server was linked with NUMA support.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway on i386 it is very memory
limited platform even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 12 +-
configure | 87 ++++++++++++++
configure.ac | 13 +++
doc/src/sgml/func.sgml | 13 +++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 23 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 1 +
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 46 ++++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 168 ++++++++++++++++++++++++++++
17 files changed, 397 insertions(+), 5 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 86a1fa9bbdb..e6963c774aa 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -445,8 +445,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libnuma1 \
+ libnuma-dev
matrix:
# SPECIAL:
@@ -471,6 +473,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
--with-liburing \
\
${LINUX_CONFIGURE_FEATURES} \
@@ -523,6 +526,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
@@ -839,8 +843,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libnuma1 libnuma-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/configure b/configure
index 4dd67a5cc6e..81e43a38331 100755
--- a/configure
+++ b/configure
@@ -711,6 +711,7 @@ with_libxml
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
+with_libnuma
with_uuid
LIBURING_LIBS
LIBURING_CFLAGS
@@ -872,6 +873,7 @@ with_liburing
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -1588,6 +1590,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma support
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -9279,6 +9282,33 @@ fi
+#
+# NUMA
+#
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
@@ -12517,6 +12547,63 @@ fi
fi
+if test "$with_libnuma" = yes ; then
+
+ ac_fn_c_check_header_mongrel "$LINENO" "numa.h" "ac_cv_header_numa_h" "$ac_includes_default"
+if test "x$ac_cv_header_numa_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <numa.h> is required for --with-libnuma" "$LINENO" 5
+fi
+
+
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'numa' does not provide numa_available" "$LINENO" 5
+fi
+
+fi
+
# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
# dependency on that platform?
diff --git a/configure.ac b/configure.ac
index 537e654e7b3..1879baf183a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1053,6 +1053,19 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5bf6656deca..1f98826d16d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25138,6 +25138,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if the server has been compiled with <acronym>NUMA</acronym> support.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index cc28f041330..5f0486bb335 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-liburing">
<term><option>--with-liburing</option></term>
<listitem>
@@ -2645,6 +2655,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index 187f1787a3c..52c4f3c1022 100644
--- a/meson.build
+++ b/meson.build
@@ -943,6 +943,27 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ # via pkg-config
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if not cc.has_header('numa.h', dependencies: libnuma, required: libnumaopt)
+ libnuma = not_found_dep
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: liburing
@@ -3177,6 +3198,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
liburing,
libxml,
lz4,
@@ -3833,6 +3855,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'liburing': liburing,
'libxml': libxml,
'libxslt': libxslt,
diff --git a/meson_options.txt b/meson_options.txt
index dd7126da3a7..8675e1b5d87 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('liburing', type : 'feature', value: 'auto',
description: 'io_uring support, for asynchronous I/O')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index cce29a37ac5..71479ad9018 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -196,6 +196,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_liburing = @with_liburing@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 989825d3a9c..a80616d4455 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -566,7 +566,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8b68b16d79d..d532b8c43b9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8506,6 +8506,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '9685', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index c6f055b3905..424d42b14f8 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -675,6 +675,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to build with io_uring support. (--with-liburing) */
#undef USE_LIBURING
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..986152e0942
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,46 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "c.h"
+#include "postgres.h"
+#include "fmgr.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+extern PGDLLIMPORT Datum pg_numa_available(PG_FUNCTION_ARGS);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+extern void numa_warn(int num, char *fmt,...) pg_attribute_printf(2, 3);
+extern void numa_error(char *where);
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 46d8da070e8..55da678ec27 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -200,6 +200,8 @@ pgxs_empty = [
'ICU_LIBS',
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS',
+
'LIBURING_CFLAGS', 'LIBURING_LIBS',
]
@@ -232,6 +234,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'liburing': liburing,
'libxml': libxml,
'libxslt': libxslt,
diff --git a/src/port/Makefile b/src/port/Makefile
index 7843d7b67cb..8c8e6b92910 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -45,6 +45,7 @@ OBJS = \
path.o \
pg_bitutils.o \
pg_localeconv_r.o \
+ pg_numa.o \
pg_popcount_avx512.o \
pg_strong_random.o \
pgcheckdir.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index 653539ba5b3..1eb8e38d047 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -8,6 +8,7 @@ pgport_sources = [
'path.c',
'pg_bitutils.c',
'pg_localeconv_r.c',
+ 'pg_numa.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
'pgcheckdir.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..7d905ef31f5
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "fmgr.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
+
+#ifndef FRONTEND
+/*
+ * XXX: not really tested as there is no way to trigger this in our
+ * current usage of libnuma.
+ *
+ * The libnuma built-in code can be seen here:
+ * https://github.com/numactl/numactl/blob/master/libnuma.c
+ *
+ */
+void
+numa_warn(int num, char *fmt,...)
+{
+ va_list ap;
+ int olde = errno;
+ int needed;
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+
+ va_start(ap, fmt);
+ needed = appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ if (needed > 0)
+ {
+ enlargeStringInfo(&msg, needed);
+ va_start(ap, fmt);
+ appendStringInfoVA(&msg, fmt, ap);
+ va_end(ap);
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+ errmsg_internal("libnuma: WARNING: %s", msg.data)));
+
+ pfree(msg.data);
+
+ errno = olde;
+}
+
+void
+numa_error(char *where)
+{
+ int olde = errno;
+
+ /*
+ * XXX: for now we issue just WARNING, but long-term that might depend on
+ * numa_set_strict() here.
+ */
+ elog(WARNING, "libnuma: ERROR: %s", where);
+ errno = olde;
+}
+#endif /* FRONTEND */
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+Size
+pg_numa_get_pagesize(void)
+{
+#ifndef WIN32
+ Size os_page_size = sysconf(_SC_PAGESIZE);
+#else
+ Size os_page_size;
+ SYSTEM_INFO sysinfo;
+
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#endif
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+ return os_page_size;
+}
+
+#endif
+
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_BOOL(pg_numa_init() != -1);
+}
--
2.39.5
[application/octet-stream] v16-0002-This-extracts-code-from-contrib-pg_buffercache-s.patch (13.6K, ../../CAKZiRmzQpQzgooRSMwXDj+zRzz3xu_aGSBsJKFwH9yPxaxeJ=A@mail.gmail.com/3-v16-0002-This-extracts-code-from-contrib-pg_buffercache-s.patch)
download | inline diff:
From fde52bfc05470076753dcb3e38a846ef3f6defe9 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Wed, 19 Mar 2025 09:34:56 +0100
Subject: [PATCH v16 2/4] This extracts code from contrib/pg_buffercache's
primary function to separate functions.
This commit adds pg_buffercache_init_entries(), pg_buffercache_build_tuple()
and get_buffercache_tuple() that help fill result tuplestores based on the
buffercache contents. This will be used in a follow-up commit that implements
NUMA observability in pg_buffercache.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/pg_buffercache_pages.c | 317 ++++++++++--------
1 file changed, 178 insertions(+), 139 deletions(-)
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 62602af1775..86e0b8afe01 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -14,7 +14,6 @@
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
#define NUM_BUFFERCACHE_PAGES_ELEM 9
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
@@ -68,80 +67,192 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * This is almost identical to pg_buffercache_numa_pages(), but this one performs
+ * memory mapping inquiries to display NUMA node information for each buffer.
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
- HeapTuple tuple;
- if (SRF_IS_FIRSTCALL())
- {
- int i;
+ /* Switch context when allocating stuff to be used in later calls */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
- funcctx = SRF_FIRSTCALL_INIT();
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. ee unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /* Return to original context when allocating transient memory */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Build buffer cache information for a single buffer.
+ */
+static void
+pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ bufHdr = GetBufferDescriptor(record_id);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ fctx->record[record_id].bufferid = BufferDescriptorGetBuffer(bufHdr);
+ fctx->record[record_id].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ fctx->record[record_id].reltablespace = bufHdr->tag.spcOid;
+ fctx->record[record_id].reldatabase = bufHdr->tag.dbOid;
+ fctx->record[record_id].forknum = BufTagGetForkNum(&bufHdr->tag);
+ fctx->record[record_id].blocknum = bufHdr->tag.blockNum;
+ fctx->record[record_id].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ fctx->record[record_id].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ fctx->record[record_id].isdirty = true;
+ else
+ fctx->record[record_id].isdirty = false;
+
+ /* Note if the buffer is valid, and has storage created */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ fctx->record[record_id].isvalid = true;
+ else
+ fctx->record[record_id].isvalid = false;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Format and return a tuple for a single buffer cache entry.
+ */
+static Datum
+get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
+ HeapTuple tuple;
+
+ values[0] = Int32GetDatum(fctx->record[record_id].bufferid);
+ nulls[0] = false;
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (fctx->record[record_id].blocknum == InvalidBlockNumber ||
+ fctx->record[record_id].isvalid == false)
+ {
+ nulls[1] = true;
+ nulls[2] = true;
+ nulls[3] = true;
+ nulls[4] = true;
+ nulls[5] = true;
+ nulls[6] = true;
+ nulls[7] = true;
+
+ /* unused for v1.0 callers, but the array is always long enough */
+ nulls[8] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(fctx->record[record_id].relfilenumber);
+ nulls[1] = false;
+ values[2] = ObjectIdGetDatum(fctx->record[record_id].reltablespace);
+ nulls[2] = false;
+ values[3] = ObjectIdGetDatum(fctx->record[record_id].reldatabase);
+ nulls[3] = false;
+ values[4] = ObjectIdGetDatum(fctx->record[record_id].forknum);
+ nulls[4] = false;
+ values[5] = Int64GetDatum((int64) fctx->record[record_id].blocknum);
+ nulls[5] = false;
+ values[6] = BoolGetDatum(fctx->record[record_id].isdirty);
+ nulls[6] = false;
+ values[7] = Int16GetDatum(fctx->record[record_id].usagecount);
+ nulls[7] = false;
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ values[8] = Int32GetDatum(fctx->record[record_id].pinning_backends);
+ nulls[8] = false;
+ }
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -152,36 +263,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_build_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -191,59 +273,16 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
-
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
-
- /*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
- */
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
- {
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
- }
- else
- {
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
- }
-
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
--
2.39.5
[application/octet-stream] v16-0003-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (17.5K, ../../CAKZiRmzQpQzgooRSMwXDj+zRzz3xu_aGSBsJKFwH9yPxaxeJ=A@mail.gmail.com/4-v16-0003-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From c9d7598eeaa73741ea1658e88e51cec8858751a0 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Wed, 19 Mar 2025 09:34:56 +0100
Subject: [PATCH v16 3/4] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA memory node for individual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 +++
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 42 +++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 172 +++++++++++++++++-
.../sql/pg_buffercache_numa.sql | 20 ++
doc/src/sgml/pgbuffercache.sgml | 61 ++++++-
9 files changed, 329 insertions(+), 4 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..0f4b2eaf444
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,42 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new function.
+DROP VIEW pg_buffercache;
+DROP FUNCTION pg_buffercache_pages();
+
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, node_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 86e0b8afe01..7abba723c0b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,11 +11,12 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -45,6 +46,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_node_id;
} BufferCachePagesRec;
@@ -63,10 +65,56 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+/* Only need to touch memory once per backend process lifetime */
+static bool firstNumaTouch = true;
+
+/*
+ * Helper routine to map Buffers into addresses that is used by
+ * pg_numa_query_pages().
+ *
+ * When database block size (BLCKSZ) is smaller than the OS page size (4kB),
+ * multiple database buffers will map to the same OS memory page. In this case,
+ * we only need to query the NUMA node for the first memory address of each
+ * unique OS page rather than for every buffer.
+ *
+ * In order to get reliable results we also need to touch memory pages, so that
+ * inquiry about NUMA memory node doesn't return -2 (which indicates
+ * unmapped/unallocated pages).
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk,
+ Size os_page_size,
+ void **os_page_ptrs)
+{
+ size_t blk2page = (size_t) (buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers starts from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) +
+ (os_page_size * j);
+
+ /* Only need to touch memory once per backend process lifetime */
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ }
+
+ CHECK_FOR_INTERRUPTS();
+ }
+}
+
/*
* Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
*
@@ -124,6 +172,9 @@ pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "node_id",
+ INT4OID, -1, 0);
fctx->tupdesc = BlessTupleDesc(tupledesc);
@@ -176,6 +227,8 @@ pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
else
fctx->record[record_id].isvalid = false;
+ fctx->record[record_id].numa_node_id = -1;
+
UnlockBufHdr(bufHdr, buf_state);
}
@@ -211,6 +264,7 @@ get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
/* unused for v1.0 callers, but the array is always long enough */
nulls[8] = true;
+ nulls[9] = true;
}
else
{
@@ -234,6 +288,8 @@ get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
*/
values[8] = Int32GetDatum(fctx->record[record_id].pinning_backends);
nulls[8] = false;
+ values[9] = Int32GetDatum(fctx->record[record_id].numa_node_id);
+ nulls[9] = false;
}
/* Build and return the tuple. */
@@ -285,6 +341,120 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
}
}
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings.
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ uint64 os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
+
+ /*
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used,
+ * while the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: 1. Determine the OS
+ * memory page size 2. Calculate how many OS pages are used by all
+ * buffer blocks 3. Calculate how many OS pages are contained within
+ * each database block.
+ *
+ * This information is needed before calling move_pages() for NUMA
+ * node id inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%lu os_page_size=%zu pages_per_blk=%.2f",
+ (unsigned long) os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc0(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(uint64) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here.
+ *
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+
+ /*
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
+ */
+ for (i = 0; i < NBuffers; i++)
+ {
+ pg_buffercache_build_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size,
+ os_page_ptrs);
+ }
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Set the NUMA node id for this buffer based on the first OS page
+ * it maps to.
+ *
+ * Note: We could check for errors in os_pages_status and report
+ * them. Also, a single DB block might span multiple NUMA nodes if
+ * it crosses OS pages on node boundaries, but we only record the
+ * node of the first page. This is a simplification but should be
+ * sufficient for most analyses.
+ */
+ fctx->record[i].numa_node_id = os_pages_status[blk2page];
+ }
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
+
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ firstNumaTouch = false;
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
Datum
pg_buffercache_summary(PG_FUNCTION_ARGS)
{
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..2225b879f58
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,20 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..086e0062a17 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,14 @@
convenient use.
</para>
+ <para>
+ The <function>pg_buffercache_numa_pages()</function> provides the same information
+ as <function>pg_buffercache_pages()</function> but is slower because it also
+ provides the <acronym>NUMA</acronym> node ID per shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +210,55 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are identical to the
+ <structname>pg_buffercache</structname> view, except that this one includes
+ one additional <structfield>node_id</structfield> column as defined in
+ <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Extra column</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>node_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ <acronym>NUMA</acronym> node ID. NULL if the shared buffer
+ has not been used yet. On systems without <acronym>NUMA</acronym> support
+ this returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ As <acronym>NUMA</acronym> node ID inquiry for each page requires memory pages
+ to be paged-in, the first execution of this function can take a noticeable
+ amount of time. In all the cases (first execution or not), retrieving this
+ information is costly and querying the view at a high frequency is not recommended.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
[application/octet-stream] v16-0004-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch (16.2K, ../../CAKZiRmzQpQzgooRSMwXDj+zRzz3xu_aGSBsJKFwH9yPxaxeJ=A@mail.gmail.com/5-v16-0004-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch)
download | inline diff:
From 0e7a1e8e34aa631d6420656eb5b811a0a07f11d7 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v16 4/4] Add pg_shmem_numa_allocations to show NUMA memory
node for shared memory allocations.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 79 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 131 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 9 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 273 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..e83711f9578 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA node mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,80 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> shows how shared
+ memory allocations in the server's main shared memory segment are distributed
+ across NUMA nodes. This includes both memory allocated by
+ <productname>PostgreSQL</productname> itself and memory allocated
+ by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>node_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of <acronym>NUMA</acronym> node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA memory node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 31d269b7ee0..660a9e9832b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..a011603d318 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port//pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -89,6 +90,8 @@ slock_t *ShmemLock; /* spinlock for shared memory and LWLock
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+/* To get reliable results for NUMA inquiry we need to "touch pages" once */
+static bool firstNumaTouch = true;
/*
* InitShmemAccess() --- set up basic pointers to shared memory.
@@ -568,3 +571,131 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA memory nodes for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ uint64 shm_total_page_count,
+ shm_ent_page_count,
+ max_nodes;
+ Size *nodes;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+ return (Datum) 0;
+ }
+ max_nodes = pg_numa_get_max_node();
+ nodes = palloc(sizeof(Size) * (max_nodes + 1));
+
+ /*
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used, while
+ * the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: 1. Determine the OS memory
+ * page size 2. Calculate how many OS pages are used by all buffer blocks
+ * 3. Calculate how many OS pages are contained within each database
+ * block.
+ *
+ * This information is needed before calling move_pages() for NUMA memory
+ * node inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Allocate memory for page pointers and status based on total shared
+ * memory size. This simplified approach allocates enough space for all
+ * pages in shared memory rather than calculating the exact requirements
+ * for each segment.
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc0(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here.
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages, so that inquiry about NUMA memory node doesn't return -2
+ * (which indicates unmapped/unallocated pages).
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ memset(nodes, 0, sizeof(Size) * (max_nodes + 1));
+ /* Count number of NUMA nodes used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ /* Ensure we are adding only valid index to the array */
+ if (s >= 0 && s <= max_nodes)
+ nodes[s]++;
+ }
+
+ for (i = 0; i <= max_nodes; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(nodes[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /*
+ * We are ignoring the following memory regions (as compared to
+ * pg_get_shmem_allocations()): 1. output shared memory allocated but not
+ * counted via the shmem index 2. output as-of-yet unused shared memory.
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstNumaTouch = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d532b8c43b9..db69c0231f9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8510,6 +8510,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '9686', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,node_id,size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 954f549555e..d9d62470cdc 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 47478969135..6e460e10d61 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ node_id,
+ size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, node_id, size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0a35f2f8f6a..0f38caa0d24 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..fddb21a260a
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,9 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index b81694c24f2..f93d4829702 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-27 13:15 Álvaro Herrera <[email protected]>
parent: Jakub Wartak <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Álvaro Herrera @ 2025-03-27 13:15 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Nazir Bilal Yavuz <[email protected]>; Bertrand Drouvot <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
Hello
I think you should remove numa_warn() and numa_error() from 0001.
AFAICS they are dead code (even with all your patches applied), and
furthermore would get you in trouble regarding memory allocation because
src/port is not allowed to use palloc et al. If you wanted to keep them
you'd have to have them in src/common, but looking at the rest of the
code in that patch, ISTM src/port is the right place for it. If in the
future you discover that you do need numa_warn(), you can create a
src/common/ file for it then.
Is pg_buffercache really the best place for these NUMA introspection
routines? I'm not saying that it isn't, maybe we're okay with that
(particularly if we can avoid duplicated code), but it seems a bit weird
to me.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"No me acuerdo, pero no es cierto. No es cierto, y si fuera cierto,
no me acuerdo." (Augusto Pinochet a una corte de justicia)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-27 13:40 Andres Freund <[email protected]>
parent: Jakub Wartak <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Andres Freund @ 2025-03-27 13:40 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Nazir Bilal Yavuz <[email protected]>; Bertrand Drouvot <[email protected]>; pgsql-hackers
Hi,
On 2025-03-27 14:02:03 +0100, Jakub Wartak wrote:
> setup_additional_packages_script: |
> - #apt-get update
> - #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
> + apt-get update
> + DEBIAN_FRONTEND=noninteractive apt-get -y install \
> + libnuma1 \
> + libnuma-dev
I think libnuma is installed on the relevant platforms, so you shouldn't need
to install it manually.
> +#
> +# libnuma
> +#
> +AC_MSG_CHECKING([whether to build with libnuma support])
> +PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
Most other dependencies say "build with libxyz ..."
> +/*-------------------------------------------------------------------------
> + *
> + * pg_numa.h
> + * Basic NUMA portability routines
> + *
> + *
> + * Copyright (c) 2025, PostgreSQL Global Development Group
> + *
> + * IDENTIFICATION
> + * src/include/port/pg_numa.h
> + *
> + *-------------------------------------------------------------------------
> + */
> +#ifndef PG_NUMA_H
> +#define PG_NUMA_H
> +
> +#include "c.h"
> +#include "postgres.h"
Headers should never include either of those headers. Nor should .c files
include both.
> @@ -200,6 +200,8 @@ pgxs_empty = [
>
> 'ICU_LIBS',
>
> + 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS',
> +
> 'LIBURING_CFLAGS', 'LIBURING_LIBS',
> ]
Maybe I am missing something, but are you actually defining and using those
LIBNUMA_* vars anywhere?
> +Size
> +pg_numa_get_pagesize(void)
> +{
> + Size os_page_size = sysconf(_SC_PAGESIZE);
> +
> + if (huge_pages_status == HUGE_PAGES_ON)
> + GetHugePageSize(&os_page_size, NULL);
> +
> + return os_page_size;
> +}
Should this have a comment or an assertion that it can only be used after
shared memory startup? Because before that huge_pages_status won't be
meaningful?
> +#ifndef FRONTEND
> +/*
> + * XXX: not really tested as there is no way to trigger this in our
> + * current usage of libnuma.
> + *
> + * The libnuma built-in code can be seen here:
> + * https://github.com/numactl/numactl/blob/master/libnuma.c
> + *
> + */
> +void
> +numa_warn(int num, char *fmt,...)
> +{
> + va_list ap;
> + int olde = errno;
> + int needed;
> + StringInfoData msg;
> +
> + initStringInfo(&msg);
> +
> + va_start(ap, fmt);
> + needed = appendStringInfoVA(&msg, fmt, ap);
> + va_end(ap);
> + if (needed > 0)
> + {
> + enlargeStringInfo(&msg, needed);
> + va_start(ap, fmt);
> + appendStringInfoVA(&msg, fmt, ap);
> + va_end(ap);
> + }
> +
> + ereport(WARNING,
> + (errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
> + errmsg_internal("libnuma: WARNING: %s", msg.data)));
I think you would at least have to hold interrupts across this, as
ereport(WARNING) does CHECK_FOR_INTERRUPTS() and it would not be safe to jump
out of libnuma in case an interrupt has arrived.
> +Size
> +pg_numa_get_pagesize(void)
> +{
> +#ifndef WIN32
> + Size os_page_size = sysconf(_SC_PAGESIZE);
> +#else
> + Size os_page_size;
> + SYSTEM_INFO sysinfo;
> +
> + GetSystemInfo(&sysinfo);
> + os_page_size = sysinfo.dwPageSize;
> +#endif
> + if (huge_pages_status == HUGE_PAGES_ON)
> + GetHugePageSize(&os_page_size, NULL);
> + return os_page_size;
> +}
I would probably implement this once, outside of the big ifdef, with one more
ifdef inside, given that you're sharing the same implementation.
> From fde52bfc05470076753dcb3e38a846ef3f6defe9 Mon Sep 17 00:00:00 2001
> From: Jakub Wartak <[email protected]>
> Date: Wed, 19 Mar 2025 09:34:56 +0100
> Subject: [PATCH v16 2/4] This extracts code from contrib/pg_buffercache's
> primary function to separate functions.
>
> This commit adds pg_buffercache_init_entries(), pg_buffercache_build_tuple()
> and get_buffercache_tuple() that help fill result tuplestores based on the
> buffercache contents. This will be used in a follow-up commit that implements
> NUMA observability in pg_buffercache.
>
> Author: Jakub Wartak <[email protected]>
> Reviewed-by: Bertrand Drouvot <[email protected]>
> Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
> ---
> contrib/pg_buffercache/pg_buffercache_pages.c | 317 ++++++++++--------
> 1 file changed, 178 insertions(+), 139 deletions(-)
>
> diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
> index 62602af1775..86e0b8afe01 100644
> --- a/contrib/pg_buffercache/pg_buffercache_pages.c
> +++ b/contrib/pg_buffercache/pg_buffercache_pages.c
> @@ -14,7 +14,6 @@
> #include "storage/buf_internals.h"
> #include "storage/bufmgr.h"
>
> -
> #define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
Independent change.
> #define NUM_BUFFERCACHE_PAGES_ELEM 9
> #define NUM_BUFFERCACHE_SUMMARY_ELEM 5
> @@ -68,80 +67,192 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
> PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
> PG_FUNCTION_INFO_V1(pg_buffercache_evict);
>
> -Datum
> -pg_buffercache_pages(PG_FUNCTION_ARGS)
> +/*
> + * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
> + *
> + * This is almost identical to pg_buffercache_numa_pages(), but this one performs
> + * memory mapping inquiries to display NUMA node information for each buffer.
> + */
If it's a helper routine it's probably not identical to
pg_buffercache_numa_pages()?
> +/*
> + * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
> + *
> + * Build buffer cache information for a single buffer.
> + */
> +static void
> +pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
> +{
This isn't really building a tuple tuple? Seems somewhat confusing, because
get_buffercache_tuple() does actually build one.
> + BufferDesc *bufHdr;
> + uint32 buf_state;
> +
> + bufHdr = GetBufferDescriptor(record_id);
> + /* Lock each buffer header before inspecting. */
> + buf_state = LockBufHdr(bufHdr);
> +
> + fctx->record[record_id].bufferid = BufferDescriptorGetBuffer(bufHdr);
> + fctx->record[record_id].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
> + fctx->record[record_id].reltablespace = bufHdr->tag.spcOid;
> + fctx->record[record_id].reldatabase = bufHdr->tag.dbOid;
> + fctx->record[record_id].forknum = BufTagGetForkNum(&bufHdr->tag);
> + fctx->record[record_id].blocknum = bufHdr->tag.blockNum;
> + fctx->record[record_id].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
> + fctx->record[record_id].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
As above, I think this would be more readable if you put
fctx->record[record_id] into a local var.
> +static Datum
> +get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
> +{
> + Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
> + bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
> + HeapTuple tuple;
> +
> + values[0] = Int32GetDatum(fctx->record[record_id].bufferid);
> + nulls[0] = false;
> +
> + /*
> + * Set all fields except the bufferid to null if the buffer is unused or
> + * not valid.
> + */
> + if (fctx->record[record_id].blocknum == InvalidBlockNumber ||
> + fctx->record[record_id].isvalid == false)
> + {
> + nulls[1] = true;
> + nulls[2] = true;
> + nulls[3] = true;
> + nulls[4] = true;
> + nulls[5] = true;
> + nulls[6] = true;
> + nulls[7] = true;
> +
> + /* unused for v1.0 callers, but the array is always long enough */
> + nulls[8] = true;
I'd probably just memset the entire nulls array to either true or false,
instead of doing it one-by-one.
> + }
> + else
> + {
> + values[1] = ObjectIdGetDatum(fctx->record[record_id].relfilenumber);
> + nulls[1] = false;
> + values[2] = ObjectIdGetDatum(fctx->record[record_id].reltablespace);
> + nulls[2] = false;
> + values[3] = ObjectIdGetDatum(fctx->record[record_id].reldatabase);
> + nulls[3] = false;
> + values[4] = ObjectIdGetDatum(fctx->record[record_id].forknum);
> + nulls[4] = false;
> + values[5] = Int64GetDatum((int64) fctx->record[record_id].blocknum);
> + nulls[5] = false;
> + values[6] = BoolGetDatum(fctx->record[record_id].isdirty);
> + nulls[6] = false;
> + values[7] = Int16GetDatum(fctx->record[record_id].usagecount);
> + nulls[7] = false;
Seems like it would end up a lot more readable if you put
fctx->record[record_id] into a local variable. Unfortunately that'd probably
be best done in one more commit ahead of the rest of the this one...
> @@ -0,0 +1,28 @@
> +SELECT NOT(pg_numa_available()) AS skip_test \gset
> +\if :skip_test
> +\quit
> +\endif
You could avoid the need for an alternative output file if you instead made
the queries do something like
SELECT NOT pg_numa_available() OR count(*) ...
> --- /dev/null
> +++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
> @@ -0,0 +1,42 @@
> +/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
> +
> +-- complain if script is sourced in psql, rather than via CREATE EXTENSION
> +\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
> +
> +-- Register the new function.
> +DROP VIEW pg_buffercache;
> +DROP FUNCTION pg_buffercache_pages();
I don't think we can just drop a view in the upgrade script. That will fail if
anybody created a view depending on pg_buffercache.
(Sorry, ran out of time / energy here, i had originally just wanted to comment
on the apt-get thing in the tests)
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-27 16:31 Ashutosh Bapat <[email protected]>
parent: vignesh C <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-27 16:31 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 27, 2025 at 6:01 PM vignesh C <[email protected]> wrote:
>
> On Tue, 25 Mar 2025 at 16:09, Ashutosh Bapat
> <[email protected]> wrote:
> >
> > On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera <[email protected]> wrote:
> > >
> > > On 2025-Mar-24, Ashutosh Bapat wrote:
> > >
> > > > One concern I have with directory format is the dumped database is not
> > > > readable. This might make investigating a but identified the test a
> > > > bit more complex.
> > >
> > > Oh, it's readable all right. You just need to use `pg_restore -f-` to
> > > read it. No big deal.
> > >
> > >
> > > So I ran this a few times:
> > > /usr/bin/time make -j8 -Otarget -C /pgsql/build/master check-world -s PROVE_FLAGS="-c -j6" > /dev/null
> > >
> > > commenting out the call to test_regression_dump_restore() to test how
> > > much additional runtime does the new test incur.
> > >
> > > With test:
> > >
> > > 136.95user 116.56system 1:13.23elapsed 346%CPU (0avgtext+0avgdata 250704maxresident)k
> > > 4928inputs+55333008outputs (114major+14784937minor)pagefaults 0swaps
> > >
> > > 138.11user 117.43system 1:15.54elapsed 338%CPU (0avgtext+0avgdata 278592maxresident)k
> > > 48inputs+55333464outputs (80major+14794494minor)pagefaults 0swaps
> > >
> > > 137.05user 113.13system 1:08.19elapsed 366%CPU (0avgtext+0avgdata 279272maxresident)k
> > > 48inputs+55330064outputs (83major+14758028minor)pagefaults 0swaps
> > >
> > > without the new test:
> > >
> > > 135.46user 114.55system 1:14.69elapsed 334%CPU (0avgtext+0avgdata 145372maxresident)k
> > > 32inputs+55155256outputs (105major+14737549minor)pagefaults 0swaps
> > >
> > > 135.48user 114.57system 1:09.60elapsed 359%CPU (0avgtext+0avgdata 148224maxresident)k
> > > 16inputs+55155432outputs (95major+14749502minor)pagefaults 0swaps
> > >
> > > 133.76user 113.26system 1:14.92elapsed 329%CPU (0avgtext+0avgdata 148064maxresident)k
> > > 48inputs+55154952outputs (84major+14749531minor)pagefaults 0swaps
> > >
> > > 134.06user 113.83system 1:16.09elapsed 325%CPU (0avgtext+0avgdata 145940maxresident)k
> > > 32inputs+55155032outputs (83major+14738602minor)pagefaults 0swaps
> > >
> > > The increase in duration here is less than a second.
> > >
> > >
> > > My conclusion with these numbers is that it's not worth hiding this test
> > > in PG_TEST_EXTRA.
> >
> > Thanks for the conclusion.
> >
> > On Mon, Mar 24, 2025 at 3:29 PM Daniel Gustafsson <[email protected]> wrote:
> > >
> > > > On 24 Mar 2025, at 10:54, Ashutosh Bapat <[email protected]> wrote:
> > >
> > > > 0003 - same as 0002 in the previous patch set. It excludes statistics
> > > > from comparison, otherwise the test will fail because of bug reported
> > > > at [1]. Ideally we shouldn't commit this patch so as to test
> > > > statistics dump and restore, but in case we need the test to pass till
> > > > the bug is fixed, we should merge this patch to 0001 before
> > > > committing.
> > >
> > > If the reported bug isn't fixed before feature freeze I think we should commit
> > > this regardless as it has clearly shown value by finding bugs (though perhaps
> > > under PG_TEST_EXTRA or in some disconnected till the bug is fixed to limit the
> > > blast-radius in the buildfarm).
> >
> > Combining Alvaro's and Daniel's recommendations, I think we should
> > squash all the three of my patches while committing the test if the
> > bug is not fixed by then. Otherwise we should squash first two patches
> > and commit it. Just attaching the patches again for reference.
>
> Couple of minor thoughts:
> 1) I felt this error message is not conveying the error message correctly:
> + if ($src_node->pg_version != $dst_node->pg_version
> + or defined $src_node->{_install_path})
> + {
> + fail("same version dump and restore test using default
> installation");
> + return;
> + }
>
> how about something like below:
> fail("source and destination nodes must have the same PostgreSQL
> version and default installation paths");
The text in ok(), fail() etc. are test names and not error messages.
See [1]. Your suggestion and other versions that I came up with became
too verbose to be test names. So I think the text here is compromise
between conveying enough information and not being too long. We
usually have to pick the testname and lookup the test code to
investigate the failure. This text serves that purpose.
>
> 2) Should "`" be ' or " here, we generally use "`" to enclose commands:
> +# It is expected that regression tests, which create `regression` database, are
> +# run on `src_node`, which in turn, is left in running state. A fresh node is
> +# created using given `node_params`, which are expected to be the
> same ones used
> +# to create `src_node`, so as to avoid any differences in the databases.
Looking at prologues or some other functions, I see that we don't add
any decoration around the name of the argument. Hence dropped ``
altogether. Will post it with the next set of patches.
[1] https://metacpan.org/pod/Test::More
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-27 17:15 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-27 17:15 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-27, Ashutosh Bapat wrote:
> On Thu, Mar 27, 2025 at 6:01 PM vignesh C <[email protected]> wrote:
> > Couple of minor thoughts:
> > 1) I felt this error message is not conveying the error message correctly:
> > + if ($src_node->pg_version != $dst_node->pg_version
> > + or defined $src_node->{_install_path})
> > + {
> > + fail("same version dump and restore test using default
> > installation");
> > + return;
> > + }
> >
> > how about something like below:
> > fail("source and destination nodes must have the same PostgreSQL
> > version and default installation paths");
>
> The text in ok(), fail() etc. are test names and not error messages.
> See [1]. Your suggestion and other versions that I came up with became
> too verbose to be test names. So I think the text here is compromise
> between conveying enough information and not being too long. We
> usually have to pick the testname and lookup the test code to
> investigate the failure. This text serves that purpose.
Maybe
fail("roundtrip dump/restore of the regression database")
BTW another idea to shorten this tests's runtime might be to try and
identify which of parallel_schedule tests leave objects behind and
create a shorter schedule with only those (a possible implementation
might keep a list of the slow tests that don't leave any useful object
behind, then filter parallel_schedule to exclude those; this ensures
test files created in the future are still used.)
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"I love the Postgres community. It's all about doing things _properly_. :-)"
(David Garamond)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 01:36 Michael Paquier <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Michael Paquier @ 2025-03-28 01:36 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote:
> BTW another idea to shorten this tests's runtime might be to try and
> identify which of parallel_schedule tests leave objects behind and
> create a shorter schedule with only those (a possible implementation
> might keep a list of the slow tests that don't leave any useful object
> behind, then filter parallel_schedule to exclude those; this ensures
> test files created in the future are still used.)
I'm not much a fan of approaches that require an extra schedule,
because this is prone to forget the addition of objects that we'd want
to cover for the scope of this thread with the dump/restore
inter-dependencies, failing our goal of having more coverage. And
history has proven that we are quite bad at maintaining multiple
schedules for the regression test suite (remember the serial one or
the standby one in pg_regress?). So we should really do things so as
the schedules are down to a strict minimum: 1.
If we're worried about the time taken by the test (spoiler: I am and
the upgrade tests already show always as last to finish in parallel
runs), I would recommend to put that under a PG_TEST_EXTRA. I'm OK to
add the switch to my buildfarm animals if this option is the consensus
and if it gets into the tree.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 06:32 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 2 replies; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-28 06:32 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Thu, Mar 27, 2025 at 10:45 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-27, Ashutosh Bapat wrote:
>
> > On Thu, Mar 27, 2025 at 6:01 PM vignesh C <[email protected]> wrote:
>
> > > Couple of minor thoughts:
> > > 1) I felt this error message is not conveying the error message correctly:
> > > + if ($src_node->pg_version != $dst_node->pg_version
> > > + or defined $src_node->{_install_path})
> > > + {
> > > + fail("same version dump and restore test using default
> > > installation");
> > > + return;
> > > + }
> > >
> > > how about something like below:
> > > fail("source and destination nodes must have the same PostgreSQL
> > > version and default installation paths");
> >
> > The text in ok(), fail() etc. are test names and not error messages.
> > See [1]. Your suggestion and other versions that I came up with became
> > too verbose to be test names. So I think the text here is compromise
> > between conveying enough information and not being too long. We
> > usually have to pick the testname and lookup the test code to
> > investigate the failure. This text serves that purpose.
>
> Maybe
> fail("roundtrip dump/restore of the regression database")
No, that's losing some information like default installation and the
same version.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 06:50 Ashutosh Bapat <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-28 06:50 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 28, 2025 at 7:07 AM Michael Paquier <[email protected]> wrote:
>
> On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote:
> > BTW another idea to shorten this tests's runtime might be to try and
> > identify which of parallel_schedule tests leave objects behind and
> > create a shorter schedule with only those (a possible implementation
> > might keep a list of the slow tests that don't leave any useful object
> > behind, then filter parallel_schedule to exclude those; this ensures
> > test files created in the future are still used.)
>
> I'm not much a fan of approaches that require an extra schedule,
> because this is prone to forget the addition of objects that we'd want
> to cover for the scope of this thread with the dump/restore
> inter-dependencies, failing our goal of having more coverage. And
> history has proven that we are quite bad at maintaining multiple
> schedules for the regression test suite (remember the serial one or
> the standby one in pg_regress?). So we should really do things so as
> the schedules are down to a strict minimum: 1.
I see Alvaro's point about using a different and minimal schedule. We
already have 002_pg_upgrade and 027_stream_ as candidates which could
use schedules other than default and avoid wasting CPU cycles.
But I also agree with your opinion that maintaining multiple schedules
is painful and prone to errors.
What we could do is to create the schedule files automatically during
build. The automation script will require to know which file to place
in which schedules. That information could be either part of the sql
file itself or could be in a separate text file. For example, every
SQL file has the following line listing all the schedules that this
SQL file should be part of. E.g.
-- schedules: parallel, serial, upgrade
The automated script looks at every .sql file in a given sql directory
and creates the schedule files containing all the SQL files which had
respective schedules mentioned in their "schedule" annotation. The
automation script would flag SQL files that do not have scheduled
annotation so any new file added won't be missed. However, we will
still miss a SQL file if it wasn't part of a given schedule and later
acquired some changes which required it to be added to a new schedule.
If we go this route, we could make 'make check-tests' better. We could
add another annotation for depends listing all the SQL files that a
given SQL file depends upon. make check-tests would collect all
dependencies, sort them and run all the dependencies as well.
Of course that's out of scope for this patch. We don't have time left
for this in PG 18.
>
> If we're worried about the time taken by the test (spoiler: I am and
> the upgrade tests already show always as last to finish in parallel
> runs), I would recommend to put that under a PG_TEST_EXTRA. I'm OK to
> add the switch to my buildfarm animals if this option is the consensus
> and if it gets into the tree.
I would prefer to run this test by default as Alvaro mentioned
previously. But if that means that we won't get this test committed at
all, I am ok putting it under PG_TEST_EXTRA. (Hence I have kept 0001
and 0002 separate.) But I will be disappointed if the test, which has
unearthed four bugs in a year alone, does not get committed to PG 18
because of this debate.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 09:28 Ashutosh Bapat <[email protected]>
parent: Ashutosh Bapat <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-28 09:28 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Vignesh and Alvaro
On Fri, Mar 28, 2025 at 12:02 PM Ashutosh Bapat
<[email protected]> wrote:
> >
> > Maybe
> > fail("roundtrip dump/restore of the regression database")
>
> No, that's losing some information like default installation and the
> same version.
How about "dump and restore across servers with same PostgreSQL
version using default installation". That's still mouthful but is more
readable.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 10:35 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-28 10:35 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-28, Ashutosh Bapat wrote:
> No, that's losing some information like default installation and the
> same version.
You don't need to preserve such information. This is just a test name.
People looking for more details can grep for the name and they will find
the comments.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Pido que me den el Nobel por razones humanitarias" (Nicanor Parra)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 11:20 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-28 11:20 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Michael Paquier <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 28, 2025 at 4:05 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-28, Ashutosh Bapat wrote:
>
> > No, that's losing some information like default installation and the
> > same version.
>
> You don't need to preserve such information. This is just a test name.
> People looking for more details can grep for the name and they will find
> the comments.
Ok. In that case what's wrong with the testname I have in the patch?
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 12:27 Ashutosh Bapat <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-28 12:27 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 28, 2025 at 12:20 PM Ashutosh Bapat
<[email protected]> wrote:
>
> On Fri, Mar 28, 2025 at 7:07 AM Michael Paquier <[email protected]> wrote:
> >
> > On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote:
> > > BTW another idea to shorten this tests's runtime might be to try and
> > > identify which of parallel_schedule tests leave objects behind and
> > > create a shorter schedule with only those (a possible implementation
> > > might keep a list of the slow tests that don't leave any useful object
> > > behind, then filter parallel_schedule to exclude those; this ensures
> > > test files created in the future are still used.)
> >
> > I'm not much a fan of approaches that require an extra schedule,
> > because this is prone to forget the addition of objects that we'd want
> > to cover for the scope of this thread with the dump/restore
> > inter-dependencies, failing our goal of having more coverage. And
> > history has proven that we are quite bad at maintaining multiple
> > schedules for the regression test suite (remember the serial one or
> > the standby one in pg_regress?). So we should really do things so as
> > the schedules are down to a strict minimum: 1.
>
> I see Alvaro's point about using a different and minimal schedule. We
> already have 002_pg_upgrade and 027_stream_ as candidates which could
> use schedules other than default and avoid wasting CPU cycles.
> But I also agree with your opinion that maintaining multiple schedules
> is painful and prone to errors.
>
> What we could do is to create the schedule files automatically during
> build. The automation script will require to know which file to place
> in which schedules. That information could be either part of the sql
> file itself or could be in a separate text file. For example, every
> SQL file has the following line listing all the schedules that this
> SQL file should be part of. E.g.
>
> -- schedules: parallel, serial, upgrade
>
> The automated script looks at every .sql file in a given sql directory
> and creates the schedule files containing all the SQL files which had
> respective schedules mentioned in their "schedule" annotation. The
> automation script would flag SQL files that do not have scheduled
> annotation so any new file added won't be missed. However, we will
> still miss a SQL file if it wasn't part of a given schedule and later
> acquired some changes which required it to be added to a new schedule.
>
> If we go this route, we could make 'make check-tests' better. We could
> add another annotation for depends listing all the SQL files that a
> given SQL file depends upon. make check-tests would collect all
> dependencies, sort them and run all the dependencies as well.
>
> Of course that's out of scope for this patch. We don't have time left
> for this in PG 18.
I spent several hours today examining each SQL file to decide whether
or not it has "interesting" objects that it leaves behind for
dump/restore test. I came up with attached schedule - which may not be
accurate since I it would require much more time to examine all tests
to get an accurate schedule. But what I have got may be close enough.
With that we could save about 6 seconds on my laptop. If we further
compact the schedule reorganizing the parallel groups we may shave
some more seconds.
no modifications to parallel schedule
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
41.84s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
41.80s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
41.37s 28 subtests passed
with attached modified parallel schedule
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
36.13s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
35.86s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
36.33s 28 subtests passed
1/1 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK
36.02s 28 subtests passed
However, it's a very painful process to come up with the schedule and
more painful and error prone to maintain it. It could take many days
to come up with the right schedule which can become inaccurate the
moment next SQL file is added OR an existing file is modified to
add/drop "interesting" objects.
--
Best Wishes,
Ashutosh Bapat
Attachments:
[application/octet-stream] parallel_schedule_dump_restore (4.6K, ../../CAExHW5teUDXYR+DyoTP=NJw_=gUy1g=bCmVbKP5+UhRW=Nm0qw@mail.gmail.com/2-parallel_schedule_dump_restore)
download
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 14:11 Alvaro Herrera <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-28 14:11 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Michael Paquier <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-28, Ashutosh Bapat wrote:
> However, it's a very painful process to come up with the schedule and
> more painful and error prone to maintain it. It could take many days
> to come up with the right schedule which can become inaccurate the
> moment next SQL file is added OR an existing file is modified to
> add/drop "interesting" objects.
Hmm, I didn't mean that we'd maintain a separate schedule. I meant that
we'd take the existing schedule, then apply some Perl magic to it that
grep-outs the tests that we know to contribute nothing, and generate a
new schedule file dynamically. We don't need to maintain a separate
schedule file.
You're right that if an existing uninteresting test is modified to
create interesting objects, we'd lose coverage of those objects. That
seems a much smaller problem to me. So it's just a matter of doing some
Perl map/grep to generate a new schedule file using the attached
exclusion file.
(For what it's worth, what I did to try to determine which tests to
include, rather than scan each file manually, is to run pg_regress with
"test_setup thetest tablespace", then dump the regression database, and
see if anything is there that's not in the dump when I just with just
"test_setup tablespace". I didn't carry the experiment to completion
though.)
For the future, we could annotate each test as you said, either by
adding a marker on the test file itself, or by adding something next to
its name in the schedule file, so the schedule file could look like:
test: plancache(dump_ignore) limit(stream_ignore) plpgsql copy2
temp(stream_ignore,dump_ignore) domain rangefuncs(stream_ignore)
prepare conversion truncate alter_table
sequence polymorphism rowtypes returning largeobject with xml
... and so on.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
# This file lists tests to skip on pg_upgrade/t/002_pg_upgrade.pl
advisory_lock
amutils
async
bitmapops
char
combocid
comments
copy
copy2
copydml
copyencoding
copyselect
database
dbsize
delete
drop_if_exists
drop_operator
equivclass
errors
explain
expressions
functional_deps
groupingsets
guc
hash_func
horology
incremental_sort
infinite_recurse
join_hash
json_encoding
jsonb_jsonpath
jsonpath
jsonpath_encoding
lock
md5
memoize
merge
misc_sanity
mvcc
oidjoins
opr_sanity
partition_aggregate
partition_join
partition_prune
plancache
portals
portals_p2
predicate
prepare
prepared_xacts
psql
psql_crosstab
psql_pipeline
regex
regproc
returning
sanity_check
select
select_distinct
select_distinct_on
select_having
select_implicit
select_parallel
stats_import
strings
subselect
sysviews
tablesample
temp
text
tid
tidrangescan
tidscan
transactions
truncate
tsrf
tstypes
txid
type_sanity
unicode
union
update
vacuum
vacuum_parallel
window
xmlmap
Attachments:
[text/plain] dump_roundtrip_exclude (940B, ../../[email protected]/2-dump_roundtrip_exclude)
download | inline:
# This file lists tests to skip on pg_upgrade/t/002_pg_upgrade.pl
advisory_lock
amutils
async
bitmapops
char
combocid
comments
copy
copy2
copydml
copyencoding
copyselect
database
dbsize
delete
drop_if_exists
drop_operator
equivclass
errors
explain
expressions
functional_deps
groupingsets
guc
hash_func
horology
incremental_sort
infinite_recurse
join_hash
json_encoding
jsonb_jsonpath
jsonpath
jsonpath_encoding
lock
md5
memoize
merge
misc_sanity
mvcc
oidjoins
opr_sanity
partition_aggregate
partition_join
partition_prune
plancache
portals
portals_p2
predicate
prepare
prepared_xacts
psql
psql_crosstab
psql_pipeline
regex
regproc
returning
sanity_check
select
select_distinct
select_distinct_on
select_having
select_implicit
select_parallel
stats_import
strings
subselect
sysviews
tablesample
temp
text
tid
tidrangescan
tidscan
transactions
truncate
tsrf
tstypes
txid
type_sanity
unicode
union
update
vacuum
vacuum_parallel
window
xmlmap
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 14:22 Tom Lane <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Tom Lane @ 2025-03-28 14:22 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Alvaro Herrera <[email protected]> writes:
> Hmm, I didn't mean that we'd maintain a separate schedule. I meant that
> we'd take the existing schedule, then apply some Perl magic to it that
> grep-outs the tests that we know to contribute nothing, and generate a
> new schedule file dynamically. We don't need to maintain a separate
> schedule file.
This seems like a fundamentally broken approach to me.
The entire argument for using the core regression tests as a source of
data to test dump/restore is that, more or less "for free", we can
expect to get coverage when new SQL language features are added.
That's always been a little bit questionable --- there's a temptation
to drop objects again at the end of a test script. But with this,
it becomes a complete crapshoot whether the objects you need will be
included in the dump.
I think instead of going this direction, we really need to create a
separately-purposed script that simply creates "one of everything"
without doing anything else (except maybe loading a little data).
I believe it'd be a lot easier to remember to add to that when
inventing new SQL than to remember to leave something behind from the
core regression tests. This would also be far faster to run than any
approach that involves picking a random subset of the core test
scripts.
regards, tom lane
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-28 18:12 Alvaro Herrera <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: Alvaro Herrera @ 2025-03-28 18:12 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Mar-28, Tom Lane wrote:
> I think instead of going this direction, we really need to create a
> separately-purposed script that simply creates "one of everything"
> without doing anything else (except maybe loading a little data).
> I believe it'd be a lot easier to remember to add to that when
> inventing new SQL than to remember to leave something behind from the
> core regression tests. This would also be far faster to run than any
> approach that involves picking a random subset of the core test
> scripts.
FWIW this sounds closely related to what I tried to do with
src/test/modules/test_ddl_deparse; it's currently incomplete, but maybe
we can use that as a starting point.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Always assume the user will do much worse than the stupidest thing
you can imagine." (Julien PUYDT)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-31 09:27 Jakub Wartak <[email protected]>
parent: Álvaro Herrera <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Jakub Wartak @ 2025-03-31 09:27 UTC (permalink / raw)
To: Álvaro Herrera <[email protected]>; +Cc: Nazir Bilal Yavuz <[email protected]>; Bertrand Drouvot <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
On Thu, Mar 27, 2025 at 2:15 PM Álvaro Herrera <[email protected]> wrote:
> Hello
Good morning :)
> I think you should remove numa_warn() and numa_error() from 0001.
> AFAICS they are dead code (even with all your patches applied), and
> furthermore would get you in trouble regarding memory allocation because
> src/port is not allowed to use palloc et al. If you wanted to keep them
> you'd have to have them in src/common, but looking at the rest of the
> code in that patch, ISTM src/port is the right place for it. If in the
> future you discover that you do need numa_warn(), you can create a
> src/common/ file for it then.
Understood, trimmed it out from the patch. I'm going to respond also
within minutes to Andres' review and I'm going to post a new version
(v17) there.
> Is pg_buffercache really the best place for these NUMA introspection
> routines? I'm not saying that it isn't, maybe we're okay with that
> (particularly if we can avoid duplicated code), but it seems a bit weird
> to me.
I think it is, because as I understand, Andres wanted to have
observability per single database *page* and to avoid code duplication
we are just putting it there (it's natural fit). Imagine looking up an
8kB root btree memory page being hit hard from CPUs on other NUMA
nodes (this just gives ability to see that, but you could of course
also get aggregation to get e.g. NUMA node balance for single relation
and so on).
-J.
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-31 09:27 Jakub Wartak <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Jakub Wartak @ 2025-03-31 09:27 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Alvaro Herrera <[email protected]>; +Cc: Nazir Bilal Yavuz <[email protected]>; Bertrand Drouvot <[email protected]>; pgsql-hackers
On Thu, Mar 27, 2025 at 2:40 PM Andres Freund <[email protected]> wrote:
>
> Hi,
Hi Andres,
> On 2025-03-27 14:02:03 +0100, Jakub Wartak wrote:
> > setup_additional_packages_script: |
> > - #apt-get update
> > - #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
> > + apt-get update
> > + DEBIAN_FRONTEND=noninteractive apt-get -y install \
> > + libnuma1 \
> > + libnuma-dev
>
> I think libnuma is installed on the relevant platforms, so you shouldn't need
> to install it manually.
Fixed. Right, you mentioned this earlier, I just didnt know when it went online.
> > +#
> > +# libnuma
> > +#
> > +AC_MSG_CHECKING([whether to build with libnuma support])
> > +PGAC_ARG_BOOL(with, libnuma, no, [use libnuma for NUMA awareness],
>
> Most other dependencies say "build with libxyz ..."
Done.
> > + * pg_numa.h
[..]
> > +#include "c.h"
> > +#include "postgres.h"
>
> Headers should never include either of those headers. Nor should .c files
> include both.
Fixed, huh, I've found explanation:
https://www.postgresql.org/message-id/[email protected]
> > @@ -200,6 +200,8 @@ pgxs_empty = [
> >
> > 'ICU_LIBS',
> >
> > + 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS',
> > +
> > 'LIBURING_CFLAGS', 'LIBURING_LIBS',
> > ]
>
> Maybe I am missing something, but are you actually defining and using those
> LIBNUMA_* vars anywhere?
OK, so it seems I've been missing `PKG_CHECK_MODULES(LIBNUMA, numa)`
in configure.ac that would set those *FLAGS. I'm little bit loss
dependent in how to gurantee that meson is synced with autoconf as per
project requirements - trying to use past commits as reference, but I
still could get something wrong here (especially in
src/Makefile.global.in)
> > +Size
> > +pg_numa_get_pagesize(void)
[..]
>
> Should this have a comment or an assertion that it can only be used after
> shared memory startup? Because before that huge_pages_status won't be
> meaningful?
Added both.
> > +#ifndef FRONTEND
> > +/*
> > + * XXX: not really tested as there is no way to trigger this in our
> > + * current usage of libnuma.
> > + *
> > + * The libnuma built-in code can be seen here:
> > + * https://github.com/numactl/numactl/blob/master/libnuma.c
> > + *
> > + */
> > +void
> > +numa_warn(int num, char *fmt,...)
[..]
>
> I think you would at least have to hold interrupts across this, as
> ereport(WARNING) does CHECK_FOR_INTERRUPTS() and it would not be safe to jump
> out of libnuma in case an interrupt has arrived.
On request by Alvaro I've removed it as that code is simply
unreachable/untestable, but point taken - I'm planning to re-add this
with holding interrupting in future when we start using proper
numa_interleave() one day. Anyway, please let me know if you want
still to keep it as deadcode. BTW for context , why this is deadcode
is explained in the latter part of [1] message (TL;DR; unless we use
pining/numa_interleave/local_alloc() we probably never reach that
warnings/error handlers).
"Another question without an easy answer as I never hit this error from
numa_move_pages(), one gets invalid stuff in *os_pages_status instead.
BUT!: most of our patch just uses things that cannot fail as per
libnuma usage. One way to trigger libnuma warnings is e.g. `chmod 700
/sys` (because it's hard to unmount it) and then still most of numactl
stuff works as euid != 0, but numactl --hardware gets at least
"libnuma: Warning: Cannot parse distance information in sysfs:
Permission denied" or same story with numactl -C 678 date. So unless
we start way more heavy use of libnuma (not just for observability)
there's like no point in that right now (?) Contrary to that: we can
do just do variadic elog() for that, I've put some code, but no idea
if that works fine..."
> > +Size
> > +pg_numa_get_pagesize(void)
[..]
>
> I would probably implement this once, outside of the big ifdef, with one more
> ifdef inside, given that you're sharing the same implementation.
Done.
> > From fde52bfc05470076753dcb3e38a846ef3f6defe9 Mon Sep 17 00:00:00 2001
> > From: Jakub Wartak <[email protected]>
> > Date: Wed, 19 Mar 2025 09:34:56 +0100
> > Subject: [PATCH v16 2/4] This extracts code from contrib/pg_buffercache's
> > primary function to separate functions.
> >
> > This commit adds pg_buffercache_init_entries(), pg_buffercache_build_tuple()
> > and get_buffercache_tuple() that help fill result tuplestores based on the
> > buffercache contents. This will be used in a follow-up commit that implements
> > NUMA observability in pg_buffercache.
> >
> > Author: Jakub Wartak <[email protected]>
> > Reviewed-by: Bertrand Drouvot <[email protected]>
> > Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
> > ---
> > contrib/pg_buffercache/pg_buffercache_pages.c | 317 ++++++++++--------
> > 1 file changed, 178 insertions(+), 139 deletions(-)
> >
> > diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
> > index 62602af1775..86e0b8afe01 100644
> > --- a/contrib/pg_buffercache/pg_buffercache_pages.c
> > +++ b/contrib/pg_buffercache/pg_buffercache_pages.c
> > @@ -14,7 +14,6 @@
> > #include "storage/buf_internals.h"
> > #include "storage/bufmgr.h"
> >
> > -
> > #define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
>
> Independent change.
Fixed.
> > #define NUM_BUFFERCACHE_PAGES_ELEM 9
> > #define NUM_BUFFERCACHE_SUMMARY_ELEM 5
> > @@ -68,80 +67,192 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
> > PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
> > PG_FUNCTION_INFO_V1(pg_buffercache_evict);
> >
> > -Datum
> > -pg_buffercache_pages(PG_FUNCTION_ARGS)
> > +/*
> > + * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
> > + *
> > + * This is almost identical to pg_buffercache_numa_pages(), but this one performs
> > + * memory mapping inquiries to display NUMA node information for each buffer.
> > + */
>
> If it's a helper routine it's probably not identical to
> pg_buffercache_numa_pages()?
Of course not, fixed by removing that comment.
> > +/*
> > + * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
> > + *
> > + * Build buffer cache information for a single buffer.
> > + */
> > +static void
> > +pg_buffercache_build_tuple(int record_id, BufferCachePagesContext *fctx)
> > +{
>
> This isn't really building a tuple tuple? Seems somewhat confusing, because
> get_buffercache_tuple() does actually build one.
s/pg_buffercache_build_tuple/pg_buffercache_save_tuple/g , unless
someone wants to come with better name.
> > + BufferDesc *bufHdr;
> > + uint32 buf_state;
> > +
> > + bufHdr = GetBufferDescriptor(record_id);
> > + /* Lock each buffer header before inspecting. */
> > + buf_state = LockBufHdr(bufHdr);
> > +
> > + fctx->record[record_id].bufferid = BufferDescriptorGetBuffer(bufHdr);
> > + fctx->record[record_id].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
> > + fctx->record[record_id].reltablespace = bufHdr->tag.spcOid;
> > + fctx->record[record_id].reldatabase = bufHdr->tag.dbOid;
> > + fctx->record[record_id].forknum = BufTagGetForkNum(&bufHdr->tag);
> > + fctx->record[record_id].blocknum = bufHdr->tag.blockNum;
> > + fctx->record[record_id].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
> > + fctx->record[record_id].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
>
> As above, I think this would be more readable if you put
> fctx->record[record_id] into a local var.
Done.
> > +static Datum
> > +get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
> > +{
> > + Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
> > + bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
> > + HeapTuple tuple;
> > +
> > + values[0] = Int32GetDatum(fctx->record[record_id].bufferid);
> > + nulls[0] = false;
> > +
> > + /*
> > + * Set all fields except the bufferid to null if the buffer is unused or
> > + * not valid.
> > + */
> > + if (fctx->record[record_id].blocknum == InvalidBlockNumber ||
> > + fctx->record[record_id].isvalid == false)
> > + {
> > + nulls[1] = true;
> > + nulls[2] = true;
> > + nulls[3] = true;
> > + nulls[4] = true;
> > + nulls[5] = true;
> > + nulls[6] = true;
> > + nulls[7] = true;
> > +
> > + /* unused for v1.0 callers, but the array is always long enough */
> > + nulls[8] = true;
>
> I'd probably just memset the entire nulls array to either true or false,
> instead of doing it one-by-one.
Done.
> > + }
> > + else
> > + {
> > + values[1] = ObjectIdGetDatum(fctx->record[record_id].relfilenumber);
> > + nulls[1] = false;
> > + values[2] = ObjectIdGetDatum(fctx->record[record_id].reltablespace);
> > + nulls[2] = false;
> > + values[3] = ObjectIdGetDatum(fctx->record[record_id].reldatabase);
> > + nulls[3] = false;
> > + values[4] = ObjectIdGetDatum(fctx->record[record_id].forknum);
> > + nulls[4] = false;
> > + values[5] = Int64GetDatum((int64) fctx->record[record_id].blocknum);
> > + nulls[5] = false;
> > + values[6] = BoolGetDatum(fctx->record[record_id].isdirty);
> > + nulls[6] = false;
> > + values[7] = Int16GetDatum(fctx->record[record_id].usagecount);
> > + nulls[7] = false;
>
> Seems like it would end up a lot more readable if you put
> fctx->record[record_id] into a local variable. Unfortunately that'd probably
> be best done in one more commit ahead of the rest of the this one...
Done, i've put it those refactorig changes into the commit already
dedicated only for a refactor. For the record Bertrand also asked for
something about this, but I was somehow afraid to touch Tom's code.
> > @@ -0,0 +1,28 @@
> > +SELECT NOT(pg_numa_available()) AS skip_test \gset
> > +\if :skip_test
> > +\quit
> > +\endif
>
> You could avoid the need for an alternative output file if you instead made
> the queries do something like
> SELECT NOT pg_numa_available() OR count(*) ...
ITEM REMAINING: Is this for the future or can it stay like that? I
don't have a hard opinion on this, but I've already wasted lots of
cycles to discover that one can have those ".1" alternative expected
result files.
> > --- /dev/null
> > +++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
> > @@ -0,0 +1,42 @@
> > +/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
> > +
> > +-- complain if script is sourced in psql, rather than via CREATE EXTENSION
> > +\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
> > +
> > +-- Register the new function.
> > +DROP VIEW pg_buffercache;
> > +DROP FUNCTION pg_buffercache_pages();
>
> I don't think we can just drop a view in the upgrade script. That will fail if
> anybody created a view depending on pg_buffercache.
Ugh, fixed, thanks. That must have been some leftover (we later do
CREATE OR REPLACE those anyway).
> (Sorry, ran out of time / energy here, i had originally just wanted to comment
> on the apt-get thing in the tests)
Thanks! AIO intensifies ... :)
-J.
[1] - https://www.postgresql.org/message-id/CAKZiRmzpvBtqrz5Jr2DDcfk4Ar1ppsXkUhEX9RpA%2Bs%2B_5hcTOg%40mail...
Attachments:
[application/octet-stream] v17-0003-Extend-pg_buffercache-with-new-view-pg_buffercac.patch (17.2K, ../../CAKZiRmzKEkoYUB1_PR0uijPeqihyXCwBRY7N+WN4JGPz9Nk0Yw@mail.gmail.com/2-v17-0003-Extend-pg_buffercache-with-new-view-pg_buffercac.patch)
download | inline diff:
From a920535a78df30661e10936e4750ad4c3bfc1818 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Wed, 19 Mar 2025 09:34:56 +0100
Subject: [PATCH v17 3/4] Extend pg_buffercache with new view
pg_buffercache_numa to show NUMA memory node for individual buffer.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/Makefile | 3 +-
.../expected/pg_buffercache_numa.out | 28 +++
.../expected/pg_buffercache_numa_1.out | 3 +
contrib/pg_buffercache/meson.build | 2 +
.../pg_buffercache--1.5--1.6.sql | 39 ++++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 170 +++++++++++++++++-
.../sql/pg_buffercache_numa.sql | 20 +++
doc/src/sgml/pgbuffercache.sgml | 61 ++++++-
9 files changed, 324 insertions(+), 4 deletions(-)
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa.out
create mode 100644 contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
create mode 100644 contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
create mode 100644 contrib/pg_buffercache/sql/pg_buffercache_numa.sql
diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index eae65ead9e5..2a33602537e 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -8,7 +8,8 @@ OBJS = \
EXTENSION = pg_buffercache
DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
- pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql
+ pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
+ pg_buffercache--1.5--1.6.sql
PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
REGRESS = pg_buffercache
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa.out b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
new file mode 100644
index 00000000000..d4de5ea52fc
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa.out
@@ -0,0 +1,28 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ERROR: permission denied for view pg_buffercache_numa
+RESET role;
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET role;
diff --git a/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/contrib/pg_buffercache/expected/pg_buffercache_numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 12d1fe48717..7cd039a1df9 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -23,6 +23,7 @@ install_data(
'pg_buffercache--1.2.sql',
'pg_buffercache--1.3--1.4.sql',
'pg_buffercache--1.4--1.5.sql',
+ 'pg_buffercache--1.5--1.6.sql',
'pg_buffercache.control',
kwargs: contrib_data_args,
)
@@ -34,6 +35,7 @@ tests += {
'regress': {
'sql': [
'pg_buffercache',
+ 'pg_buffercache_numa',
],
},
}
diff --git a/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
new file mode 100644
index 00000000000..720dc84b2c9
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql
@@ -0,0 +1,39 @@
+/* contrib/pg_buffercache/pg_buffercache--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_buffercache" to load this file. \quit
+
+-- Register the new functions.
+CREATE OR REPLACE FUNCTION pg_buffercache_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pages'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE OR REPLACE FUNCTION pg_buffercache_numa_pages()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_numa_pages'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE OR REPLACE VIEW pg_buffercache AS
+ SELECT P.* FROM pg_buffercache_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4);
+
+CREATE OR REPLACE VIEW pg_buffercache_numa AS
+ SELECT P.* FROM pg_buffercache_numa_pages() AS P
+ (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid,
+ relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2,
+ pinning_backends int4, node_id int4);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pages() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_numa_pages() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_numa FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pages() TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_buffercache_numa_pages() TO pg_monitor;
+GRANT SELECT ON pg_buffercache TO pg_monitor;
+GRANT SELECT ON pg_buffercache_numa TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 5ee875f77dd..b030ba3a6fa 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
# pg_buffercache extension
comment = 'examine the shared buffer cache'
-default_version = '1.5'
+default_version = '1.6'
module_pathname = '$libdir/pg_buffercache'
relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index cad7429a21b..1ec9ac25d58 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -11,12 +11,13 @@
#include "access/htup_details.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
+#include "port/pg_numa.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
#define NUM_BUFFERCACHE_PAGES_MIN_ELEM 8
-#define NUM_BUFFERCACHE_PAGES_ELEM 9
+#define NUM_BUFFERCACHE_PAGES_ELEM 10
#define NUM_BUFFERCACHE_SUMMARY_ELEM 5
#define NUM_BUFFERCACHE_USAGE_COUNTS_ELEM 4
@@ -46,6 +47,7 @@ typedef struct
* because of bufmgr.c's PrivateRefCount infrastructure.
*/
int32 pinning_backends;
+ int32 numa_node_id;
} BufferCachePagesRec;
@@ -64,10 +66,56 @@ typedef struct
* relation node/tablespace/database/blocknum and dirty indicator.
*/
PG_FUNCTION_INFO_V1(pg_buffercache_pages);
+PG_FUNCTION_INFO_V1(pg_buffercache_numa_pages);
PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
+/* Only need to touch memory once per backend process lifetime */
+static bool firstNumaTouch = true;
+
+/*
+ * Helper routine to map Buffers into addresses that is used by
+ * pg_numa_query_pages().
+ *
+ * When database block size (BLCKSZ) is smaller than the OS page size (4kB),
+ * multiple database buffers will map to the same OS memory page. In this case,
+ * we only need to query the NUMA node for the first memory address of each
+ * unique OS page rather than for every buffer.
+ *
+ * In order to get reliable results we also need to touch memory pages, so that
+ * inquiry about NUMA memory node doesn't return -2 (which indicates
+ * unmapped/unallocated pages).
+ */
+static inline void
+pg_buffercache_numa_prepare_ptrs(int buffer_id, float pages_per_blk,
+ Size os_page_size,
+ void **os_page_ptrs)
+{
+ size_t blk2page = (size_t) (buffer_id * pages_per_blk);
+
+ for (size_t j = 0; j < pages_per_blk; j++)
+ {
+ size_t blk2pageoff = blk2page + j;
+
+ if (os_page_ptrs[blk2pageoff] == 0)
+ {
+ volatile uint64 touch pg_attribute_unused();
+
+ /* NBuffers starts from 1 */
+ os_page_ptrs[blk2pageoff] = (char *) BufferGetBlock(buffer_id + 1) +
+ (os_page_size * j);
+
+ /* Only need to touch memory once per backend process lifetime */
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, os_page_ptrs[blk2pageoff]);
+
+ }
+
+ CHECK_FOR_INTERRUPTS();
+ }
+}
+
/*
* Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
*/
@@ -122,6 +170,9 @@ pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 10, "node_id",
+ INT4OID, -1, 0);
fctx->tupdesc = BlessTupleDesc(tupledesc);
@@ -175,6 +226,8 @@ pg_buffercache_save_tuple(int record_id, BufferCachePagesContext *fctx)
else
bufRecord->isvalid = false;
+ bufRecord->numa_node_id = -1;
+
UnlockBufHdr(bufHdr, buf_state);
}
@@ -220,6 +273,7 @@ get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
* unused for v1.0 callers, but the array is always long enough
*/
values[8] = Int32GetDatum(bufRecord->pinning_backends);
+ values[9] = Int32GetDatum(bufRecord->numa_node_id);
}
/* Build and return the tuple. */
@@ -271,6 +325,120 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
}
}
+/*
+ * This is almost identical to the above, but performs
+ * NUMA inuqiry about memory mappings.
+ */
+Datum
+pg_buffercache_numa_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+ Size os_page_size = 0;
+ void **os_page_ptrs = NULL;
+ int *os_pages_status = NULL;
+ uint64 os_page_count = 0;
+ float pages_per_blk = 0;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+
+ if (pg_numa_init() == -1)
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
+
+ /*
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used,
+ * while the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: 1. Determine the OS
+ * memory page size 2. Calculate how many OS pages are used by all
+ * buffer blocks 3. Calculate how many OS pages are contained within
+ * each database block.
+ *
+ * This information is needed before calling move_pages() for NUMA
+ * node id inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+ os_page_count = ((uint64) NBuffers * BLCKSZ) / os_page_size;
+ pages_per_blk = (float) BLCKSZ / os_page_size;
+
+ elog(DEBUG1, "NUMA: os_page_count=%lu os_page_size=%zu pages_per_blk=%.2f",
+ (unsigned long) os_page_count, os_page_size, pages_per_blk);
+
+ os_page_ptrs = palloc0(sizeof(void *) * os_page_count);
+ os_pages_status = palloc(sizeof(uint64) * os_page_count);
+
+ /*
+ * If we ever get 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here.
+ *
+ */
+ memset(os_pages_status, 0xff, sizeof(int) * os_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting the buffercache for proper NUMA readouts");
+
+ /*
+ * Scan through all the buffers, saving the relevant fields in the
+ * fctx->record structure.
+ *
+ * We don't hold the partition locks, so we don't get a consistent
+ * snapshot across all buffers, but we do grab the buffer header
+ * locks, so the information of each buffer is self-consistent.
+ */
+ for (i = 0; i < NBuffers; i++)
+ {
+ pg_buffercache_save_tuple(i, fctx);
+ pg_buffercache_numa_prepare_ptrs(i, pages_per_blk, os_page_size,
+ os_page_ptrs);
+ }
+
+ if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry: %m");
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ int blk2page = (int) i * pages_per_blk;
+
+ /*
+ * Set the NUMA node id for this buffer based on the first OS page
+ * it maps to.
+ *
+ * Note: We could check for errors in os_pages_status and report
+ * them. Also, a single DB block might span multiple NUMA nodes if
+ * it crosses OS pages on node boundaries, but we only record the
+ * node of the first page. This is a simplification but should be
+ * sufficient for most analyses.
+ */
+ fctx->record[i].numa_node_id = os_pages_status[blk2page];
+ }
+ }
+
+ funcctx = SRF_PERCALL_SETUP();
+
+ /* Get the saved state */
+ fctx = funcctx->user_fctx;
+
+ if (funcctx->call_cntr < funcctx->max_calls)
+ {
+ Datum result;
+ uint32 i = funcctx->call_cntr;
+
+ result = get_buffercache_tuple(i, fctx);
+ SRF_RETURN_NEXT(funcctx, result);
+ }
+ else
+ {
+ firstNumaTouch = false;
+ SRF_RETURN_DONE(funcctx);
+ }
+}
+
Datum
pg_buffercache_summary(PG_FUNCTION_ARGS)
{
diff --git a/contrib/pg_buffercache/sql/pg_buffercache_numa.sql b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
new file mode 100644
index 00000000000..2225b879f58
--- /dev/null
+++ b/contrib/pg_buffercache/sql/pg_buffercache_numa.sql
@@ -0,0 +1,20 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+select count(*) = (select setting::bigint
+ from pg_settings
+ where name = 'shared_buffers')
+from pg_buffercache_numa;
+
+-- Check that the functions / views can't be accessed by default. To avoid
+-- having to create a dedicated user, use the pg_database_owner pseudo-role.
+SET ROLE pg_database_owner;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
+
+-- Check that pg_monitor is allowed to query view / function
+SET ROLE pg_monitor;
+SELECT count(*) > 0 FROM pg_buffercache_numa;
+RESET role;
diff --git a/doc/src/sgml/pgbuffercache.sgml b/doc/src/sgml/pgbuffercache.sgml
index 802a5112d77..086e0062a17 100644
--- a/doc/src/sgml/pgbuffercache.sgml
+++ b/doc/src/sgml/pgbuffercache.sgml
@@ -30,7 +30,9 @@
<para>
This module provides the <function>pg_buffercache_pages()</function>
function (wrapped in the <structname>pg_buffercache</structname> view),
- the <function>pg_buffercache_summary()</function> function, the
+ <function>pg_buffercache_numa_pages()</function> function (wrapped in the
+ <structname>pg_buffercache_numa</structname> view), the
+ <function>pg_buffercache_summary()</function> function, the
<function>pg_buffercache_usage_counts()</function> function and
the <function>pg_buffercache_evict()</function> function.
</para>
@@ -42,6 +44,14 @@
convenient use.
</para>
+ <para>
+ The <function>pg_buffercache_numa_pages()</function> provides the same information
+ as <function>pg_buffercache_pages()</function> but is slower because it also
+ provides the <acronym>NUMA</acronym> node ID per shared buffer entry.
+ The <structname>pg_buffercache_numa</structname> view wraps the function for
+ convenient use.
+ </para>
+
<para>
The <function>pg_buffercache_summary()</function> function returns a single
row summarizing the state of the shared buffer cache.
@@ -200,6 +210,55 @@
</para>
</sect2>
+ <sect2 id="pgbuffercache-pg-buffercache_numa">
+ <title>The <structname>pg_buffercache_numa</structname> View</title>
+
+ <para>
+ The definitions of the columns exposed are identical to the
+ <structname>pg_buffercache</structname> view, except that this one includes
+ one additional <structfield>node_id</structfield> column as defined in
+ <xref linkend="pgbuffercache-numa-columns"/>.
+ </para>
+
+ <table id="pgbuffercache-numa-columns">
+ <title><structname>pg_buffercache_numa</structname> Extra column</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>node_id</structfield> <type>integer</type>
+ </para>
+ <para>
+ <acronym>NUMA</acronym> node ID. NULL if the shared buffer
+ has not been used yet. On systems without <acronym>NUMA</acronym> support
+ this returns 0.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ As <acronym>NUMA</acronym> node ID inquiry for each page requires memory pages
+ to be paged-in, the first execution of this function can take a noticeable
+ amount of time. In all the cases (first execution or not), retrieving this
+ information is costly and querying the view at a high frequency is not recommended.
+ </para>
+
+ </sect2>
+
<sect2 id="pgbuffercache-summary">
<title>The <function>pg_buffercache_summary()</function> Function</title>
--
2.39.5
[application/octet-stream] v17-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch (21.4K, ../../CAKZiRmzKEkoYUB1_PR0uijPeqihyXCwBRY7N+WN4JGPz9Nk0Yw@mail.gmail.com/3-v17-0001-Add-optional-dependency-to-libnuma-Linux-only-fo.patch)
download | inline diff:
From 2d0e0bf27d5cc55c97b8e660fd22e6ae94db27d0 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 10:19:35 +0100
Subject: [PATCH v17 1/4] Add optional dependency to libnuma (Linux-only) for
basic NUMA awareness routines and add minimal src/port/pg_numa.c portability
wrapper. Other platforms can be added later.
This also adds function pg_numa_available() that can be used to check if
the server was linked with NUMA support.
libnuma is unavailable on 32-bit builds, so due to lack of i386 shared object,
we disable it there (it does not make sense anyway on i386 it is very memory
limited platform even with PAE)
Author: Jakub Wartak <[email protected]>
Co-authored-by: Bertrand Drouvot <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
.cirrus.tasks.yml | 2 +
configure | 187 ++++++++++++++++++++++++++++
configure.ac | 14 +++
doc/src/sgml/func.sgml | 13 ++
doc/src/sgml/installation.sgml | 21 ++++
meson.build | 23 ++++
meson_options.txt | 3 +
src/Makefile.global.in | 6 +-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/pg_config.h.in | 3 +
src/include/port/pg_numa.h | 41 ++++++
src/include/storage/pg_shmem.h | 1 +
src/makefiles/meson.build | 3 +
src/port/Makefile | 1 +
src/port/meson.build | 1 +
src/port/pg_numa.c | 109 ++++++++++++++++
17 files changed, 432 insertions(+), 2 deletions(-)
create mode 100644 src/include/port/pg_numa.h
create mode 100644 src/port/pg_numa.c
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 86a1fa9bbdb..6f4f5c674a1 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -471,6 +471,7 @@ task:
--enable-cassert --enable-injection-points --enable-debug \
--enable-tap-tests --enable-nls \
--with-segsize-blocks=6 \
+ --with-libnuma \
--with-liburing \
\
${LINUX_CONFIGURE_FEATURES} \
@@ -523,6 +524,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.36-i386-linux-gnu \
+ -Dlibnuma=disabled \
build-32
EOF
diff --git a/configure b/configure
index 30d949c3c46..bc195975c2e 100755
--- a/configure
+++ b/configure
@@ -708,6 +708,9 @@ XML2_LIBS
XML2_CFLAGS
XML2_CONFIG
with_libxml
+LIBNUMA_LIBS
+LIBNUMA_CFLAGS
+with_libnuma
LIBCURL_LIBS
LIBCURL_CFLAGS
with_libcurl
@@ -872,6 +875,7 @@ with_liburing
with_uuid
with_ossp_uuid
with_libcurl
+with_libnuma
with_libxml
with_libxslt
with_system_tzdata
@@ -906,6 +910,8 @@ LIBURING_CFLAGS
LIBURING_LIBS
LIBCURL_CFLAGS
LIBCURL_LIBS
+LIBNUMA_CFLAGS
+LIBNUMA_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
@@ -1588,6 +1594,7 @@ Optional Packages:
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libcurl build with libcurl support
+ --with-libnuma build with libnuma for NUMA awareness
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
@@ -1629,6 +1636,10 @@ Some influential environment variables:
C compiler flags for LIBCURL, overriding pkg-config
LIBCURL_LIBS
linker flags for LIBCURL, overriding pkg-config
+ LIBNUMA_CFLAGS
+ C compiler flags for LIBNUMA, overriding pkg-config
+ LIBNUMA_LIBS
+ linker flags for LIBNUMA, overriding pkg-config
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
@@ -9063,6 +9074,182 @@ $as_echo "$as_me: WARNING: *** OAuth support tests require --with-python to run"
fi
+#
+# libnuma
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with libnuma support" >&5
+$as_echo_n "checking whether to build with libnuma support... " >&6; }
+
+
+
+# Check whether --with-libnuma was given.
+if test "${with_libnuma+set}" = set; then :
+ withval=$with_libnuma;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LIBNUMA 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-libnuma option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_libnuma=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_libnuma" >&5
+$as_echo "$with_libnuma" >&6; }
+
+
+if test "$with_libnuma" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa_available in -lnuma" >&5
+$as_echo_n "checking for numa_available in -lnuma... " >&6; }
+if ${ac_cv_lib_numa_numa_available+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lnuma $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char numa_available ();
+int
+main ()
+{
+return numa_available ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_numa_numa_available=yes
+else
+ ac_cv_lib_numa_numa_available=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_numa_numa_available" >&5
+$as_echo "$ac_cv_lib_numa_numa_available" >&6; }
+if test "x$ac_cv_lib_numa_numa_available" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBNUMA 1
+_ACEOF
+
+ LIBS="-lnuma $LIBS"
+
+else
+ as_fn_error $? "library 'libnuma' is required for NUMA awareness" "$LINENO" 5
+fi
+
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for numa" >&5
+$as_echo_n "checking for numa... " >&6; }
+
+if test -n "$LIBNUMA_CFLAGS"; then
+ pkg_cv_LIBNUMA_CFLAGS="$LIBNUMA_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"numa\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "numa") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LIBNUMA_CFLAGS=`$PKG_CONFIG --cflags "numa" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LIBNUMA_LIBS"; then
+ pkg_cv_LIBNUMA_LIBS="$LIBNUMA_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"numa\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "numa") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LIBNUMA_LIBS=`$PKG_CONFIG --libs "numa" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LIBNUMA_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "numa" 2>&1`
+ else
+ LIBNUMA_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "numa" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LIBNUMA_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (numa) were not met:
+
+$LIBNUMA_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LIBNUMA_CFLAGS
+and LIBNUMA_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LIBNUMA_CFLAGS
+and LIBNUMA_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LIBNUMA_CFLAGS=$pkg_cv_LIBNUMA_CFLAGS
+ LIBNUMA_LIBS=$pkg_cv_LIBNUMA_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+fi
+
#
# XML
#
diff --git a/configure.ac b/configure.ac
index 25cdfcf65af..064dfee5ad0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1053,6 +1053,20 @@ if test "$with_libcurl" = yes ; then
fi
+#
+# libnuma
+#
+AC_MSG_CHECKING([whether to build with libnuma support])
+PGAC_ARG_BOOL(with, libnuma, no, [build with libnuma for NUMA awareness],
+ [AC_DEFINE([USE_LIBNUMA], 1, [Define to build with NUMA awareness support. (--with-libnuma)])])
+AC_MSG_RESULT([$with_libnuma])
+AC_SUBST(with_libnuma)
+
+if test "$with_libnuma" = yes ; then
+ AC_CHECK_LIB(numa, numa_available, [], [AC_MSG_ERROR([library 'libnuma' is required for NUMA awareness])])
+ PKG_CHECK_MODULES(LIBNUMA, numa)
+fi
+
#
# XML
#
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5bf6656deca..1f98826d16d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25138,6 +25138,19 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_numa_available</primary>
+ </indexterm>
+ <function>pg_numa_available</function> ()
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Returns true if the server has been compiled with <acronym>NUMA</acronym> support.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index cc28f041330..5f0486bb335 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1156,6 +1156,16 @@ build-postgresql:
</listitem>
</varlistentry>
+ <varlistentry id="configure-option-with-libnuma">
+ <term><option>--with-libnuma</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-option-with-liburing">
<term><option>--with-liburing</option></term>
<listitem>
@@ -2645,6 +2655,17 @@ ninja install
</listitem>
</varlistentry>
+ <varlistentry id="configure-with-libnuma-meson">
+ <term><option>-Dlibnuma={ auto | enabled | disabled }</option></term>
+ <listitem>
+ <para>
+ Build with libnuma support for basic NUMA support.
+ Only supported on platforms for which the libnuma library is implemented.
+ The default for this option is auto.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="configure-with-libxml-meson">
<term><option>-Dlibxml={ auto | enabled | disabled }</option></term>
<listitem>
diff --git a/meson.build b/meson.build
index b8da4966297..f509370ee42 100644
--- a/meson.build
+++ b/meson.build
@@ -943,6 +943,27 @@ else
endif
+###############################################################
+# Library: libnuma
+###############################################################
+
+libnumaopt = get_option('libnuma')
+if not libnumaopt.disabled()
+ # via pkg-config
+ libnuma = dependency('numa', required: libnumaopt)
+ if not libnuma.found()
+ libnuma = cc.find_library('numa', required: libnumaopt)
+ endif
+ if not cc.has_header('numa.h', dependencies: libnuma, required: libnumaopt)
+ libnuma = not_found_dep
+ endif
+ if libnuma.found()
+ cdata.set('USE_LIBNUMA', 1)
+ endif
+else
+ libnuma = not_found_dep
+endif
+
###############################################################
# Library: liburing
@@ -3225,6 +3246,7 @@ backend_both_deps += [
icu_i18n,
ldap,
libintl,
+ libnuma,
liburing,
libxml,
lz4,
@@ -3881,6 +3903,7 @@ if meson.version().version_compare('>=0.57')
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'liburing': liburing,
'libxml': libxml,
'libxslt': libxslt,
diff --git a/meson_options.txt b/meson_options.txt
index dd7126da3a7..8675e1b5d87 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -106,6 +106,9 @@ option('libcurl', type : 'feature', value: 'auto',
option('libedit_preferred', type: 'boolean', value: false,
description: 'Prefer BSD Libedit over GNU Readline')
+option('libnuma', type: 'feature', value: 'auto',
+ description: 'NUMA awareness support')
+
option('liburing', type : 'feature', value: 'auto',
description: 'io_uring support, for asynchronous I/O')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index cce29a37ac5..8b61d1ed492 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -196,6 +196,7 @@ with_gssapi = @with_gssapi@
with_krb_srvnam = @with_krb_srvnam@
with_ldap = @with_ldap@
with_libcurl = @with_libcurl@
+with_libnuma = @with_libnuma@
with_liburing = @with_liburing@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
@@ -223,6 +224,9 @@ krb_srvtab = @krb_srvtab@
ICU_CFLAGS = @ICU_CFLAGS@
ICU_LIBS = @ICU_LIBS@
+LIBNUMA_CFLAGS = @LIBNUMA_CFLAGS@
+LIBNUMA_LIBS = @LIBNUMA_LIBS@
+
LIBURING_CFLAGS = @LIBURING_CFLAGS@
LIBURING_LIBS = @LIBURING_LIBS@
@@ -250,7 +254,7 @@ CPP = @CPP@
CPPFLAGS = @CPPFLAGS@
PG_SYSROOT = @PG_SYSROOT@
-override CPPFLAGS := $(ICU_CFLAGS) $(LIBURING_CFLAGS) $(CPPFLAGS)
+override CPPFLAGS := $(ICU_CFLAGS) $(LIBNUMA_CFLAGS) $(LIBURING_CFLAGS) $(CPPFLAGS)
ifdef PGXS
override CPPFLAGS := -I$(includedir_server) -I$(includedir_internal) $(CPPFLAGS)
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..ea8d796e7c4 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -566,7 +566,7 @@ static int ssl_renegotiation_limit;
*/
int huge_pages = HUGE_PAGES_TRY;
int huge_page_size;
-static int huge_pages_status = HUGE_PAGES_UNKNOWN;
+int huge_pages_status = HUGE_PAGES_UNKNOWN;
/*
* These variables are all dummies that don't do anything, except in some
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8b68b16d79d..d532b8c43b9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8506,6 +8506,10 @@
proargnames => '{name,off,size,allocated_size}',
prosrc => 'pg_get_shmem_allocations' },
+{ oid => '9685', descr => 'Is NUMA compilation available?',
+ proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_numa_available' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 92f0616c400..e67f81da167 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -675,6 +675,9 @@
/* Define to 1 to build with libcurl support. (--with-libcurl) */
#undef USE_LIBCURL
+/* Define to 1 to build with NUMA awareness support. (--with-libnuma) */
+#undef USE_LIBNUMA
+
/* Define to build with io_uring support. (--with-liburing) */
#undef USE_LIBURING
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
new file mode 100644
index 00000000000..2fa0bc82a90
--- /dev/null
+++ b/src/include/port/pg_numa.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.h
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/include/port/pg_numa.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_NUMA_H
+#define PG_NUMA_H
+
+#include "fmgr.h"
+
+extern PGDLLIMPORT int pg_numa_init(void);
+extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
+extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_get_pagesize(void);
+extern PGDLLIMPORT Datum pg_numa_available(PG_FUNCTION_ARGS);
+
+#ifdef USE_LIBNUMA
+
+/*
+ * This is required on Linux, before pg_numa_query_pages() as we
+ * need to page-fault before move_pages(2) syscall returns valid results.
+ */
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ ro_volatile_var = *(uint64 *)ptr
+
+#else
+
+#define pg_numa_touch_mem_if_required(ro_volatile_var, ptr) \
+ do {} while(0)
+
+#endif
+
+#endif /* PG_NUMA_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index b99ebc9e86f..5f7d4b83a60 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -45,6 +45,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int huge_pages_status;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 46d8da070e8..55da678ec27 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -200,6 +200,8 @@ pgxs_empty = [
'ICU_LIBS',
+ 'LIBNUMA_CFLAGS', 'LIBNUMA_LIBS',
+
'LIBURING_CFLAGS', 'LIBURING_LIBS',
]
@@ -232,6 +234,7 @@ pgxs_deps = {
'icu': icu,
'ldap': ldap,
'libcurl': libcurl,
+ 'libnuma': libnuma,
'liburing': liburing,
'libxml': libxml,
'libxslt': libxslt,
diff --git a/src/port/Makefile b/src/port/Makefile
index f11896440d5..4274949dfa4 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -45,6 +45,7 @@ OBJS = \
path.o \
pg_bitutils.o \
pg_localeconv_r.o \
+ pg_numa.o \
pg_popcount_aarch64.o \
pg_popcount_avx512.o \
pg_strong_random.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index cf7f07644b9..3b26c68fda7 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -8,6 +8,7 @@ pgport_sources = [
'path.c',
'pg_bitutils.c',
'pg_localeconv_r.c',
+ 'pg_numa.c',
'pg_popcount_aarch64.c',
'pg_popcount_avx512.c',
'pg_strong_random.c',
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
new file mode 100644
index 00000000000..076d0bb5904
--- /dev/null
+++ b/src/port/pg_numa.c
@@ -0,0 +1,109 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_numa.c
+ * Basic NUMA portability routines
+ *
+ *
+ * Copyright (c) 2025, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_numa.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include <unistd.h>
+
+#ifdef WIN32
+#include <windows.h>
+#endif
+
+#include "fmgr.h"
+#include "miscadmin.h"
+#include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+/*
+ * At this point we provide support only for Linux thanks to libnuma, but in
+ * future support for other platforms e.g. Win32 or FreeBSD might be possible
+ * too. For Win32 NUMA APIs see
+ * https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support
+ */
+#ifdef USE_LIBNUMA
+
+#include <numa.h>
+#include <numaif.h>
+
+/* libnuma requires initialization as per numa(3) on Linux */
+int
+pg_numa_init(void)
+{
+ int r = numa_available();
+
+ return r;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return numa_move_pages(pid, count, pages, NULL, status, 0);
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return numa_max_node();
+}
+
+#else
+
+/* Empty wrappers */
+int
+pg_numa_init(void)
+{
+ /* We state that NUMA is not available */
+ return -1;
+}
+
+int
+pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status)
+{
+ return 0;
+}
+
+int
+pg_numa_get_max_node(void)
+{
+ return 0;
+}
+
+#endif
+
+Datum
+pg_numa_available(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_BOOL(pg_numa_init() != -1);
+}
+
+/* This should be used only after the server is started */
+Size
+pg_numa_get_pagesize(void)
+{
+ Size os_page_size;
+#ifdef WIN32
+ SYSTEM_INFO sysinfo;
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+#else
+ os_page_size= sysconf(_SC_PAGESIZE);
+#endif
+
+ Assert(IsUnderPostmaster);
+
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
--
2.39.5
[application/octet-stream] v17-0004-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch (16.2K, ../../CAKZiRmzKEkoYUB1_PR0uijPeqihyXCwBRY7N+WN4JGPz9Nk0Yw@mail.gmail.com/4-v17-0004-Add-pg_shmem_numa_allocations-to-show-NUMA-memor.patch)
download | inline diff:
From 2a9af45ab222c73e835b26caa5e6a596c7e9b6f6 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Fri, 21 Feb 2025 14:20:18 +0100
Subject: [PATCH v17 4/4] Add pg_shmem_numa_allocations to show NUMA memory
node for shared memory allocations.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
doc/src/sgml/system-views.sgml | 79 ++++++++++++++
src/backend/catalog/system_views.sql | 8 ++
src/backend/storage/ipc/shmem.c | 131 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 8 ++
src/test/regress/expected/numa.out | 12 +++
src/test/regress/expected/numa_1.out | 3 +
src/test/regress/expected/privileges.out | 16 ++-
src/test/regress/expected/rules.out | 4 +
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/numa.sql | 9 ++
src/test/regress/sql/privileges.sql | 6 +-
11 files changed, 273 insertions(+), 5 deletions(-)
create mode 100644 src/test/regress/expected/numa.out
create mode 100644 src/test/regress/expected/numa_1.out
create mode 100644 src/test/regress/sql/numa.sql
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 3f5a306247e..e83711f9578 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -176,6 +176,11 @@
<entry>shared memory allocations</entry>
</row>
+ <row>
+ <entry><link linkend="view-pg-shmem-numa-allocations"><structname>pg_shmem_numa_allocations</structname></link></entry>
+ <entry>NUMA node mappings for shared memory allocations</entry>
+ </row>
+
<row>
<entry><link linkend="view-pg-stats"><structname>pg_stats</structname></link></entry>
<entry>planner statistics</entry>
@@ -3746,6 +3751,80 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para>
</sect1>
+ <sect1 id="view-pg-shmem-numa-allocations">
+ <title><structname>pg_shmem_numa_allocations</structname></title>
+
+ <indexterm zone="view-pg-shmem-numa-allocations">
+ <primary>pg_shmem_numa_allocations</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_shmem_numa_allocations</structname> shows how shared
+ memory allocations in the server's main shared memory segment are distributed
+ across NUMA nodes. This includes both memory allocated by
+ <productname>PostgreSQL</productname> itself and memory allocated
+ by extensions using the mechanisms detailed in
+ <xref linkend="xfunc-shared-addin" />.
+ </para>
+
+ <para>
+ Note that this view does not include memory allocated using the dynamic
+ shared memory infrastructure.
+ </para>
+
+ <table>
+ <title><structname>pg_shmem_numa_allocations</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>name</structfield> <type>text</type>
+ </para>
+ <para>
+ The name of the shared memory allocation.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>node_id</structfield> <type>int4</type>
+ </para>
+ <para>
+ ID of <acronym>NUMA</acronym> node
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>size</structfield> <type>int4</type>
+ </para>
+ <para>
+ Size of the allocation on this particular NUMA memory node in bytes
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ By default, the <structname>pg_shmem_numa_allocations</structname> view can be
+ read only by superusers or roles with privileges of the
+ <literal>pg_read_all_stats</literal> role.
+ </para>
+ </sect1>
+
<sect1 id="view-pg-stats">
<title><structname>pg_stats</structname></title>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 31d269b7ee0..660a9e9832b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -658,6 +658,14 @@ GRANT SELECT ON pg_shmem_allocations TO pg_read_all_stats;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_shmem_allocations() TO pg_read_all_stats;
+CREATE VIEW pg_shmem_numa_allocations AS
+ SELECT * FROM pg_get_shmem_numa_allocations();
+
+REVOKE ALL ON pg_shmem_numa_allocations FROM PUBLIC;
+GRANT SELECT ON pg_shmem_numa_allocations TO pg_read_all_stats;
+REVOKE EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_shmem_numa_allocations() TO pg_read_all_stats;
+
CREATE VIEW pg_backend_memory_contexts AS
SELECT * FROM pg_get_backend_memory_contexts();
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 895a43fb39e..e83f066171a 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -68,6 +68,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "port/pg_numa.h"
#include "storage/lwlock.h"
#include "storage/pg_shmem.h"
#include "storage/shmem.h"
@@ -89,6 +90,8 @@ slock_t *ShmemLock; /* spinlock for shared memory and LWLock
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+/* To get reliable results for NUMA inquiry we need to "touch pages" once */
+static bool firstNumaTouch = true;
/*
* InitShmemAccess() --- set up basic pointers to shared memory.
@@ -568,3 +571,131 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
return (Datum) 0;
}
+
+/* SQL SRF showing NUMA memory nodes for allocated shared memory */
+Datum
+pg_get_shmem_numa_allocations(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SHMEM_NUMA_SIZES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ HASH_SEQ_STATUS hstat;
+ ShmemIndexEnt *ent;
+ Datum values[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ bool nulls[PG_GET_SHMEM_NUMA_SIZES_COLS];
+ Size os_page_size;
+ void **page_ptrs;
+ int *pages_status;
+ uint64 shm_total_page_count,
+ shm_ent_page_count,
+ max_nodes;
+ Size *nodes;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ if (pg_numa_init() == -1)
+ {
+ elog(ERROR, "libnuma initialization failed or NUMA is not supported on this platform");
+ return (Datum) 0;
+ }
+ max_nodes = pg_numa_get_max_node();
+ nodes = palloc(sizeof(Size) * (max_nodes + 1));
+
+ /*
+ * Different database block sizes (4kB, 8kB, ..., 32kB) can be used, while
+ * the OS may have different memory page sizes.
+ *
+ * To correctly map between them, we need to: 1. Determine the OS memory
+ * page size 2. Calculate how many OS pages are used by all buffer blocks
+ * 3. Calculate how many OS pages are contained within each database
+ * block.
+ *
+ * This information is needed before calling move_pages() for NUMA memory
+ * node inquiry.
+ */
+ os_page_size = pg_numa_get_pagesize();
+
+ /*
+ * Allocate memory for page pointers and status based on total shared
+ * memory size. This simplified approach allocates enough space for all
+ * pages in shared memory rather than calculating the exact requirements
+ * for each segment.
+ */
+ shm_total_page_count = ShmemSegHdr->totalsize / os_page_size;
+ page_ptrs = palloc0(sizeof(void *) * shm_total_page_count);
+ pages_status = palloc(sizeof(int) * shm_total_page_count);
+
+ if (firstNumaTouch)
+ elog(DEBUG1, "NUMA: page-faulting shared memory segments for proper NUMA readouts");
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+
+ hash_seq_init(&hstat, ShmemIndex);
+
+ /* output all allocated entries */
+ memset(nulls, 0, sizeof(nulls));
+ while ((ent = (ShmemIndexEnt *) hash_seq_search(&hstat)) != NULL)
+ {
+ int i;
+
+ shm_ent_page_count = ent->allocated_size / os_page_size;
+ /* It is always at least 1 page */
+ shm_ent_page_count = shm_ent_page_count == 0 ? 1 : shm_ent_page_count;
+
+ /*
+ * If we get ever 0xff back from kernel inquiry, then we probably have
+ * bug in our buffers to OS page mapping code here.
+ */
+ memset(pages_status, 0xff, sizeof(int) * shm_ent_page_count);
+
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ /*
+ * In order to get reliable results we also need to touch memory
+ * pages, so that inquiry about NUMA memory node doesn't return -2
+ * (which indicates unmapped/unallocated pages).
+ */
+ volatile uint64 touch pg_attribute_unused();
+
+ page_ptrs[i] = (char *) ent->location + (i * os_page_size);
+ if (firstNumaTouch)
+ pg_numa_touch_mem_if_required(touch, page_ptrs[i]);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
+ elog(ERROR, "failed NUMA pages inquiry status: %m");
+
+ memset(nodes, 0, sizeof(Size) * (max_nodes + 1));
+ /* Count number of NUMA nodes used for this shared memory entry */
+ for (i = 0; i < shm_ent_page_count; i++)
+ {
+ int s = pages_status[i];
+
+ /* Ensure we are adding only valid index to the array */
+ if (s >= 0 && s <= max_nodes)
+ nodes[s]++;
+ }
+
+ for (i = 0; i <= max_nodes; i++)
+ {
+ values[0] = CStringGetTextDatum(ent->key);
+ values[1] = i;
+ values[2] = Int64GetDatum(nodes[i] * os_page_size);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ }
+
+ /*
+ * We are ignoring the following memory regions (as compared to
+ * pg_get_shmem_allocations()): 1. output shared memory allocated but not
+ * counted via the shmem index 2. output as-of-yet unused shared memory.
+ */
+
+ LWLockRelease(ShmemIndexLock);
+ firstNumaTouch = false;
+
+ return (Datum) 0;
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d532b8c43b9..db69c0231f9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8510,6 +8510,14 @@
proname => 'pg_numa_available', provolatile => 'v', prorettype => 'bool',
proargtypes => '', prosrc => 'pg_numa_available' },
+# shared memory usage with NUMA info
+{ oid => '9686', descr => 'NUMA mappings for the main shared memory segment',
+ proname => 'pg_get_shmem_numa_allocations', prorows => '50', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,int4,int8}', proargmodes => '{o,o,o}',
+ proargnames => '{name,node_id,size}',
+ prosrc => 'pg_get_shmem_numa_allocations' },
+
# memory context of local backend
{ oid => '2282',
descr => 'information about all memory contexts of local backend',
diff --git a/src/test/regress/expected/numa.out b/src/test/regress/expected/numa.out
new file mode 100644
index 00000000000..fb882c5b771
--- /dev/null
+++ b/src/test/regress/expected/numa.out
@@ -0,0 +1,12 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- switch to superuser
+\c -
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
+ ok
+----
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/numa_1.out b/src/test/regress/expected/numa_1.out
new file mode 100644
index 00000000000..6dd6824b4e4
--- /dev/null
+++ b/src/test/regress/expected/numa_1.out
@@ -0,0 +1,3 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 954f549555e..d9d62470cdc 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -3127,8 +3127,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
-- clean up
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
CREATE ROLE regress_readallstats;
@@ -3144,6 +3144,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
f
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
+ has_table_privilege
+---------------------
+ f
+(1 row)
+
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
has_table_privilege
@@ -3157,6 +3163,12 @@ SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT
t
(1 row)
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
+ has_table_privilege
+---------------------
+ t
+(1 row)
+
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
SELECT COUNT(*) >= 0 AS ok FROM pg_backend_memory_contexts;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 47478969135..6e460e10d61 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1740,6 +1740,10 @@ pg_shmem_allocations| SELECT name,
size,
allocated_size
FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+pg_shmem_numa_allocations| SELECT name,
+ node_id,
+ size
+ FROM pg_get_shmem_numa_allocations() pg_get_shmem_numa_allocations(name, node_id, size);
pg_stat_activity| SELECT s.datid,
d.datname,
s.pid,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 0a35f2f8f6a..0f38caa0d24 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,7 +119,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# The stats test resets stats, so nothing else needing stats access can be in
# this group.
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats predicate numa
# event_trigger depends on create_am and cannot run concurrently with
# any test that runs DDL
diff --git a/src/test/regress/sql/numa.sql b/src/test/regress/sql/numa.sql
new file mode 100644
index 00000000000..fddb21a260a
--- /dev/null
+++ b/src/test/regress/sql/numa.sql
@@ -0,0 +1,9 @@
+SELECT NOT(pg_numa_available()) AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- switch to superuser
+\c -
+
+SELECT COUNT(*) >= 0 AS ok FROM pg_shmem_numa_allocations;
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index b81694c24f2..f93d4829702 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1911,8 +1911,8 @@ REVOKE MAINTAIN ON lock_table FROM regress_locktable_user;
DROP TABLE lock_table;
DROP USER regress_locktable_user;
--- test to check privileges of system views pg_shmem_allocations and
--- pg_backend_memory_contexts.
+-- test to check privileges of system views pg_shmem_allocations,
+-- pg_shmem_numa_allocations and pg_backend_memory_contexts.
-- switch to superuser
\c -
@@ -1921,11 +1921,13 @@ CREATE ROLE regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- no
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- no
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- no
GRANT pg_read_all_stats TO regress_readallstats;
SELECT has_table_privilege('regress_readallstats','pg_backend_memory_contexts','SELECT'); -- yes
SELECT has_table_privilege('regress_readallstats','pg_shmem_allocations','SELECT'); -- yes
+SELECT has_table_privilege('regress_readallstats','pg_shmem_numa_allocations','SELECT'); -- yes
-- run query to ensure that functions within views can be executed
SET ROLE regress_readallstats;
--
2.39.5
[application/octet-stream] v17-0002-This-extracts-code-from-contrib-pg_buffercache-s.patch (12.8K, ../../CAKZiRmzKEkoYUB1_PR0uijPeqihyXCwBRY7N+WN4JGPz9Nk0Yw@mail.gmail.com/5-v17-0002-This-extracts-code-from-contrib-pg_buffercache-s.patch)
download | inline diff:
From 9f032a8c43c837a0cce44ef4b2e0dbf4d7331dda Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email protected]>
Date: Wed, 19 Mar 2025 09:34:56 +0100
Subject: [PATCH v17 2/4] This extracts code from contrib/pg_buffercache's
primary function to separate functions.
This commit adds pg_buffercache_init_entries(), pg_buffercache_build_tuple()
and get_buffercache_tuple() that help fill result tuplestores based on the
buffercache contents. This will be used in a follow-up commit that implements
NUMA observability in pg_buffercache.
Author: Jakub Wartak <[email protected]>
Reviewed-by: Bertrand Drouvot <[email protected]>
Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com
---
contrib/pg_buffercache/pg_buffercache_pages.c | 299 ++++++++++--------
1 file changed, 162 insertions(+), 137 deletions(-)
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 62602af1775..cad7429a21b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -68,80 +68,177 @@ PG_FUNCTION_INFO_V1(pg_buffercache_summary);
PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
PG_FUNCTION_INFO_V1(pg_buffercache_evict);
-Datum
-pg_buffercache_pages(PG_FUNCTION_ARGS)
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ */
+static BufferCachePagesContext *
+pg_buffercache_init_entries(FuncCallContext *funcctx, FunctionCallInfo fcinfo)
{
- FuncCallContext *funcctx;
- Datum result;
- MemoryContext oldcontext;
BufferCachePagesContext *fctx; /* User function context. */
+ MemoryContext oldcontext;
TupleDesc tupledesc;
TupleDesc expected_tupledesc;
+
+ /* Switch context when allocating stuff to be used in later calls */
+ oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+
+ /* Create a user function context for cross-call persistence */
+ fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+
+ /*
+ * To smoothly support upgrades from version 1.0 of this extension
+ * transparently handle the (non-)existence of the pinning_backends
+ * column. ee unfortunately have to get the result type for that... - we
+ * can't use the result type determined by the function definition without
+ * potentially crashing when somebody uses the old (or even wrong)
+ * function definition though.
+ */
+ if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
+ expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
+ elog(ERROR, "incorrect number of output arguments");
+
+ /* Construct a tuple descriptor for the result rows. */
+ tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
+ OIDOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
+ INT2OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
+ BOOLOID, -1, 0);
+ TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
+ INT2OID, -1, 0);
+
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
+
+ fctx->tupdesc = BlessTupleDesc(tupledesc);
+
+ /* Allocate NBuffers worth of BufferCachePagesRec records. */
+ fctx->record = (BufferCachePagesRec *)
+ MemoryContextAllocHuge(CurrentMemoryContext,
+ sizeof(BufferCachePagesRec) * NBuffers);
+
+ /* Set max calls and remember the user function context. */
+ funcctx->max_calls = NBuffers;
+ funcctx->user_fctx = fctx;
+
+ /* Return to original context when allocating transient memory */
+ MemoryContextSwitchTo(oldcontext);
+ return fctx;
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Save buffer cache information for a single buffer.
+ */
+static void
+pg_buffercache_save_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+ BufferCachePagesRec *bufRecord = &(fctx->record[record_id]);
+
+ bufHdr = GetBufferDescriptor(record_id);
+ /* Lock each buffer header before inspecting. */
+ buf_state = LockBufHdr(bufHdr);
+
+ bufRecord->bufferid = BufferDescriptorGetBuffer(bufHdr);
+ bufRecord->relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
+ bufRecord->reltablespace = bufHdr->tag.spcOid;
+ bufRecord->reldatabase = bufHdr->tag.dbOid;
+ bufRecord->forknum = BufTagGetForkNum(&bufHdr->tag);
+ bufRecord->blocknum = bufHdr->tag.blockNum;
+ bufRecord->usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
+ bufRecord->pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
+
+ if (buf_state & BM_DIRTY)
+ bufRecord->isdirty = true;
+ else
+ bufRecord->isdirty = false;
+
+ /* Note if the buffer is valid, and has storage created */
+ if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
+ bufRecord->isvalid = true;
+ else
+ bufRecord->isvalid = false;
+
+ UnlockBufHdr(bufHdr, buf_state);
+}
+
+/*
+ * Helper routine for pg_buffercache_pages() and pg_buffercache_numa_pages().
+ *
+ * Format and return a tuple for a single buffer cache entry.
+ */
+static Datum
+get_buffercache_tuple(int record_id, BufferCachePagesContext *fctx)
+{
+ Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
+ bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
HeapTuple tuple;
+ BufferCachePagesRec *bufRecord = &(fctx->record[record_id]);
- if (SRF_IS_FIRSTCALL())
+ values[0] = Int32GetDatum(bufRecord->bufferid);
+ memset(nulls, false, NUM_BUFFERCACHE_PAGES_ELEM);
+
+ /*
+ * Set all fields except the bufferid to null if the buffer is unused or
+ * not valid.
+ */
+ if (bufRecord->blocknum == InvalidBlockNumber ||
+ bufRecord->isvalid == false)
{
int i;
- funcctx = SRF_FIRSTCALL_INIT();
-
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
-
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ for (i = 1; i <= 9; i++)
+ nulls[i] = true;
+ }
+ else
+ {
+ values[1] = ObjectIdGetDatum(bufRecord->relfilenumber);
+ values[2] = ObjectIdGetDatum(bufRecord->reltablespace);
+ values[3] = ObjectIdGetDatum(bufRecord->reldatabase);
+ values[4] = ObjectIdGetDatum(bufRecord->forknum);
+ values[5] = Int64GetDatum((int64) bufRecord->blocknum);
+ values[6] = BoolGetDatum(bufRecord->isdirty);
+ values[7] = Int16GetDatum(bufRecord->usagecount);
/*
- * To smoothly support upgrades from version 1.0 of this extension
- * transparently handle the (non-)existence of the pinning_backends
- * column. We unfortunately have to get the result type for that... -
- * we can't use the result type determined by the function definition
- * without potentially crashing when somebody uses the old (or even
- * wrong) function definition though.
+ * unused for v1.0 callers, but the array is always long enough
*/
- if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
- elog(ERROR, "return type must be a row type");
+ values[8] = Int32GetDatum(bufRecord->pinning_backends);
+ }
- if (expected_tupledesc->natts < NUM_BUFFERCACHE_PAGES_MIN_ELEM ||
- expected_tupledesc->natts > NUM_BUFFERCACHE_PAGES_ELEM)
- elog(ERROR, "incorrect number of output arguments");
+ /* Build and return the tuple. */
+ tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
+ return HeapTupleGetDatum(tuple);
+}
- /* Construct a tuple descriptor for the result rows. */
- tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
- TupleDescInitEntry(tupledesc, (AttrNumber) 1, "bufferid",
- INT4OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 2, "relfilenode",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 3, "reltablespace",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 4, "reldatabase",
- OIDOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 5, "relforknumber",
- INT2OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 6, "relblocknumber",
- INT8OID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 7, "isdirty",
- BOOLOID, -1, 0);
- TupleDescInitEntry(tupledesc, (AttrNumber) 8, "usage_count",
- INT2OID, -1, 0);
-
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
-
- fctx->tupdesc = BlessTupleDesc(tupledesc);
-
- /* Allocate NBuffers worth of BufferCachePagesRec records. */
- fctx->record = (BufferCachePagesRec *)
- MemoryContextAllocHuge(CurrentMemoryContext,
- sizeof(BufferCachePagesRec) * NBuffers);
-
- /* Set max calls and remember the user function context. */
- funcctx->max_calls = NBuffers;
- funcctx->user_fctx = fctx;
-
- /* Return to original context when allocating transient memory */
- MemoryContextSwitchTo(oldcontext);
+Datum
+pg_buffercache_pages(PG_FUNCTION_ARGS)
+{
+ FuncCallContext *funcctx;
+ BufferCachePagesContext *fctx; /* User function context. */
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ int i;
+
+ funcctx = SRF_FIRSTCALL_INIT();
+ fctx = pg_buffercache_init_entries(funcctx, fcinfo);
/*
* Scan through all the buffers, saving the relevant fields in the
@@ -152,36 +249,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
* locks, so the information of each buffer is self-consistent.
*/
for (i = 0; i < NBuffers; i++)
- {
- BufferDesc *bufHdr;
- uint32 buf_state;
-
- bufHdr = GetBufferDescriptor(i);
- /* Lock each buffer header before inspecting. */
- buf_state = LockBufHdr(bufHdr);
-
- fctx->record[i].bufferid = BufferDescriptorGetBuffer(bufHdr);
- fctx->record[i].relfilenumber = BufTagGetRelNumber(&bufHdr->tag);
- fctx->record[i].reltablespace = bufHdr->tag.spcOid;
- fctx->record[i].reldatabase = bufHdr->tag.dbOid;
- fctx->record[i].forknum = BufTagGetForkNum(&bufHdr->tag);
- fctx->record[i].blocknum = bufHdr->tag.blockNum;
- fctx->record[i].usagecount = BUF_STATE_GET_USAGECOUNT(buf_state);
- fctx->record[i].pinning_backends = BUF_STATE_GET_REFCOUNT(buf_state);
-
- if (buf_state & BM_DIRTY)
- fctx->record[i].isdirty = true;
- else
- fctx->record[i].isdirty = false;
-
- /* Note if the buffer is valid, and has storage created */
- if ((buf_state & BM_VALID) && (buf_state & BM_TAG_VALID))
- fctx->record[i].isvalid = true;
- else
- fctx->record[i].isvalid = false;
-
- UnlockBufHdr(bufHdr, buf_state);
- }
+ pg_buffercache_save_tuple(i, fctx);
}
funcctx = SRF_PERCALL_SETUP();
@@ -191,59 +259,16 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
if (funcctx->call_cntr < funcctx->max_calls)
{
+ Datum result;
uint32 i = funcctx->call_cntr;
- Datum values[NUM_BUFFERCACHE_PAGES_ELEM];
- bool nulls[NUM_BUFFERCACHE_PAGES_ELEM];
-
- values[0] = Int32GetDatum(fctx->record[i].bufferid);
- nulls[0] = false;
-
- /*
- * Set all fields except the bufferid to null if the buffer is unused
- * or not valid.
- */
- if (fctx->record[i].blocknum == InvalidBlockNumber ||
- fctx->record[i].isvalid == false)
- {
- nulls[1] = true;
- nulls[2] = true;
- nulls[3] = true;
- nulls[4] = true;
- nulls[5] = true;
- nulls[6] = true;
- nulls[7] = true;
- /* unused for v1.0 callers, but the array is always long enough */
- nulls[8] = true;
- }
- else
- {
- values[1] = ObjectIdGetDatum(fctx->record[i].relfilenumber);
- nulls[1] = false;
- values[2] = ObjectIdGetDatum(fctx->record[i].reltablespace);
- nulls[2] = false;
- values[3] = ObjectIdGetDatum(fctx->record[i].reldatabase);
- nulls[3] = false;
- values[4] = ObjectIdGetDatum(fctx->record[i].forknum);
- nulls[4] = false;
- values[5] = Int64GetDatum((int64) fctx->record[i].blocknum);
- nulls[5] = false;
- values[6] = BoolGetDatum(fctx->record[i].isdirty);
- nulls[6] = false;
- values[7] = Int16GetDatum(fctx->record[i].usagecount);
- nulls[7] = false;
- /* unused for v1.0 callers, but the array is always long enough */
- values[8] = Int32GetDatum(fctx->record[i].pinning_backends);
- nulls[8] = false;
- }
-
- /* Build and return the tuple. */
- tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = get_buffercache_tuple(i, fctx);
SRF_RETURN_NEXT(funcctx, result);
}
else
+ {
SRF_RETURN_DONE(funcctx);
+ }
}
Datum
--
2.39.5
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-31 11:37 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 1 reply; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-31 11:37 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, Mar 28, 2025 at 11:43 PM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-28, Tom Lane wrote:
>
> > I think instead of going this direction, we really need to create a
> > separately-purposed script that simply creates "one of everything"
> > without doing anything else (except maybe loading a little data).
> > I believe it'd be a lot easier to remember to add to that when
> > inventing new SQL than to remember to leave something behind from the
> > core regression tests. This would also be far faster to run than any
> > approach that involves picking a random subset of the core test
> > scripts.
It's easier to remember to do something or not do something in the
same file than in some other file. I find it hard to believe that
introducing another set of SQL files somewhere far from regress would
make this problem easier.
The number of states in which objects can be left behind in the
regress/sql is very large - and maintaining that 1:1 in some other set
of scripts is impossible unless it's automated.
> FWIW this sounds closely related to what I tried to do with
> src/test/modules/test_ddl_deparse; it's currently incomplete, but maybe
> we can use that as a starting point.
create_table.sql in test_ddl_deparse has only one statement creating
an inheritance table whereas there are dozens of different states of
parent/child tables created by regress. It will require a lot of work
to bridge the gap between regress_ddl_deparse and regress and more
work to maintain it.
I might be missing something in your ideas.
IMO, whatever we do it should rely on a single set of files. One
possible way could be to break the existing files into three files
each, containing DDL, DML and queries from those files respectively
and create three schedules DDL, DML and queries containing the
respective files. These schedules will be run as required. Standard
regression run runs all the three schedules one by one. But
002_pg_upgrade will run DDL and DML on the source database and run
queries on target - thus checking sanity of the dump/restore or
pg_upgrade beyond just the dump comparison. 027_stream_regress might
run DDL, DML on the source server and queries on the target.
But that too is easier said than done for:
1. Our tests mix all three kinds of statements and also rely on the
order in which they are run. It will require some significant effort
to carefully separate the statements.
2. With the new set of files backpatching would become hard.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-31 11:54 Ashutosh Bapat <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Ashutosh Bapat @ 2025-03-31 11:54 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Mar 31, 2025 at 5:07 PM Ashutosh Bapat
<[email protected]> wrote:
>
The bug related to materialized views has been fixed and now the test
passes even if we compare statistics from dumped and restored
databases. Hence removing 0003. In the attached patchset I have also
addressed Vignesh's below comment
On Thu, Mar 27, 2025 at 10:01 PM Ashutosh Bapat
<[email protected]> wrote:
>
> On Thu, Mar 27, 2025 at 6:01 PM vignesh C <[email protected]> wrote:
> >
> > 2) Should "`" be ' or " here, we generally use "`" to enclose commands:
> > +# It is expected that regression tests, which create `regression` database, are
> > +# run on `src_node`, which in turn, is left in running state. A fresh node is
> > +# created using given `node_params`, which are expected to be the
> > same ones used
> > +# to create `src_node`, so as to avoid any differences in the databases.
>
> Looking at prologues or some other functions, I see that we don't add
> any decoration around the name of the argument. Hence dropped ``
> altogether. Will post it with the next set of patches.
--
Best Wishes,
Ashutosh Bapat
Attachments:
[text/x-patch] 0002-Use-only-one-format-and-make-the-test-run-d-20250331.patch (5.4K, ../../CAExHW5vVFtCejh+UYzNxMGSXOfJ_1xwi5aQHQfemqJgFmkyK5Q@mail.gmail.com/2-0002-Use-only-one-format-and-make-the-test-run-d-20250331.patch)
download | inline diff:
From 5ef4a15bf229d104028eac3a046636453e1e05fc Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 24 Mar 2025 11:21:12 +0530
Subject: [PATCH 2/2] Use only one format and make the test run default
According to Alvaro (and I agree with him), the test should be run by
default. Otherwise we get to know about a bug only after buildfarm
animal where it's enabled reports a failure. Further testing only one
format may suffice; since all the formats have shown the same bugs till
now.
If we use --directory format we can use -j which reduces the time taken
by dump/restore test by about 12%.
This patch removes PG_TEST_EXTRA option as well as runs the test only in
directory format with parallelism enabled.
Note for committer: If we decide to accept this change, it should be
merged with the previous commit.
---
doc/src/sgml/regress.sgml | 12 ----
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 76 +++++++++-----------------
2 files changed, 25 insertions(+), 63 deletions(-)
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 237b974b3ab..0e5e8e8f309 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,18 +357,6 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
-
- <varlistentry>
- <term><literal>regress_dump_test</literal></term>
- <listitem>
- <para>
- When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
- tests dump and restore of regression database left behind by the
- regression run. Not enabled by default because it is time and resource
- consuming.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 8d22d538529..f7d5b96ecd2 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -268,16 +268,9 @@ else
# should be done in a separate TAP test, but doing it here saves us one full
# regression run.
#
- # This step takes several extra seconds and some extra disk space, so
- # requires an opt-in with the PG_TEST_EXTRA environment variable.
- #
# Do this while the old cluster is running before it is shut down by the
# upgrade test.
- if ( $ENV{PG_TEST_EXTRA}
- && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
- {
- test_regression_dump_restore($oldnode, %node_params);
- }
+ test_regression_dump_restore($oldnode, %node_params);
}
# Initialize a new node for the upgrade.
@@ -590,53 +583,34 @@ sub test_regression_dump_restore
$dst_node->append_conf('postgresql.conf', 'autovacuum = off');
$dst_node->start;
- # Test all formats one by one.
- for my $format ('plain', 'tar', 'directory', 'custom')
- {
- my $dump_file = "$tempdir/regression_dump.$format";
- my $restored_db = 'regression_' . $format;
-
- # Use --create in dump and restore commands so that the restored
- # database has the same configurable variable settings as the original
- # database and the plain dumps taken for comparsion do not differ
- # because of locale changes. Additionally this provides test coverage
- # for --create option.
- $src_node->command_ok(
- [
- 'pg_dump', "-F$format", '--no-sync',
- '-d', $src_node->connstr('regression'),
- '--create', '-f', $dump_file
- ],
- "pg_dump on source instance in $format format");
+ my $dump_file = "$tempdir/regression.dump";
- my @restore_command;
- if ($format eq 'plain')
- {
- # Restore dump in "plain" format using `psql`.
- @restore_command = [ 'psql', '-d', 'postgres', '-f', $dump_file ];
- }
- else
- {
- @restore_command = [
- 'pg_restore', '--create',
- '-d', 'postgres', $dump_file
- ];
- }
- $dst_node->command_ok(@restore_command,
- "restored dump taken in $format format on destination instance");
+ # Use --create in dump and restore commands so that the restored database
+ # has the same configurable variable settings as the original database so
+ # that the plain dumps taken from both the database taken for comparisong do
+ # not differ because of locale changes. Additionally this provides test
+ # coverage for --create option.
+ #
+ # We use directory format which allows dumping and restoring in parallel to
+ # reduce the test's run time.
+ $src_node->command_ok(
+ [
+ 'pg_dump', '-Fd', '-j2', '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance succeeded");
- my $dst_dump =
- get_dump_for_comparison($dst_node, 'regression',
- 'dest_dump.' . $format, 0);
+ $dst_node->command_ok(
+ [ 'pg_restore', '--create', '-j2', '-d', 'postgres', $dump_file ],
+ "restored dump to destination instance");
- compare_files($src_dump, $dst_dump,
- "dump outputs from original and restored regression database (using $format format) match"
- );
+ my $dst_dump = get_dump_for_comparison($dst_node, 'regression',
+ 'dest_dump', 0);
- # Rename the restored database so that it is available for debugging in
- # case the test fails.
- $dst_node->safe_psql('postgres', "ALTER DATABASE regression RENAME TO $restored_db");
- }
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database match"
+ );
}
# Dump database db from the given node in plain format and adjust it for
--
2.34.1
[text/x-patch] 0001-Test-pg_dump-restore-of-regression-objects-20250331.patch (16.5K, ../../CAExHW5vVFtCejh+UYzNxMGSXOfJ_1xwi5aQHQfemqJgFmkyK5Q@mail.gmail.com/3-0001-Test-pg_dump-restore-of-regression-objects-20250331.patch)
download | inline diff:
From aa1c74951b3b557de8330230185fd5f2ee46ecda Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Thu, 27 Jun 2024 10:03:53 +0530
Subject: [PATCH 1/2] Test pg_dump/restore of regression objects
002_pg_upgrade.pl tests pg_upgrade of the regression database left
behind by regression run. Modify it to test dump and restore of the
regression database as well.
Regression database created by regression run contains almost all the
database objects supported by PostgreSQL in various states. Hence the
new testcase covers dump and restore scenarios not covered by individual
dump/restore cases. Till now 002_pg_upgrade only tested dump/restore
through pg_upgrade which only uses binary mode. Many regression tests
mention that they leave objects behind for dump/restore testing but they
are not tested in a non-binary mode. The new testcase closes that
gap.
Testing dump and restore of regression database makes this test run
longer for a relatively smaller benefit. Hence run it only when
explicitly requested by user by specifying "regress_dump_test" in
PG_TEST_EXTRA.
Note For the reviewers:
The new test has uncovered many bugs so far in one year.
1. Introduced by 14e87ffa5c54. Fixed in fd41ba93e4630921a72ed5127cd0d552a8f3f8fc.
2. Introduced by 0413a556990ba628a3de8a0b58be020fd9a14ed0. Reverted in 74563f6b90216180fc13649725179fc119dddeb5.
3. Fixed by d611f8b1587b8f30caa7c0da99ae5d28e914d54f
3. Being discussed on hackers at https://www.postgresql.org/message-id/CAExHW5s47kmubpbbRJzSM-Zfe0Tj2O3GBagB7YAyE8rQ-V24Uw@mail.gmail.com
Author: Ashutosh Bapat
Reviewed by: Michael Pacquire, Daniel Gustafsson, Tom Lane, Alvaro Herrera
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2Wf61PT+hfquzjBqouRzQJQ@mail.gmail.com
---
doc/src/sgml/regress.sgml | 12 ++
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 144 ++++++++++++++++-
src/test/perl/Makefile | 2 +
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 167 ++++++++++++++++++++
src/test/perl/meson.build | 1 +
5 files changed, 324 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/AdjustDump.pm
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 0e5e8e8f309..237b974b3ab 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -357,6 +357,18 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>regress_dump_test</literal></term>
+ <listitem>
+ <para>
+ When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
+ tests dump and restore of regression database left behind by the
+ regression run. Not enabled by default because it is time and resource
+ consuming.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 00051b85035..8d22d538529 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -12,6 +12,7 @@ use File::Path qw(rmtree);
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::AdjustUpgrade;
+use PostgreSQL::Test::AdjustDump;
use Test::More;
# Can be changed to test the other modes.
@@ -35,8 +36,8 @@ sub generate_db
"created database with ASCII characters from $from_char to $to_char");
}
-# Filter the contents of a dump before its use in a content comparison.
-# This returns the path to the filtered dump.
+# Filter the contents of a dump before its use in a content comparison for
+# upgrade testing. This returns the path to the filtered dump.
sub filter_dump
{
my ($is_old, $old_version, $dump_file) = @_;
@@ -262,6 +263,21 @@ else
}
}
is($rc, 0, 'regression tests pass');
+
+ # Test dump/restore of the objects left behind by regression. Ideally it
+ # should be done in a separate TAP test, but doing it here saves us one full
+ # regression run.
+ #
+ # This step takes several extra seconds and some extra disk space, so
+ # requires an opt-in with the PG_TEST_EXTRA environment variable.
+ #
+ # Do this while the old cluster is running before it is shut down by the
+ # upgrade test.
+ if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
+ {
+ test_regression_dump_restore($oldnode, %node_params);
+ }
}
# Initialize a new node for the upgrade.
@@ -539,4 +555,128 @@ my $dump2_filtered = filter_dump(0, $oldnode->pg_version, $dump2_file);
compare_files($dump1_filtered, $dump2_filtered,
'old and new dumps match after pg_upgrade');
+# Test dump and restore of objects left behind by the regression run.
+#
+# It is expected that regression tests, which create 'regression' database, are
+# run on src_node, which in turn, is left in running state. A fresh node is
+# created using given node_params, which are expected to be the same ones use
+# to create src_node, so as to avoid any differences in the databases.
+#
+# Plain dumps from both the nodes are compared to make sure that all the dumped
+# objects are restored faithfully.
+sub test_regression_dump_restore
+{
+ my ($src_node, %node_params) = @_;
+ my $dst_node = PostgreSQL::Test::Cluster->new('dst_node');
+
+ # Make sure that the source and destination nodes have the same version and
+ # do not use custom install paths. In both the cases, the dump files may
+ # require additional adjustments unknown to code here. Do not run this test
+ # in such a case to avoid utilizing the time and resources unnecessarily.
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default installation");
+ return;
+ }
+
+ # Dump the original database for comparison later.
+ my $src_dump =
+ get_dump_for_comparison($src_node, 'regression', 'src_dump', 1);
+
+ # Setup destination database cluster
+ $dst_node->init(%node_params);
+ # Stabilize stats for comparison.
+ $dst_node->append_conf('postgresql.conf', 'autovacuum = off');
+ $dst_node->start;
+
+ # Test all formats one by one.
+ for my $format ('plain', 'tar', 'directory', 'custom')
+ {
+ my $dump_file = "$tempdir/regression_dump.$format";
+ my $restored_db = 'regression_' . $format;
+
+ # Use --create in dump and restore commands so that the restored
+ # database has the same configurable variable settings as the original
+ # database and the plain dumps taken for comparsion do not differ
+ # because of locale changes. Additionally this provides test coverage
+ # for --create option.
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '--create', '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [ 'psql', '-d', 'postgres', '-f', $dump_file ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '--create',
+ '-d', 'postgres', $dump_file
+ ];
+ }
+ $dst_node->command_ok(@restore_command,
+ "restored dump taken in $format format on destination instance");
+
+ my $dst_dump =
+ get_dump_for_comparison($dst_node, 'regression',
+ 'dest_dump.' . $format, 0);
+
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database (using $format format) match"
+ );
+
+ # Rename the restored database so that it is available for debugging in
+ # case the test fails.
+ $dst_node->safe_psql('postgres', "ALTER DATABASE regression RENAME TO $restored_db");
+ }
+}
+
+# Dump database db from the given node in plain format and adjust it for
+# comparing dumps from the original and the restored database.
+#
+# file_prefix is used to create unique names for all dump files so that they
+# remain available for debugging in case the test fails.
+#
+# adjust_child_columns is passed to adjust_regress_dumpfile() which actually
+# adjusts the dump output.
+#
+# The name of the file containting adjusted dump is returned.
+sub get_dump_for_comparison
+{
+ my ($node, $db, $file_prefix, $adjust_child_columns) = @_;
+
+ my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
+ my $dump_adjusted = "${dumpfile}_adjusted";
+
+ # Usually we avoid comparing statistics in our tests since it is flaky by
+ # nature. However, if statistics is dumped and restored it is expected to be
+ # restored as it is i.e. the statistics from the original database and that
+ # from the restored database should match. We turn off autovacuum on the
+ # source and the target database to avoid any statistics update during
+ # restore operation. Hence we do not exclude statistics from dump.
+ $node->command_ok(
+ [
+ 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ $dumpfile
+ ],
+ 'dump for comparison succeeded');
+
+ open(my $dh, '>', $dump_adjusted)
+ || die
+ "could not open $dump_adjusted for writing the adjusted dump: $!";
+ print $dh adjust_regress_dumpfile(slurp_file($dumpfile),
+ $adjust_child_columns);
+ close($dh);
+
+ return $dump_adjusted;
+}
+
done_testing();
diff --git a/src/test/perl/Makefile b/src/test/perl/Makefile
index d82fb67540e..def89650ead 100644
--- a/src/test/perl/Makefile
+++ b/src/test/perl/Makefile
@@ -26,6 +26,7 @@ install: all installdirs
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/Cluster.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/BackgroundPsql.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustUpgrade.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustDump.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Version.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
uninstall:
@@ -36,6 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
endif
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..74b9a60cf34
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,167 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::AdjustDump - helper module for dump and restore tests
+
+=head1 SYNOPSIS
+
+ use PostgreSQL::Test::AdjustDump;
+
+ # Adjust contents of dump output file so that dump output from original
+ # regression database and that from the restored regression database match
+ $dump = adjust_regress_dumpfile($dump, $adjust_child_columns);
+
+=head1 DESCRIPTION
+
+C<PostgreSQL::Test::AdjustDump> encapsulates various hacks needed to
+compare the results of dump and restore tests
+
+=cut
+
+package PostgreSQL::Test::AdjustDump;
+
+use strict;
+use warnings FATAL => 'all';
+
+use Exporter 'import';
+use Test::More;
+
+our @EXPORT = qw(
+ adjust_regress_dumpfile
+);
+
+=pod
+
+=head1 ROUTINES
+
+=over
+
+=item $dump = adjust_regress_dumpfile($dump, $adjust_child_columns)
+
+If we take dump of the regression database left behind after running regression
+tests, restore the dump, and take dump of the restored regression database, the
+outputs of both the dumps differ in the following cases. This routine adjusts
+the given dump so that dump outputs from the original and restored database,
+respectively, match.
+
+Case 1: Some regression tests purposefully create child tables in such a way
+that the order of their inherited columns differ from column orders of their
+respective parents. In the restored database, however, the order of their
+inherited columns are same as that of their respective parents. Thus the column
+orders of these child tables in the original database and those in the restored
+database differ, causing difference in the dump outputs. See MergeAttributes()
+and dumpTableSchema() for details. This routine rearranges the column
+declarations in the relevant C<CREATE TABLE... INHERITS> statements in the dump
+file from original database to match those from the restored database. We could,
+instead, adjust the statements in the dump from the restored database to match
+those from original database or adjust both to a canonical order. But we have
+chosen to adjust the statements in the dump from original database for no
+particular reason.
+
+Case 2: When dumping COPY statements the columns are ordered by their attribute
+number by fmtCopyColumnList(). If a column is added to a parent table after a
+child has inherited the parent and the child has its own columns, the attribute
+number of the column changes after restoring the child table. This is because
+when executing the dumped C<CREATE TABLE... INHERITS> statement all the parent
+attributes are created before any child attributes. Thus the order of columns in
+COPY statements dumped from the original and the restored databases,
+respectively, differs. Such tables in regression tests are listed below. It is
+hard to adjust the column order in the COPY statement along with the data. Hence
+we just remove such COPY statements from the dump output.
+
+Additionally the routine adjusts blank and new lines to avoid noise.
+
+Note: Usually we avoid comparing statistics in our tests since it is flaky by
+nature. However, if statistics is dumped and restored it is expected to be
+restored as it is i.e. the statistics from the original database and that from
+the restored database should match. Hence we do not filter statistics from dump,
+if it's dumped.
+
+Arguments:
+
+=over
+
+=item C<dump>: Contents of dump file
+
+=item C<adjust_child_columns>: 1 indicates that the given dump file requires
+adjusting columns in the child tables; usually when the dump is from original
+database. 0 indicates no such adjustment is needed; usually when the dump is
+from restored database.
+
+=back
+
+Returns the adjusted dump text.
+
+=cut
+
+sub adjust_regress_dumpfile
+{
+ my ($dump, $adjust_child_columns) = @_;
+
+ # use Unix newlines
+ $dump =~ s/\r\n/\n/g;
+
+ # Adjust the CREATE TABLE ... INHERITS statements.
+ if ($adjust_child_columns)
+ {
+ my $saved_dump = $dump;
+
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_stored_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_stored_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_virtual_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_virtual_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c1\s\()
+ (\n\s+int_four\sbigint),
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint)/$1$4,$2,$3/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c1 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c2\s\()
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint),
+ (\n\s+int_four\sbigint)/$1$3,$4,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c2 adjustments');
+ }
+
+ # Remove COPY statements with differing column order
+ for my $table (
+ 'public\.b_star', 'public\.c_star',
+ 'public\.cc2', 'public\.d_star',
+ 'public\.e_star', 'public\.f_star',
+ 'public\.renamecolumnanother', 'public\.renamecolumnchild',
+ 'public\.test_type_diff2_c1', 'public\.test_type_diff2_c2',
+ 'public\.test_type_diff_c')
+ {
+ $dump =~ s/^COPY\s$table\s\(.+?^\\\.$//sm;
+ }
+
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
+ return $dump;
+}
+
+=pod
+
+=back
+
+=cut
+
+1;
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
index 58e30f15f9d..492ca571ff8 100644
--- a/src/test/perl/meson.build
+++ b/src/test/perl/meson.build
@@ -14,4 +14,5 @@ install_data(
'PostgreSQL/Test/Cluster.pm',
'PostgreSQL/Test/BackgroundPsql.pm',
'PostgreSQL/Test/AdjustUpgrade.pm',
+ 'PostgreSQL/Test/AdjustDump.pm',
install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
base-commit: e2809e3a1015697832ee4d37b75ba1cd0caac0f0
--
2.34.1
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Draft for basic NUMA observability
@ 2025-03-31 14:59 Bertrand Drouvot <[email protected]>
parent: Jakub Wartak <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: Bertrand Drouvot @ 2025-03-31 14:59 UTC (permalink / raw)
To: Jakub Wartak <[email protected]>; +Cc: Andres Freund <[email protected]>; Alvaro Herrera <[email protected]>; Nazir Bilal Yavuz <[email protected]>; pgsql-hackers
Hi,
On Mon, Mar 31, 2025 at 11:27:50AM +0200, Jakub Wartak wrote:
> On Thu, Mar 27, 2025 at 2:40 PM Andres Freund <[email protected]> wrote:
> >
> > > +Size
> > > +pg_numa_get_pagesize(void)
> [..]
> >
> > Should this have a comment or an assertion that it can only be used after
> > shared memory startup? Because before that huge_pages_status won't be
> > meaningful?
>
> Added both.
Thanks for the updated version!
+ Assert(IsUnderPostmaster);
I wonder if that would make more sense to add an assertion on huge_pages_status
and HUGE_PAGES_UNKNOWN instead (more or less as it is done in
CreateSharedMemoryAndSemaphores()).
=== About v17-0002-This-extracts-code-from-contrib-pg_buffercache-s.patch
Once applied I can see mention to pg_buffercache_numa_pages() while it
only comes in v17-0003-Extend-pg_buffercache-with-new-view-pg_buffercac.patch.
I think pg_buffercache_numa_pages() should not be mentioned before it's actually
implemented.
=== 1
+ bufRecord->isvalid == false)
{
int i;
- funcctx = SRF_FIRSTCALL_INIT();
-
- /* Switch context when allocating stuff to be used in later calls */
- oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
-
- /* Create a user function context for cross-call persistence */
- fctx = (BufferCachePagesContext *) palloc(sizeof(BufferCachePagesContext));
+ for (i = 1; i <= 9; i++)
+ nulls[i] = true;
"i <= 9" will be correct only once v17-0003 is applied (when NUM_BUFFERCACHE_PAGES_ELEM
is increased to 10).
In v17-0002 that should be i < 9 (even better i < NUM_BUFFERCACHE_PAGES_ELEM).
That could also make sense to remove the loop and use memset() that way:
"
memset(&nulls[1], true, (NUM_BUFFERCACHE_PAGES_ELEM - 1) * sizeof(bool));
"
instead. It's done that way in some other places (hbafuncs.c for example).
=== 2
- if (expected_tupledesc->natts == NUM_BUFFERCACHE_PAGES_ELEM)
- TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
- INT4OID, -1, 0);
+ if (expected_tupledesc->natts >= NUM_BUFFERCACHE_PAGES_ELEM - 1)
+ TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
+ INT4OID, -1, 0);
I think we should not change the "expected_tupledesc->natts" check here until
v17-0003 is applied.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-03-31 21:17 Daniel Gustafsson <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: Daniel Gustafsson @ 2025-03-31 21:17 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Tom Lane <[email protected]>; Ashutosh Bapat <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
> On 28 Mar 2025, at 19:12, Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Mar-28, Tom Lane wrote:
>
>> I think instead of going this direction, we really need to create a
>> separately-purposed script that simply creates "one of everything"
>> without doing anything else (except maybe loading a little data).
>> I believe it'd be a lot easier to remember to add to that when
>> inventing new SQL than to remember to leave something behind from the
>> core regression tests. This would also be far faster to run than any
>> approach that involves picking a random subset of the core test
>> scripts.
>
> FWIW this sounds closely related to what I tried to do with
> src/test/modules/test_ddl_deparse; it's currently incomplete, but maybe
> we can use that as a starting point.
Given where we are in the cycle, it seems to make sense to stick to using the
schedule we already have rather than invent a new process for generating it,
and work on that for 19?
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-04-03 03:59 vignesh C <[email protected]>
0 siblings, 0 replies; 95+ messages in thread
From: vignesh C @ 2025-04-03 03:59 UTC (permalink / raw)
To: Ashutosh Bapat <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, 2 Apr 2025 at 13:49, Ashutosh Bapat
<[email protected]> wrote:
>
> On Tue, Apr 1, 2025 at 10:31 PM Alvaro Herrera <[email protected]> wrote:
> >
> > On 2025-Apr-01, Ashutosh Bapat wrote:
> >
> > > Just today morning, I found something which looks like another bug in
> > > statistics dump/restore [1]. As Daniel has expressed upthread [2], we
> > > should go ahead and commit the test even if the bug is not fixed. But
> > > in case it creates a lot of noise and makes the build farm red, we
> > > could suppress the failure by not dumping statistics for comparison
> > > till the bug is fixed. PFA patchset which reintroduces 0003 which
> > > suppresses the statistics dump - in case we think it's needed. I have
> > > made some minor cosmetic changes to 0001 and 0002 as well.
> >
> > I have made some changes of my own, and included --no-statistics.
> > But I had already started messing with your patch, so I didn't look at
> > the cosmetic changes you did here. If they're still relevant, please
> > send them my way.
>
> Thanks a lot. I hope the test will now reveal the problems before they
> are committed :)
>
> You have edited those places anyway. So it's ok.
>
> I have closed the CF entry
> https://commitfest.postgresql.org/patch/4564/ committed. I will
> create another CF entry to park --no-statistics reversal change. That
> way, we will know when statistics dump/restore has become stable.
>
> >
> > Hopefully it won't break, and if it does, it's likely fault of the
> > changes I made. I've run it through CI and all is well though, so
> > fingers crossed.
> > https://cirrus-ci.com/build/6327169669922816
> >
> >
> > I observe in the CI results that the pg_upgrade test is not necessarily
> > the last one to finish. In one case it even finished in place 12!
> >
> > [16:36:48.447] 12/332 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade OK 112.16s 22 subtests passed
> > https://api.cirrus-ci.com/v1/task/5803071017582592/logs/test_world.log
>
> Yes. Few animals that I sampled, the test is finishing pretty early
> even though it's taking longer than many other tests. But it's not the
> longest. I also looked at red animals, but none of them report this
> test to be failing.
I believe this commitfest entry at [1] can be closed now, as the
buildfarm has been running stably for the past few days.
[1] - https://commitfest.postgresql.org/patch/4956/
Regards,
Vignesh
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-04-03 13:40 Andres Freund <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Andres Freund @ 2025-04-03 13:40 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 2025-04-03 10:20:09 +0200, Alvaro Herrera wrote:
> On 2025-Apr-03, Ashutosh Bapat wrote:
>
> > Looks like the problem is in the test itself as pointed out by Jeff in
> > [1]. PFA patch fixing the test and enabling statistics back.
>
> Thanks, pushed.
Since then the pg_upgrade tests have been failing on skink/valgrind, due to
exceeding the already substantially increased timeout.
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=skink&dt=2025-04-03%2007%3A06%3A19...
(note that there are other issues in that run)
284/333 postgresql:pg_upgrade / pg_upgrade/002_pg_upgrade TIMEOUT 10000.66s killed by signal 15 SIGTERM
[10:38:19.815](16.712s) ok 20 - check that locales in new cluster match original cluster
...
# Running: pg_dumpall --no-sync --dbname port=15114 host=/tmp/bh_AdT5uvQ dbname='postgres' --file /home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/pg_upgrade/002_pg_upgrade/data/tmp_test_gp2G/dump2.sql
death by signal at /home/bf/bf-build/skink-master/HEAD/pgsql/src/test/perl/PostgreSQL/Test/Cluster.pm line 181.
...
[10:44:11.720](351.905s) # Tests were run but no plan was declared and done_testing() was not seen.
I've increased the timeout even further, but I can't say that I am happy about
the slowest test getting even slower. Adding test time in the serially slowest
test is way worse than adding the same time in a concurrent test.
I suspect that the test will go a bit faster if log_statement weren't forced
on, printing that many log lines, with context, does make valgrind slower,
IME. But Cluster.pm forces it to on, and I suspect that putting a global
log_statement=false into TEMP_CONFIG would have it's own disadvantages.
/me and checks prices for increasing the size of skink's host.
Greetings,
Andres
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-04-04 16:07 Andres Freund <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Andres Freund @ 2025-04-04 16:07 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 2025-04-04 12:01:16 -0400, Andres Freund wrote:
> FWIW, for me 027 is actually considerably faster. In an cassert -O0 build (my
> normal development env, I find even -Og too problematic for debugging):
>
> pg_upgrade/002_pg_upgrade 96.61s
> recovery/027_stream_regress 66.04s
>
> After
> git revert 8806e4e8deb1e21715e031e17181d904825a410e abe56227b2e213755dd3e194c530f5f06467bd7c 172259afb563d35001410dc6daad78b250924038
>
> pg_upgrade/002_pg_upgrade 75.09s
>
> Slower by 29%, far from the 3s increased time I saw mentioned somewhere.
>
>
> And this really affects the overall test time:
>
> All tests before:
> real 1m38.173s
> user 5m52.500s
> sys 4m23.574s
>
> All tests after:
> real 2m0.397s
> user 5m53.625s
> sys 4m30.518s
>
> The CPU time increase is rather minimal, but the wall clock time increase is
> 22%.
>
> 17:
> real 1m14.822s
> user 4m2.630s
> sys 3m22.384s
>
> We regressed wall clock time *60%* from 17->18. Some test cycle increase is
> reasonable and can largely be compensated with hardware, but this cycle we're
> growing way faster than hardware gets faster. I don't think that's
> sustainable.
FWIW, with cassert and -O2, it's:
17:
real 0m53.981s
user 3m22.837s
sys 3m24.237s
HEAD:
real 1m19.749s
user 4m54.526s
sys 4m15.657s
so this isn't just due to me using -O0. A 48% increase is better than a 60%
increase, but it's still not sustainable.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-08-05 14:33 Alvaro Herrera <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Alvaro Herrera @ 2025-08-05 14:33 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Ashutosh Bapat <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Hello,
On 2025-Apr-04, Andres Freund wrote:
> FWIW, with cassert and -O2, it's:
>
> 17:
> real 0m53.981s
> user 3m22.837s
> sys 3m24.237s
>
> HEAD:
> real 1m19.749s
> user 4m54.526s
> sys 4m15.657s
>
> so this isn't just due to me using -O0. A 48% increase is better than a 60%
> increase, but it's still not sustainable.
I happened to notice that this item was still open in the commitfest,
and rereading the thread I now have second thoughts about having it
enabled by default, giving your complaints about speed. How about
applying this to 18 and master?
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"This is a foot just waiting to be shot" (Andrew Dunstan)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-08-05 14:41 Daniel Gustafsson <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Daniel Gustafsson @ 2025-08-05 14:41 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Andres Freund <[email protected]>; Ashutosh Bapat <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
> On 5 Aug 2025, at 16:33, Alvaro Herrera <[email protected]> wrote:
> I happened to notice that this item was still open in the commitfest,
> and rereading the thread I now have second thoughts about having it
> enabled by default, giving your complaints about speed. How about
> applying this to 18 and master?
Thanks for reviving this. I am +1 on placing this behind PG_TEST_EXTRA as was
discussed upthread.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-08-05 14:59 Tom Lane <[email protected]>
parent: Daniel Gustafsson <[email protected]>
0 siblings, 1 reply; 95+ messages in thread
From: Tom Lane @ 2025-08-05 14:59 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Ashutosh Bapat <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
Daniel Gustafsson <[email protected]> writes:
> Thanks for reviving this. I am +1 on placing this behind PG_TEST_EXTRA as was
> discussed upthread.
+1 here too.
regards, tom lane
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-08-05 18:11 Alvaro Herrera <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 95+ messages in thread
From: Alvaro Herrera @ 2025-08-05 18:11 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Andres Freund <[email protected]>; Ashutosh Bapat <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2025-Aug-05, Tom Lane wrote:
> Daniel Gustafsson <[email protected]> writes:
> > Thanks for reviving this. I am +1 on placing this behind PG_TEST_EXTRA as was
> > discussed upthread.
>
> +1 here too.
Cool, thanks, done. Now we need a volunteer to set up a buildfarm
animal with this flag ...
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"If it is not right, do not do it.
If it is not true, do not say it." (Marcus Aurelius, Meditations)
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-08-06 01:22 Michael Paquier <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: Michael Paquier @ 2025-08-06 01:22 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Andres Freund <[email protected]>; Ashutosh Bapat <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Aug 05, 2025 at 08:11:41PM +0200, Alvaro Herrera wrote:
> Cool, thanks, done. Now we need a volunteer to set up a buildfarm
> animal with this flag ...
Sure. I have added regress_dump_restore to the configuration of
batta, hachi and gokiburi.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 95+ messages in thread
* Re: Test to dump and restore objects left behind by regression
@ 2025-08-06 15:10 Ashutosh Bapat <[email protected]>
parent: Alvaro Herrera <[email protected]>
1 sibling, 0 replies; 95+ messages in thread
From: Ashutosh Bapat @ 2025-08-06 15:10 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; vignesh C <[email protected]>; Peter Eisentraut <[email protected]>; PostgreSQL Hackers <[email protected]>
On Wed, Aug 6, 2025 at 8:21 AM Alvaro Herrera <[email protected]> wrote:
>
> On 2025-Aug-05, Tom Lane wrote:
>
> > Daniel Gustafsson <[email protected]> writes:
> > > Thanks for reviving this. I am +1 on placing this behind PG_TEST_EXTRA as was
> > > discussed upthread.
> >
> > +1 here too.
>
> Cool, thanks, done. Now we need a volunteer to set up a buildfarm
> animal with this flag ...
Your patch didn't contain doc changes. But the commit has it. Thanks.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 95+ messages in thread
end of thread, other threads:[~2025-08-06 15:10 UTC | newest]
Thread overview: 95+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-09-25 20:32 [PATCH v7 05/16] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v6 05/15] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v2 05/11] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v4 05/12] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v3 05/11] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v5 05/14] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v2 05/11] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v4 05/12] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2023-09-25 20:32 [PATCH v3 05/11] Improve sentences in overview of system configuration parameters Karl O. Pinc <[email protected]>
2025-02-13 15:28 Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-02-17 12:02 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-02-20 08:32 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-02-24 11:57 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-02-24 14:06 ` Re: Draft for basic NUMA observability Andres Freund <[email protected]>
2025-02-24 16:11 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-02-26 08:48 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-02-26 08:38 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-02-26 09:58 ` Re: Draft for basic NUMA observability Andres Freund <[email protected]>
2025-02-26 13:05 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-02-26 17:13 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-02-27 09:05 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-02-27 15:34 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-04 10:48 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-04 16:02 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-05 09:30 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-07 10:20 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-07 11:33 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-10 10:14 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-12 15:41 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-13 09:22 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-13 14:15 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-13 15:38 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-14 10:05 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-14 12:08 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-17 07:28 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-17 16:11 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-18 10:19 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-18 14:29 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-19 09:06 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-27 11:30 ` Re: Draft for basic NUMA observability Nazir Bilal Yavuz <[email protected]>
2025-03-27 13:02 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-27 13:15 ` Re: Draft for basic NUMA observability Álvaro Herrera <[email protected]>
2025-03-31 09:27 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-27 13:40 ` Re: Draft for basic NUMA observability Andres Freund <[email protected]>
2025-03-31 09:27 ` Re: Draft for basic NUMA observability Jakub Wartak <[email protected]>
2025-03-31 14:59 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-02-27 15:42 ` Re: Draft for basic NUMA observability Bertrand Drouvot <[email protected]>
2025-03-11 10:44 Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-12 12:05 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-12 15:58 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-12 16:09 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-13 06:52 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-13 08:42 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-13 12:39 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-13 12:40 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-19 11:43 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-20 15:06 ` Re: Test to dump and restore objects left behind by regression vignesh C <[email protected]>
2025-03-20 16:39 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-21 12:45 ` Re: Test to dump and restore objects left behind by regression vignesh C <[email protected]>
2025-03-21 13:09 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-21 14:39 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-21 16:03 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-21 18:07 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-24 09:54 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-24 09:59 ` Re: Test to dump and restore objects left behind by regression Daniel Gustafsson <[email protected]>
2025-03-24 12:14 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-25 10:39 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-27 12:31 ` Re: Test to dump and restore objects left behind by regression vignesh C <[email protected]>
2025-03-27 16:31 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-27 17:15 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-28 01:36 ` Re: Test to dump and restore objects left behind by regression Michael Paquier <[email protected]>
2025-03-28 06:50 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-28 12:27 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-28 14:11 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-28 14:22 ` Re: Test to dump and restore objects left behind by regression Tom Lane <[email protected]>
2025-03-28 18:12 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-31 11:37 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-31 11:54 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-31 21:17 ` Re: Test to dump and restore objects left behind by regression Daniel Gustafsson <[email protected]>
2025-03-28 06:32 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-28 09:28 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-28 10:35 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-28 11:20 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-21 11:51 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-03-21 12:34 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-03-21 12:41 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
2025-04-03 03:59 Re: Test to dump and restore objects left behind by regression vignesh C <[email protected]>
2025-04-03 13:40 Re: Test to dump and restore objects left behind by regression Andres Freund <[email protected]>
2025-04-04 16:07 ` Re: Test to dump and restore objects left behind by regression Andres Freund <[email protected]>
2025-08-05 14:33 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-08-05 14:41 ` Re: Test to dump and restore objects left behind by regression Daniel Gustafsson <[email protected]>
2025-08-05 14:59 ` Re: Test to dump and restore objects left behind by regression Tom Lane <[email protected]>
2025-08-05 18:11 ` Re: Test to dump and restore objects left behind by regression Alvaro Herrera <[email protected]>
2025-08-06 01:22 ` Re: Test to dump and restore objects left behind by regression Michael Paquier <[email protected]>
2025-08-06 15:10 ` Re: Test to dump and restore objects left behind by regression Ashutosh Bapat <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox