pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedNUMA settings
7+ messages / 3 participants
[nested] [flat]
* NUMA settings
@ 2020-04-29 06:54 Marc Rechté <marc4@rechte.fr>
0 siblings, 1 reply; 7+ messages in thread
From: Marc Rechté @ 2020-04-29 06:54 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org
Hello,
I am trying to figure out the recommended settings for a PG dedicated
machine regarding NUMA.
I assume that the shared buffers are using Huge Phages only. Please
correct if I am wrong:
1) postgres is started with numactl --interleave=all, in order to spread
memory pages evenly on nodes.
2) wm.swappiness is left to the default 60 value, because Huge Pages
never swap, and we wish the idle backend to be swapped out if necessary.
3) vm.zone_reclaim_mode = 0. I am not sure it is the right choice.
4) kernel.numa_balancing = 1. Only if it is confirmed that it will not
affect postgres, because started with the interleave policy.
Thanks
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: NUMA settings
@ 2020-04-29 08:50 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Marc Rechté <marc4@rechte.fr>
0 siblings, 1 reply; 7+ messages in thread
From: Laurenz Albe @ 2020-04-29 08:50 UTC (permalink / raw)
To: Marc Rechté <marc4@rechte.fr>; pgsql-performance@lists.postgresql.org
On Wed, 2020-04-29 at 08:54 +0200, Marc Rechté wrote:
> I am trying to figure out the recommended settings for a PG dedicated
> machine regarding NUMA.
>
> I assume that the shared buffers are using Huge Phages only. Please
> correct if I am wrong:
>
> 1) postgres is started with numactl --interleave=all, in order to spread
> memory pages evenly on nodes.
> 2) wm.swappiness is left to the default 60 value, because Huge Pages
> never swap, and we wish the idle backend to be swapped out if necessary.
> 3) vm.zone_reclaim_mode = 0. I am not sure it is the right choice.
> 4) kernel.numa_balancing = 1. Only if it is confirmed that it will not
> affect postgres, because started with the interleave policy.
I am not the top expert on this, but as far as I can tell:
- Disabling NUMA is good if you want to run a single database cluster
on the machine that should use all resources.
If you want to run several clusters that share the resources, leaving
NUMA support enabled might be the better thing to do.
- If you can, disable NUMA in the BIOS, on as low a level as possible.
- I think "kernel.numa_balancing" should be 0.
Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: NUMA settings
@ 2020-05-04 16:20 Andres Freund <andres@anarazel.de>
parent: Laurenz Albe <laurenz.albe@cybertec.at>
0 siblings, 1 reply; 7+ messages in thread
From: Andres Freund @ 2020-05-04 16:20 UTC (permalink / raw)
To: Laurenz Albe <laurenz.albe@cybertec.at>; +Cc: Marc Rechté <marc4@rechte.fr>; pgsql-performance@lists.postgresql.org
Hi,
On 2020-04-29 10:50:54 +0200, Laurenz Albe wrote:
> On Wed, 2020-04-29 at 08:54 +0200, Marc Rechté wrote:
> > I am trying to figure out the recommended settings for a PG dedicated
> > machine regarding NUMA.
> >
> > I assume that the shared buffers are using Huge Phages only. Please
> > correct if I am wrong:
> >
> > 1) postgres is started with numactl --interleave=all, in order to spread
> > memory pages evenly on nodes.
> > 2) wm.swappiness is left to the default 60 value, because Huge Pages
> > never swap, and we wish the idle backend to be swapped out if necessary.
> > 3) vm.zone_reclaim_mode = 0. I am not sure it is the right choice.
> > 4) kernel.numa_balancing = 1. Only if it is confirmed that it will not
> > affect postgres, because started with the interleave policy.
>
> I am not the top expert on this, but as far as I can tell:
>
> - Disabling NUMA is good if you want to run a single database cluster
> on the machine that should use all resources.
>
> If you want to run several clusters that share the resources, leaving
> NUMA support enabled might be the better thing to do.
>
> - If you can, disable NUMA in the BIOS, on as low a level as possible.
I am doubtful that that's generally going to be beneficial. I think the
strategy of starting postgres with interleave is probably a better
answer.
- Andres
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: NUMA settings
@ 2020-05-05 05:56 Marc Rechté <marc4@rechte.fr>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 7+ messages in thread
From: Marc Rechté @ 2020-05-05 05:56 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org
> Hi,
>
> On 2020-04-29 10:50:54 +0200, Laurenz Albe wrote:
>> On Wed, 2020-04-29 at 08:54 +0200, Marc Rechté wrote:
>>> I am trying to figure out the recommended settings for a PG dedicated
>>> machine regarding NUMA.
>>>
>>> I assume that the shared buffers are using Huge Phages only. Please
>>> correct if I am wrong:
>>>
>>> 1) postgres is started with numactl --interleave=all, in order to spread
>>> memory pages evenly on nodes.
>>> 2) wm.swappiness is left to the default 60 value, because Huge Pages
>>> never swap, and we wish the idle backend to be swapped out if necessary.
>>> 3) vm.zone_reclaim_mode = 0. I am not sure it is the right choice.
>>> 4) kernel.numa_balancing = 1. Only if it is confirmed that it will not
>>> affect postgres, because started with the interleave policy.
>>
>> I am not the top expert on this, but as far as I can tell:
>>
>> - Disabling NUMA is good if you want to run a single database cluster
>> on the machine that should use all resources.
>>
>> If you want to run several clusters that share the resources, leaving
>> NUMA support enabled might be the better thing to do.
>>
>> - If you can, disable NUMA in the BIOS, on as low a level as possible.
>
> I am doubtful that that's generally going to be beneficial. I think the
> strategy of starting postgres with interleave is probably a better
> answer.
>
> - Andres
>
>
Thanks for answers. Further readings make me think that we should *not*
start postgres with numactl --interleave=all: this may have counter
productive effect on backends anon memory (heap, stack). IMHO, what is
important is to use Huge Pages for shared buffers: they are allocated
(reserved) by the kernel at boot time and spread evenly on all nodes. On
top of that they never swap.
My (temp) conclusions are following:
vm.zone_reclaim_mode = 0
kernel.numa_balancing = 0 (still not sure with that choice)
wm.swappiness = 60 (default)
start postgres as usual (no numactl)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: NUMA settings
@ 2020-05-05 08:00 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Marc Rechté <marc4@rechte.fr>
0 siblings, 1 reply; 7+ messages in thread
From: Laurenz Albe @ 2020-05-05 08:00 UTC (permalink / raw)
To: Marc Rechté <marc4@rechte.fr>; pgsql-performance@lists.postgresql.org
On Tue, 2020-05-05 at 07:56 +0200, Marc Rechté wrote:
> Thanks for answers. Further readings make me think that we should *not*
> start postgres with numactl --interleave=all: this may have counter
> productive effect on backends anon memory (heap, stack). IMHO, what is
> important is to use Huge Pages for shared buffers: they are allocated
> (reserved) by the kernel at boot time and spread evenly on all nodes. On
> top of that they never swap.
>
> My (temp) conclusions are following:
> vm.zone_reclaim_mode = 0
> kernel.numa_balancing = 0 (still not sure with that choice)
> wm.swappiness = 60 (default)
> start postgres as usual (no numactl)
Thanks for sharing your insights.
I think that "vm.swappiness" should be 0.
PostgreSQL does its own memory management, any swapping by the kernel
would go against that.
Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: NUMA settings
@ 2020-05-05 08:11 Marc Rechté <marc4@rechte.fr>
parent: Laurenz Albe <laurenz.albe@cybertec.at>
0 siblings, 1 reply; 7+ messages in thread
From: Marc Rechté @ 2020-05-05 08:11 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org
> On Tue, 2020-05-05 at 07:56 +0200, Marc Rechté wrote:
>> Thanks for answers. Further readings make me think that we should *not*
>> start postgres with numactl --interleave=all: this may have counter
>> productive effect on backends anon memory (heap, stack). IMHO, what is
>> important is to use Huge Pages for shared buffers: they are allocated
>> (reserved) by the kernel at boot time and spread evenly on all nodes. On
>> top of that they never swap.
>>
>> My (temp) conclusions are following:
>> vm.zone_reclaim_mode = 0
>> kernel.numa_balancing = 0 (still not sure with that choice)
>> wm.swappiness = 60 (default)
>> start postgres as usual (no numactl)
>
> Thanks for sharing your insights.
>
> I think that "vm.swappiness" should be 0.
> PostgreSQL does its own memory management, any swapping by the kernel
> would go against that.
>
> Yours,
> Laurenz Albe
>
As said in the post, we wish the idle backends to be swapped out if
necessary. Therefore lowering swappiness would produce the opposite
effect: swapping out Linux file cache rather than backends memory.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: NUMA settings
@ 2020-05-05 08:25 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Marc Rechté <marc4@rechte.fr>
0 siblings, 0 replies; 7+ messages in thread
From: Laurenz Albe @ 2020-05-05 08:25 UTC (permalink / raw)
To: Marc Rechté <marc4@rechte.fr>; pgsql-performance@lists.postgresql.org
On Tue, 2020-05-05 at 10:11 +0200, Marc Rechté wrote:
> > I think that "vm.swappiness" should be 0.
> > PostgreSQL does its own memory management, any swapping by the kernel
> > would go against that.
> >
> > Yours,
> > Laurenz Albe
> >
> As said in the post, we wish the idle backends to be swapped out if
> necessary. Therefore lowering swappiness would produce the opposite
> effect: swapping out Linux file cache rather than backends memory.
I see. Sorry for not paying attention.
An idle backend consumes only a few MB of RAM, though.
Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2020-05-05 08:25 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-04-29 06:54 NUMA settings Marc Rechté <marc4@rechte.fr>
2020-04-29 08:50 ` Laurenz Albe <laurenz.albe@cybertec.at>
2020-05-04 16:20 ` Andres Freund <andres@anarazel.de>
2020-05-05 05:56 ` Marc Rechté <marc4@rechte.fr>
2020-05-05 08:00 ` Laurenz Albe <laurenz.albe@cybertec.at>
2020-05-05 08:11 ` Marc Rechté <marc4@rechte.fr>
2020-05-05 08:25 ` Laurenz Albe <laurenz.albe@cybertec.at>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox