public inbox for [email protected]
help / color / mirror / Atom feedRe: Memory settings when running postgres in a docker container
2+ messages / 2 participants
[nested] [flat]
* Re: Memory settings when running postgres in a docker container
@ 2024-11-20 22:10 David Mullineux <[email protected]>
2024-11-22 20:59 ` Re: Memory settings when running postgres in a docker container Koen De Groote <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: David Mullineux @ 2024-11-20 22:10 UTC (permalink / raw)
To: Koen De Groote <[email protected]>; +Cc: PostgreSQL General <[email protected]>
i dont get why you think all memroy will be used.
When you say
shared_buffers = 16GB
effective_cache_size = 48GB
...then this is using only 16GB for shared buffers.
The effective _cache_size doesn't cause any memory to.be allocated. It's
just a hint to optomizer ....
On Wed, 20 Nov 2024, 11:16 Koen De Groote, <[email protected]> wrote:
> Assuming a machine with:
>
> * 16 CPU cores
> * 64GB RAM
>
> Set to 500 max connections
>
> A tool like this: https://pgtune.leopard.in.ua/
>
> Will output recommended settings:
>
> max_connections = 500
> shared_buffers = 16GB
> effective_cache_size = 48GB
> maintenance_work_mem = 2GB
> checkpoint_completion_target = 0.9
> wal_buffers = 16MB
> default_statistics_target = 100
> random_page_cost = 1.1
> effective_io_concurrency = 200
> work_mem = 8388kB
> huge_pages = try
> min_wal_size = 1GB
> max_wal_size = 4GB
> max_worker_processes = 16
> max_parallel_workers_per_gather = 4
> max_parallel_workers = 16
> max_parallel_maintenance_workers = 4
>
> And they basically use up all the memory of the machine.
>
> 16GB shared buffers, 48GB effective cache size, 8MB of work_mem for some
> reason...
>
> This seems rather extreme. I feel there should be free memory for
> emergencies and monitoring solutions.
>
> And then there's the fact that postgres on this machine will be run in a
> docker container. Which, on Linux, receives 64MB of /dev/shm shared memory
> by default, but can be increased.
>
> I feel like I should probably actually lower my upper limit for memory,
> regardless of what the machine actually has, so I can have free memory, and
> also not bring the container process itself into danger.
>
> Is it as straightforward as putting my limit on, say 20GB, and then giving
> more /dev/shm to the container? Or is there more to consider?
>
> Regards,
> Koen De Groote
>
>
>
>
>
>
>
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Memory settings when running postgres in a docker container
2024-11-20 22:10 Re: Memory settings when running postgres in a docker container David Mullineux <[email protected]>
@ 2024-11-22 20:59 ` Koen De Groote <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Koen De Groote @ 2024-11-22 20:59 UTC (permalink / raw)
To: David Mullineux <[email protected]>; +Cc: PostgreSQL General <[email protected]>
Ah, see, I didn't know that.
On Wed, Nov 20, 2024 at 11:10 PM David Mullineux <[email protected]> wrote:
> i dont get why you think all memroy will be used.
> When you say
> shared_buffers = 16GB
> effective_cache_size = 48GB
>
> ...then this is using only 16GB for shared buffers.
>
> The effective _cache_size doesn't cause any memory to.be allocated. It's
> just a hint to optomizer ....
>
> On Wed, 20 Nov 2024, 11:16 Koen De Groote, <[email protected]> wrote:
>
>> Assuming a machine with:
>>
>> * 16 CPU cores
>> * 64GB RAM
>>
>> Set to 500 max connections
>>
>> A tool like this: https://pgtune.leopard.in.ua/
>>
>> Will output recommended settings:
>>
>> max_connections = 500
>> shared_buffers = 16GB
>> effective_cache_size = 48GB
>> maintenance_work_mem = 2GB
>> checkpoint_completion_target = 0.9
>> wal_buffers = 16MB
>> default_statistics_target = 100
>> random_page_cost = 1.1
>> effective_io_concurrency = 200
>> work_mem = 8388kB
>> huge_pages = try
>> min_wal_size = 1GB
>> max_wal_size = 4GB
>> max_worker_processes = 16
>> max_parallel_workers_per_gather = 4
>> max_parallel_workers = 16
>> max_parallel_maintenance_workers = 4
>>
>> And they basically use up all the memory of the machine.
>>
>> 16GB shared buffers, 48GB effective cache size, 8MB of work_mem for some
>> reason...
>>
>> This seems rather extreme. I feel there should be free memory for
>> emergencies and monitoring solutions.
>>
>> And then there's the fact that postgres on this machine will be run in a
>> docker container. Which, on Linux, receives 64MB of /dev/shm shared memory
>> by default, but can be increased.
>>
>> I feel like I should probably actually lower my upper limit for memory,
>> regardless of what the machine actually has, so I can have free memory, and
>> also not bring the container process itself into danger.
>>
>> Is it as straightforward as putting my limit on, say 20GB, and then
>> giving more /dev/shm to the container? Or is there more to consider?
>>
>> Regards,
>> Koen De Groote
>>
>>
>>
>>
>>
>>
>>
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-11-22 20:59 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-11-20 22:10 Re: Memory settings when running postgres in a docker container David Mullineux <[email protected]>
2024-11-22 20:59 ` Koen De Groote <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox