public inbox for [email protected]  

help / color / mirror / Atom feed

From: Koen De Groote <[email protected]>
To: David Mullineux <[email protected]>
Cc: PostgreSQL General <[email protected]>
Subject: Re: Memory settings when running postgres in a docker container
Date: Fri, 22 Nov 2024 21:59:46 +0100
Message-ID: <CAGbX52FendzTdDgTrs_DqF39Oj6pedigmc4Pu7GXy7VUtPxGVw@mail.gmail.com> (raw)
In-Reply-To: <CAGsyd8Xf=o=0tL7ttqEWg7mPehyL2e=7Nc+OpoNcpSny-JVB9g@mail.gmail.com>
References: <CAGbX52Fm=k8hHJKEzo6-mnh7gn91s=Lz_t6B5uF1SotpXH3UeA@mail.gmail.com>
	<CAGsyd8Xf=o=0tL7ttqEWg7mPehyL2e=7Nc+OpoNcpSny-JVB9g@mail.gmail.com>

Ah, see, I didn't know that.

On Wed, Nov 20, 2024 at 11:10 PM David Mullineux <[email protected]> wrote:

> i dont get why you think all memroy will be used.
>  When you say
> shared_buffers = 16GB
> effective_cache_size = 48GB
>
> ...then this is using only 16GB for shared buffers.
>
> The effective _cache_size doesn't cause any memory to.be allocated. It's
> just a hint to optomizer ....
>
> On Wed, 20 Nov 2024, 11:16 Koen De Groote, <[email protected]> wrote:
>
>> Assuming a machine with:
>>
>> * 16 CPU cores
>> * 64GB RAM
>>
>> Set to 500 max connections
>>
>> A tool like this: https://pgtune.leopard.in.ua/
>>
>> Will output recommended settings:
>>
>> max_connections = 500
>> shared_buffers = 16GB
>> effective_cache_size = 48GB
>> maintenance_work_mem = 2GB
>> checkpoint_completion_target = 0.9
>> wal_buffers = 16MB
>> default_statistics_target = 100
>> random_page_cost = 1.1
>> effective_io_concurrency = 200
>> work_mem = 8388kB
>> huge_pages = try
>> min_wal_size = 1GB
>> max_wal_size = 4GB
>> max_worker_processes = 16
>> max_parallel_workers_per_gather = 4
>> max_parallel_workers = 16
>> max_parallel_maintenance_workers = 4
>>
>> And they basically use up all the memory of the machine.
>>
>> 16GB shared buffers, 48GB effective cache size, 8MB of work_mem for some
>> reason...
>>
>> This seems rather extreme. I feel there should be free memory for
>> emergencies and monitoring solutions.
>>
>> And then there's the fact that postgres on this machine will be run in a
>> docker container. Which, on Linux, receives 64MB of /dev/shm shared memory
>> by default, but can be increased.
>>
>> I feel like I should probably actually lower my upper limit for memory,
>> regardless of what the machine actually has, so I can have free memory, and
>> also not bring the container process itself into danger.
>>
>> Is it as straightforward as putting my limit on, say 20GB, and then
>> giving more /dev/shm to the container? Or is there more to consider?
>>
>> Regards,
>> Koen De Groote
>>
>>
>>
>>
>>
>>
>>

view thread (2+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Memory settings when running postgres in a docker container
  In-Reply-To: <CAGbX52FendzTdDgTrs_DqF39Oj6pedigmc4Pu7GXy7VUtPxGVw@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox