public inbox for [email protected]
help / color / mirror / Atom feedFrom: Dmitry Dolgov <[email protected]>
To: Thomas Munro <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Automatically sizing the IO worker pool
Date: Tue, 14 Apr 2026 12:26:46 +0200
Message-ID: <kzmvf33qb2mt6qqwbd5xixafxje7iblubzd7vetk4bw5b2vfp6@34dtpx4jgldh> (raw)
In-Reply-To: <CA+hUKGLh5vzNfw85aumuMU5WdCnVCpB49LF60q7Vnu0g+2EYGg@mail.gmail.com>
References: <CA+hUKG+m4xV0LMoH2c=oRAdEXuCnh+tGBTWa7uFeFMGgTLAw+Q@mail.gmail.com>
<it7fexpclowjku57bsdh4uqr366wa2fxtq5ahzxczoxonmbh5s@g2f5oesiakzq>
<CA+hUKGJULUTkT2LpeHTSt3KHbJrYNBT-kj1-OhMRV_PnUQ_57A@mail.gmail.com>
<CA+q6zcW0qh2aPKg0z58mp-Ba8avp7MWkS00ADrOOv=CBzJJMLA@mail.gmail.com>
<CA+hUKGJLe-0E-wZ-is78CEHhjbC=ihMVCQLoN1dmD-j05s9qRg@mail.gmail.com>
<pdrbpbvhscwckb6nqkfgoo7o63nnmeq7taaazgfi3bk627gb6p@rzcauoqabb3k>
<CA+hUKGLh5vzNfw85aumuMU5WdCnVCpB49LF60q7Vnu0g+2EYGg@mail.gmail.com>
> On Sat, Apr 11, 2026 at 06:35:18PM +1200, Thomas Munro wrote:
> On Wed, Jul 30, 2025 at 10:15 PM Dmitry Dolgov <[email protected]> wrote:
> > As a side note, I was trying to experiment with this patch using
> > dm-mapper's delay feature to introduce an arbitrary large io latency and
> > see how the io queue is growing.
>
> FWIW, here's what I came up with while experimenting with that sort of thing:
>
> shared_preload_libraries=io_limit
> io_limit.ios_per_second=6000
>
> That differs from eg dm-mapper delays by making everything seem like
> slow direct I/O, which seemed more interesting for this project. For
> example if you run some continuous workload while you SET
> io_limit.ios_per_second to various numbers, with
> io_workers_idle_timeout set fairly low, you can monitor the pool
> adjustments.
Yeah, sounds like a good idea. Do you plan to introduce such an
extension long term for testing, or is it just one off?
As to me it looks worth keeping, maybe even use injections points to
allow for more flexibility. And I know I sound like a broken record, but
if I understand correctly the delays introduced via ios_per_seconds and
others are constant in time -- I've experimented a bit and found some
reference implementations in numpy for geometric distribution sampling,
which allow to make the delay a random variable. Since the geometric
distribution is a discrete analog of the exponential one, and the latter
represents delays between events in Poisson distribution, such random
variable would give an approximation for more real load.
view thread (24+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Automatically sizing the IO worker pool
In-Reply-To: <kzmvf33qb2mt6qqwbd5xixafxje7iblubzd7vetk4bw5b2vfp6@34dtpx4jgldh>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox