MIME-Version: 1.0
References: <ZmaXmWDL829fzAVX@ip-10-97-1-34.eu-west-3.compute.internal>
 <CAD21AoDOu=DZcC+PemYmCNGSwbgL1s-5OZkZ1Spd5pSxofWNCw@mail.gmail.com>
 <ZmgKL8zUm3qxbY9V@ip-10-97-1-34.eu-west-3.compute.internal>
 <ZmreasV0BKZ5BHIM@ip-10-97-1-34.eu-west-3.compute.internal>
 <ZnbIIUQpFJKAJx2W@ip-10-97-1-34.eu-west-3.compute.internal> <ZnlPZZZJCRu/8fka@ip-10-97-1-34.eu-west-3.compute.internal>
In-Reply-To: <ZnlPZZZJCRu/8fka@ip-10-97-1-34.eu-west-3.compute.internal>
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Thu, 27 Jun 2024 14:17:38 +0900
Message-ID: <CAD21AoBAXD05prx2VPgi989UOiZmhzFV+PUB=pa86zhObjTCxA@mail.gmail.com>
Subject: Re: Track the amount of time waiting due to cost_delay
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org, Robert Haas <robertmhaas@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAD21AoBAXD05prx2VPgi989UOiZmhzFV%2BPUB%3Dpa86zhObjTCxA%40mail.gmail.com>
Precedence: bulk

On Mon, Jun 24, 2024 at 7:50=E2=80=AFPM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
>
> Hi,
>
> On Sat, Jun 22, 2024 at 12:48:33PM +0000, Bertrand Drouvot wrote:
> > 1. vacuuming indexes time has been longer on master because with v2, th=
e leader
> > has been interrupted 342605 times while waiting, then making v2 "faster=
".
> >
> > 2. the leader being interrupted while waiting is also already happening=
 on master
> > due to the pgstat_progress_parallel_incr_param() calls in
> > parallel_vacuum_process_one_index() (that have been added in
> > 46ebdfe164). It has been the case "only" 36 times during my test case.
> >
> > I think that 2. is less of a concern but I think that 1. is something t=
hat needs
> > to be addressed because the leader process is not honouring its cost de=
lay wait
> > time in a noticeable way (at least during my test case).
> >
> > I did not think of a proposal yet, just sharing my investigation as to =
why
> > v2 has been faster than master during the vacuuming indexes phase.

Thank you for the benchmarking and analyzing the results! I agree with
your analysis and was surprised by the fact that the more times
workers go to sleep, the more times the leader wakes up.

>
> I think that a reasonable approach is to make the reporting from the para=
llel
> workers to the leader less aggressive (means occur less frequently).
>
> Please find attached v3, that:
>
> - ensures that there is at least 1 second between 2 reports, per parallel=
 worker,
> to the leader.
>
> - ensures that the reported delayed time is still correct (keep track of =
the
> delayed time between 2 reports).
>
> - does not add any extra pg_clock_gettime_ns() calls (as compare to v2).
>

Sounds good to me. I think it's better to keep the logic for
throttling the reporting the delay message simple. It's an important
consideration but executing parallel vacuum with delays would be less
likely to be used in practice.

Regards,

--=20
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com