public inbox for [email protected]  

help / color / mirror / Atom feed

From: James Pang <[email protected]>
To: Laurenz Albe <[email protected]>
Cc: Frits Hoogland <[email protected]>
Cc: [email protected]
Subject: Re: many sessions waiting DataFileRead and extend
Date: Thu, 26 Jun 2025 11:48:45 +0800
Message-ID: <CAHgTRfcZPWiw7zfBuFub59W5OURcZ2q67HsydTzLhU+54uowXQ@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <CAHgTRfdDkiX5bLRCYhrP18kFJjcvs1eHvL+jUtX1Ep-Y3=cFVg@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<CAHgTRfff0+i4PV_9G-fAREdDcMi4J1StcCDG6NcJ=hGhT2-SvQ@mail.gmail.com>
	<[email protected]>

    Thanks,  I make a summary of the issue, no connection storm(fork)
either, just suddenly many session waiting on "extend" and "DataFileRead",
it last 2 seconds, this server has 64 vcpu and running there long time
without issue, only last weekend, we patch from 14.8 to 14.14. We checked
with Infra and VMWARE, and OS monitoring, except very high %sys cpu, not
found other abnormal events. so, we are thinking if Postgresql buffer cache
or OS page cache level contention leading to that(if that can contribute to
IO wait, DataFileRead, extend).
attached please find details, just masking dbname, username,and ip
addresses.

Laurenz Albe <[email protected]> 於 2025年6月26日週四 上午11:03寫道：

> On Thu, 2025-06-26 at 10:32 +0800, James Pang wrote:
> > thans for you suggestions, we have iowait from sar command too, copy
> here, checking with infra team not found abnormal IO activities either.
> > 02:00:01 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal
>  %guest  %gnice   %idle
> > 02:00:03 PM  all   15.92    0.00   43.02    0.65    0.76    2.56    0.00
>    0.00    0.00   37.09
>
> Crazy high system time.
>
> If it is not transparent hugepages, the other suspect is fork().
> Turn on "log_connections" for a minute or two and see if there are lots
> of connections established.  That can easily use your CPU.
>
> Other than that, I am out of guesses.
>
> > Frits Hoogland <[email protected]> 於 2025年6月25日週三 下午10:27寫道：
> > > > On 25 Jun 2025, at 07:59, Laurenz Albe <[email protected]>
> wrote:
> > > >
> > > > Run "sar -P all 1" and see if "%iowait" is high.
> > >
> > > I would (strongly) advise against the use of iowait as an indicator.
>
> I have heard that before, and I am sure you are right.
>
> I grant that if it is low, it may just mean that the CPU is under load.
> But if %iowait is high, my experience is that that indicates an I/O
> problem.
>
> Yours,
> Laurenz Albe
>


Attachments:

  [application/octet-stream] many_sessions_waiting_extend_Datafileread_but_no_ioload_spike (8.9K, ../CAHgTRfcZPWiw7zfBuFub59W5OURcZ2q67HsydTzLhU+54uowXQ@mail.gmail.com/3-many_sessions_waiting_extend_Datafileread_but_no_ioload_spike)
  download

view thread (14+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected]
  Subject: Re: many sessions waiting DataFileRead and extend
  In-Reply-To: <CAHgTRfcZPWiw7zfBuFub59W5OURcZ2q67HsydTzLhU+54uowXQ@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox