pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Ho Fat Tsang <namiwf@gmail.com>
To: Richard Huxton <dev@archonet.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: PostgreSQL 8.0 occasionally slow down
Date: Wed, 4 Jul 2007 01:14:05 +0800
Message-ID: <97e6e20f0707031014y4146f7a7p4016df43210653ea@mail.gmail.com> (raw)
In-Reply-To: <4684CF12.5010003@archonet.com>
References: <97e6e20f0706272354m5e42c91em177c8b80abb2ea52@mail.gmail.com>
	<468361A9.1040308@archonet.com>
	<97e6e20f0706280101y543b871ao4cbf85e57f07f7e9@mail.gmail.com>
	<46836B66.7050109@archonet.com>
	<97e6e20f0706280117h20ab2e03n770abc588f17742@mail.gmail.com>
	<46837C50.20603@archonet.com>
	<97e6e20f0706281904v4b7c4a97p570deda85e590ee4@mail.gmail.com>
	<4684CF12.5010003@archonet.com>

2007/6/29, Richard Huxton <dev@archonet.com>:
>
> Ho Fat Tsang wrote:
> > Hi Richard,
> >
> >    I've tested again according your suggestion. I noticed that for each
> > time the pgsql slow down, there is a short period a process called
> > "pdflush"
> > eating up lot of I/O. I've goolgled and know it is a process for writing
> > dirty pages back to the disk by the Linux kernel. I will have further
> > investigation on this process with my limited knowledge on Linux kernel.
>
> Well, pdflush is responsible for flushing dirty pages to disk on behalf
> of all processes.
>
> If it's doing it every 3 minutes while checkpoints are happening every
> 30 seconds then I don't see how it's PG that's responsible.
>
> There are three possibilities:
> 1. PG isn't actually checkpointing every 30 seconds.
> 2. There is a burst of query activity every 3 minutes that causes a lot
> of writing.
> 3. Some other process is responsible.


Exactly ! you are right, finally i have found that the root cause for this
is the application that use PG. There is memory leak using MappedByteBuffer
(writing in java), it leads high I/O loading and finally reaches the ratio
that pdflush is being kicked start in the kernel.

Thank you for helping a lot in digging out this issue ! learn much for you
guys !

>   Correct me if i am wrong. It seems postgresql 8.0 does not bundle
> > auto-vacuum by default. So all vacuum and analyse are done manually ? So
> > what i have tested related to vaccuum is running auto-vacuum (a
> executeable
> > located in /bin) parallel under normal production load but it seems
> won't
> > help.
>
> Can't remember whether 8.0 had autovacuum bundled and turned off or not
> bundled at all. If it's not running it can't be causing this problem
> though.
>
> --
>    Richard Huxton
>    Archonet Ltd
>

view thread (18+ messages)  latest in thread

Message-ID: <97e6e20f0707031014y4146f7a7p4016df43210653ea@mail.gmail.com>
Permalink:  ../97e6e20f0707031014y4146f7a7p4016df43210653ea@mail.gmail.com/
Also on:    postgresql.org/message-id/97e6e20f0707031014y4146f7a7p4016df43210653ea@mail.gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: namiwf@gmail.com, dev@archonet.com
  Subject: Re: PostgreSQL 8.0 occasionally slow down
  In-Reply-To: <97e6e20f0707031014y4146f7a7p4016df43210653ea@mail.gmail.com>
  References: <97e6e20f0706272354m5e42c91em177c8b80abb2ea52@mail.gmail.com> <468361A9.1040308@archonet.com> <97e6e20f0706280101y543b871ao4cbf85e57f07f7e9@mail.gmail.com> <46836B66.7050109@archonet.com> <97e6e20f0706280117h20ab2e03n770abc588f17742@mail.gmail.com> <46837C50.20603@archonet.com> <97e6e20f0706281904v4b7c4a97p570deda85e590ee4@mail.gmail.com> <4684CF12.5010003@archonet.com> <97e6e20f0707031014y4146f7a7p4016df43210653ea@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox