pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedStats collector constant I/O
4+ messages / 4 participants
[nested] [flat]
* Stats collector constant I/O
@ 2014-05-15 04:18 Craig James <cjames@emolecules.com>
2014-05-15 08:18 ` Re: Stats collector constant I/O Jeff Janes <jeff.janes@gmail.com>
2014-05-15 08:25 ` Re: Stats collector constant I/O Pavel Stehule <pavel.stehule@gmail.com>
2014-05-15 23:27 ` Re: Stats collector constant I/O Tomas Vondra <tv@fuzzy.cz>
0 siblings, 3 replies; 4+ messages in thread
From: Craig James @ 2014-05-15 04:18 UTC (permalink / raw)
To: pgsql-performance
Day and night, the postgres stats collector process runs at about 20 MB/sec
output. vmstat shows this:
$ vmstat 2
procs -----------memory---------- ---swap-- -----io---- -system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy id
wa
0 0 55864 135740 123804 10712928 4 6 445 2642 0 0 5 1
92 2
1 0 55864 134820 123804 10713012 0 0 0 34880 540 338 1 1
98 0
0 0 55864 135820 123812 10712896 0 0 0 20980 545 422 1 1
98 0
iotop(1) shows that it's the stats collector, running at 20 MB/sec.
Is this normal?
Craig
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Stats collector constant I/O
2014-05-15 04:18 Stats collector constant I/O Craig James <cjames@emolecules.com>
@ 2014-05-15 08:18 ` Jeff Janes <jeff.janes@gmail.com>
2 siblings, 0 replies; 4+ messages in thread
From: Jeff Janes @ 2014-05-15 08:18 UTC (permalink / raw)
To: Craig James <cjames@emolecules.com>; +Cc: pgsql-performance
On May 14, 2014 9:19 PM, "Craig James" <cjames@emolecules.com> wrote:
>
> Day and night, the postgres stats collector process runs at about 20
MB/sec output. vmstat shows this:
>
> $ vmstat 2
> procs -----------memory---------- ---swap-- -----io---- -system--
----cpu----
> r b swpd free buff cache si so bi bo in cs us sy
id wa
> 0 0 55864 135740 123804 10712928 4 6 445 2642 0 0 5 1
92 2
> 1 0 55864 134820 123804 10713012 0 0 0 34880 540 338 1 1
98 0
> 0 0 55864 135820 123812 10712896 0 0 0 20980 545 422 1 1
98 0
>
> iotop(1) shows that it's the stats collector, running at 20 MB/sec.
This is normal for 9.2 and below if you have hundreds of databases in the
cluster.
Cheers,
Jeff
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Stats collector constant I/O
2014-05-15 04:18 Stats collector constant I/O Craig James <cjames@emolecules.com>
@ 2014-05-15 08:25 ` Pavel Stehule <pavel.stehule@gmail.com>
2 siblings, 0 replies; 4+ messages in thread
From: Pavel Stehule @ 2014-05-15 08:25 UTC (permalink / raw)
To: Craig James <cjames@emolecules.com>; +Cc: pgsql-performance
Hello
we had similar issue - you can try to move statfile to ramdisc
http://serverfault.com/questions/495057/too-much-i-o-generated-by-postgres-stats-collector-process
Regards
Pavel Stehule
2014-05-15 6:18 GMT+02:00 Craig James <cjames@emolecules.com>:
> Day and night, the postgres stats collector process runs at about 20
> MB/sec output. vmstat shows this:
>
> $ vmstat 2
> procs -----------memory---------- ---swap-- -----io---- -system--
> ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy id
> wa
> 0 0 55864 135740 123804 10712928 4 6 445 2642 0 0 5 1
> 92 2
> 1 0 55864 134820 123804 10713012 0 0 0 34880 540 338 1 1
> 98 0
> 0 0 55864 135820 123812 10712896 0 0 0 20980 545 422 1 1
> 98 0
>
> iotop(1) shows that it's the stats collector, running at 20 MB/sec.
>
> Is this normal?
>
> Craig
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Stats collector constant I/O
2014-05-15 04:18 Stats collector constant I/O Craig James <cjames@emolecules.com>
@ 2014-05-15 23:27 ` Tomas Vondra <tv@fuzzy.cz>
2 siblings, 0 replies; 4+ messages in thread
From: Tomas Vondra @ 2014-05-15 23:27 UTC (permalink / raw)
To: pgsql-performance
On 15.5.2014 06:18, Craig James wrote:
> Day and night, the postgres stats collector process runs at about 20
> MB/sec output. vmstat shows this:
>
> $ vmstat 2
> procs -----------memory---------- ---swap-- -----io---- -system--
> ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy
> id wa
> 0 0 55864 135740 123804 10712928 4 6 445 2642 0 0 5
> 1 92 2
> 1 0 55864 134820 123804 10713012 0 0 0 34880 540 338 1
> 1 98 0
> 0 0 55864 135820 123812 10712896 0 0 0 20980 545 422 1
> 1 98 0
>
> iotop(1) shows that it's the stats collector, running at 20 MB/sec.
Which PostgreSQL version are you running? And how many databases /
objects (tables, indexes) are there?
With versions up to 9.1, and large number of objects this is quite
normal. The file size is proportional to the number of objects, and may
get written quite frequently.
The newer versions (since 9.2) have per-database file, which usually
significantly decreases the load - both CPU and I/O. But if you have all
the objects are in a single database, this is not going to help.
I'd recommend moving the stat directory to tmpfs (i.e. memory-based
filesystem) - this improves the I/O load, but it may still consume
nontrivial amount of CPU time to read/write it.
Tomas
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2014-05-15 23:27 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-05-15 04:18 Stats collector constant I/O Craig James <cjames@emolecules.com>
2014-05-15 08:18 ` Jeff Janes <jeff.janes@gmail.com>
2014-05-15 08:25 ` Pavel Stehule <pavel.stehule@gmail.com>
2014-05-15 23:27 ` Tomas Vondra <tv@fuzzy.cz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox