pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Linux I/O tuning:  CFQ vs. deadline
20+ messages / 10 participants
[nested] [flat]

* Linux I/O tuning:  CFQ vs. deadline
@ 2010-02-08 09:45 Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  0 siblings, 1 reply; 20+ messages in thread

From: Greg Smith @ 2010-02-08 09:45 UTC (permalink / raw)
  To: pgsql-performance

Recently I've made a number of unsubstantiated claims that the deadline 
scheduler on Linux does bad things compared to CFQ when running 
real-world mixed I/O database tests.  Unfortunately every time I do one 
of these I end up unable to release the results due to client 
confidentiality issues.  However, I do keep an eye out for people who 
run into the same issues in public benchmarks, and I just found one:  
http://insights.oetiker.ch/linux/fsopbench/

The problem analyzed in the "Deadline considered harmful" section looks 
exactly like what I run into:  deadline just does some bad things when 
the I/O workload gets complicated.  And the conclusion reached there, 
"the deadline scheduler did not have advantages in any of our test 
cases", has been my conclusion for every round of pgbench-based testing 
I've done too.  In that case, the specific issue is that reads get 
blocked badly when checkpoint writes are doing heavier work; you can see 
the read I/O numbers reported by "vmstat 1" go completely to zero for a 
second or more when it happens.  That can happen with CFQ, too, but it 
consistently seems more likely to occur with deadline.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-08 14:57 ` Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  0 siblings, 1 reply; 20+ messages in thread

From: Albe Laurenz @ 2010-02-08 14:57 UTC (permalink / raw)
  To: Greg Smith *EXTERN* <greg@2ndquadrant.com>; pgsql-performance

Greg Smith wrote:
> Recently I've made a number of unsubstantiated claims that the deadline 
> scheduler on Linux does bad things compared to CFQ when running 
> real-world mixed I/O database tests.  Unfortunately every time I do one 
> of these I end up unable to release the results due to client 
> confidentiality issues.  However, I do keep an eye out for people who 
> run into the same issues in public benchmarks, and I just found one:  
> http://insights.oetiker.ch/linux/fsopbench/

That is interesting; particularly since I have made one quite different
experience in which deadline outperformed CFQ by a factor of approximately 4.

So I tried to look for differences, and I found two possible places:
- My test case was read-only, our production system is read-mostly.
- We did not have a RAID array, but a SAN box (with RAID inside).

The "noop" scheduler performed about as well as "deadline".
I wonder if the two differences above could explain the different
result.

Yours,
Laurenz Albe



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
@ 2010-02-08 15:24   ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  0 siblings, 1 reply; 20+ messages in thread

From: Kevin Grittner @ 2010-02-08 15:24 UTC (permalink / raw)
  To: Greg Smith *EXTERN* <greg@2ndquadrant.com>; pgsql-performance; Albe Laurenz <laurenz.albe@wien.gv.at>

"Albe Laurenz" <laurenz.albe@wien.gv.at> wrote:
> Greg Smith wrote:
 
>> http://insights.oetiker.ch/linux/fsopbench/
> 
> That is interesting; particularly since I have made one quite
> different experience in which deadline outperformed CFQ by a
> factor of approximately 4.
 
I haven't benchmarked it per se, but when we started using
PostgreSQL on Linux, the benchmarks and posts I could find
recommended deadline=elevator, so we went with that, and when the
setting was missed on a machine it was generally found fairly
quickly because people complained that the machine wasn't performing
to expectations; changing this to deadline corrected the problem.
 
> So I tried to look for differences, and I found two possible
> places:
> - My test case was read-only, our production system is
>   read-mostly.
 
Yeah, our reads are typically several times our writes -- up to
maybe 10 to 1.
 
> - We did not have a RAID array, but a SAN box (with RAID inside).
 
No SAN here, but if I recall correctly, this was mostly an issue on
our larger arrays -- RAID 5 with dozens of spindles on a BBU
hardware controller.
 
Other differences between our environment and that of the benchmarks
cited above:
 
 - We use SuSE Linux Enterprise Server, so we've been on *much*
   earlier kernel versions that this benchmark.
 
 - We've been using xfs, with noatime,nobarrier.
 
I'll keep this in mind as something to try if we have problem
performance in line with what that page describes, though....
 
-Kevin



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
@ 2010-02-08 15:30     ` Greg Smith <greg@2ndquadrant.com>
  2010-02-08 17:49       ` Re: Linux I/O tuning:  CFQ vs. deadline Josh Berkus <josh@agliodbs.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  0 siblings, 2 replies; 20+ messages in thread

From: Greg Smith @ 2010-02-08 15:30 UTC (permalink / raw)
  To: Kevin Grittner <Kevin.Grittner@wicourts.gov>; +Cc: pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

Kevin Grittner wrote:
> I'll keep this in mind as something to try if we have problem
> performance in line with what that page describes, though....
>   

That's basically what I've been trying to make clear all along:  people 
should keep an open mind, watch what happens, and not make any 
assumptions.  There's no clear cut preference for one scheduler or the 
other in all situations.  I've seen CFQ do much better, you and Albe 
report situations where the opposite is true.  I was just happy to see 
another report of someone running into the same sort of issue I've been 
seeing, because I didn't have very much data to offer about why the 
standard advice of "always use deadline for a database app" might not 
apply to everyone.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-08 17:49       ` Josh Berkus <josh@agliodbs.com>
  2010-02-08 18:59         ` Re: Linux I/O tuning: CFQ vs. deadline Scott Marlowe <scott.marlowe@gmail.com>
  2010-02-09 01:14         ` Re: Linux I/O tuning: CFQ vs. deadline Mark Wong <markwkm@gmail.com>
  2010-02-09 03:50         ` Re: Linux I/O tuning:  CFQ vs. deadline Scott Carey <scott@richrelevance.com>
  2010-02-09 04:14         ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-16 21:34         ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff Davis <pgsql@j-davis.com>
  1 sibling, 5 replies; 20+ messages in thread

From: Josh Berkus @ 2010-02-08 17:49 UTC (permalink / raw)
  To: Greg Smith <greg@2ndquadrant.com>; +Cc: Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>


> That's basically what I've been trying to make clear all along:  people
> should keep an open mind, watch what happens, and not make any
> assumptions.  There's no clear cut preference for one scheduler or the
> other in all situations.  I've seen CFQ do much better, you and Albe
> report situations where the opposite is true.  I was just happy to see
> another report of someone running into the same sort of issue I've been
> seeing, because I didn't have very much data to offer about why the
> standard advice of "always use deadline for a database app" might not
> apply to everyone.

Damn, you would have to make things complicated, eh?

FWIW, back when deadline was first introduced Mark Wong did some tests
and found Deadline to be the fastest of 4 on DBT2 ... but only by about
5%.  If the read vs. checkpoint analysis is correct, what was happening
is the penalty for checkpoints on deadline was almost wiping out the
advantage for reads, but not quite.

Those tests were also done on attached storage.

So, what this suggests is:
reads:  deadline > CFQ
writes: CFQ > deadline
attached storage:  deadline > CFQ

Man, we'd need a lot of testing to settle this.  I guess that's why
Linux gives us the choice of 4 ...

--Josh Berkus



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning: CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 17:49       ` Re: Linux I/O tuning:  CFQ vs. deadline Josh Berkus <josh@agliodbs.com>
@ 2010-02-08 18:59         ` Scott Marlowe <scott.marlowe@gmail.com>
  4 siblings, 0 replies; 20+ messages in thread

From: Scott Marlowe @ 2010-02-08 18:59 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: Greg Smith <greg@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

On Mon, Feb 8, 2010 at 10:49 AM, Josh Berkus <josh@agliodbs.com> wrote:
>
>> That's basically what I've been trying to make clear all along:  people
>> should keep an open mind, watch what happens, and not make any
>> assumptions.  There's no clear cut preference for one scheduler or the
>> other in all situations.  I've seen CFQ do much better, you and Albe
>> report situations where the opposite is true.  I was just happy to see
>> another report of someone running into the same sort of issue I've been
>> seeing, because I didn't have very much data to offer about why the
>> standard advice of "always use deadline for a database app" might not
>> apply to everyone.
>
> Damn, you would have to make things complicated, eh?
>
> FWIW, back when deadline was first introduced Mark Wong did some tests
> and found Deadline to be the fastest of 4 on DBT2 ... but only by about
> 5%.  If the read vs. checkpoint analysis is correct, what was happening
> is the penalty for checkpoints on deadline was almost wiping out the
> advantage for reads, but not quite.
>
> Those tests were also done on attached storage.
>
> So, what this suggests is:
> reads:  deadline > CFQ
> writes: CFQ > deadline
> attached storage:  deadline > CFQ
>
> Man, we'd need a lot of testing to settle this.  I guess that's why
> Linux gives us the choice of 4 ...

Just to add to the data points.  On an 8 core opteron Areca 1680 and a
12 disk RAID-10 for data and 2 disk RAID-1 for WAL, I get noticeably
better performance (approximately 15%) and lower load factors (they
drop from about 8 to 5 or 6) running noop over the default scheduler,
with RHEL 5.4 with the 2.6.18-92.el5 kernel from RHEL 5.2.



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning: CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 17:49       ` Re: Linux I/O tuning:  CFQ vs. deadline Josh Berkus <josh@agliodbs.com>
@ 2010-02-09 01:14         ` Mark Wong <markwkm@gmail.com>
  4 siblings, 0 replies; 20+ messages in thread

From: Mark Wong @ 2010-02-09 01:14 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: Greg Smith <greg@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

On Mon, Feb 8, 2010 at 9:49 AM, Josh Berkus <josh@agliodbs.com> wrote:
>
>> That's basically what I've been trying to make clear all along:  people
>> should keep an open mind, watch what happens, and not make any
>> assumptions.  There's no clear cut preference for one scheduler or the
>> other in all situations.  I've seen CFQ do much better, you and Albe
>> report situations where the opposite is true.  I was just happy to see
>> another report of someone running into the same sort of issue I've been
>> seeing, because I didn't have very much data to offer about why the
>> standard advice of "always use deadline for a database app" might not
>> apply to everyone.
>
> Damn, you would have to make things complicated, eh?
>
> FWIW, back when deadline was first introduced Mark Wong did some tests
> and found Deadline to be the fastest of 4 on DBT2 ... but only by about
> 5%.  If the read vs. checkpoint analysis is correct, what was happening
> is the penalty for checkpoints on deadline was almost wiping out the
> advantage for reads, but not quite.
>
> Those tests were also done on attached storage.
>
> So, what this suggests is:
> reads:  deadline > CFQ
> writes: CFQ > deadline
> attached storage:  deadline > CFQ
>
> Man, we'd need a lot of testing to settle this.  I guess that's why
> Linux gives us the choice of 4 ...

I wonder what the impact is from the underlying RAID configuration.
Those DBT2 tests were also LVM striped volumes on top of single RAID0
LUNS (no jbod option).

Regards.
Mark



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 17:49       ` Re: Linux I/O tuning:  CFQ vs. deadline Josh Berkus <josh@agliodbs.com>
@ 2010-02-09 03:50         ` Scott Carey <scott@richrelevance.com>
  4 siblings, 0 replies; 20+ messages in thread

From: Scott Carey @ 2010-02-09 03:50 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: Greg Smith <greg@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance; Albe Laurenz <laurenz.albe@wien.gv.at>


On Feb 8, 2010, at 9:49 AM, Josh Berkus wrote:

> 
> Those tests were also done on attached storage.
> 
> So, what this suggests is:
> reads:  deadline > CFQ
> writes: CFQ > deadline
> attached storage:  deadline > CFQ
> 



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 17:49       ` Re: Linux I/O tuning:  CFQ vs. deadline Josh Berkus <josh@agliodbs.com>
@ 2010-02-09 04:14         ` Greg Smith <greg@2ndquadrant.com>
  4 siblings, 0 replies; 20+ messages in thread

From: Greg Smith @ 2010-02-09 04:14 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

Josh Berkus wrote:
> FWIW, back when deadline was first introduced Mark Wong did some tests
> and found Deadline to be the fastest of 4 on DBT2 ... but only by about
> 5%.  If the read vs. checkpoint analysis is correct, what was happening
> is the penalty for checkpoints on deadline was almost wiping out the
> advantage for reads, but not quite.
>   

Wasn't that before 8.3, where the whole checkpoint spreading logic 
showed up?  That's really a whole different write pattern now than it 
was then.  8.2 checkpoint writes were one big batch write amenable to 
optimizing for throughput.  The new ones are not; the I/O is intermixed 
with reads most of the time.

> Man, we'd need a lot of testing to settle this.  I guess that's why
> Linux gives us the choice of 4 ...
>   

A recent on of these I worked on started with 4096 possible I/O 
configurations we pruned down the most likely good candidates from.  I'm 
almost ready to schedule a week on Mark's HP performance test system in 
the lab now, to try and nail this down in a fully public environment for 
once.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 17:49       ` Re: Linux I/O tuning:  CFQ vs. deadline Josh Berkus <josh@agliodbs.com>
@ 2010-02-16 21:34         ` Jeff Davis <pgsql@j-davis.com>
  4 siblings, 0 replies; 20+ messages in thread

From: Jeff Davis @ 2010-02-16 21:34 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: Greg Smith <greg@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

On Mon, 2010-02-08 at 09:49 -0800, Josh Berkus wrote:
> FWIW, back when deadline was first introduced Mark Wong did some tests
> and found Deadline to be the fastest of 4 on DBT2 ... but only by about
> 5%.  If the read vs. checkpoint analysis is correct, what was happening
> is the penalty for checkpoints on deadline was almost wiping out the
> advantage for reads, but not quite.

I also did some tests when I was putting together my Synchronized Scan
benchmarks:

http://j-davis.com/postgresql/83v82_scans.html

CFQ was so slow that I didn't include it in the results at all.

The tests weren't intended to compare schedulers, so I did most of the
tests with anticipatory (at least the ones on linux; I also tested
freebsd). However, I have some raw data from the tests I did run with
CFQ:

http://j-davis.com/postgresql/results/

They will take some interpretation (again, not intended as scheduler
benchmarks). The server was modified to record a log message every N
page accesses.

Regards,
	Jeff Davis




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-09 04:16       ` Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  1 sibling, 1 reply; 20+ messages in thread

From: Greg Smith @ 2010-02-09 04:16 UTC (permalink / raw)
  To: Hannu Krosing <hannu@2ndquadrant.com>; +Cc: Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

Hannu Krosing wrote:
> Have you kept trace of what filesystems are in use ?
>   

Almost everything I do on Linux has been with ext3.  I had a previous 
diversion into VxFS and an upcoming one into XFS that may shed more 
light on all this.

And, yes, the whole I/O scheduling approach in Linux was just completely 
redesigned for a very recent kernel update.  So even what we think we 
know is already obsolete in some respects.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-09 04:35         ` david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  0 siblings, 1 reply; 20+ messages in thread

From: david@lang.hm @ 2010-02-09 04:35 UTC (permalink / raw)
  To: Greg Smith <greg@2ndquadrant.com>; +Cc: Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

On Mon, 8 Feb 2010, Greg Smith wrote:

> Hannu Krosing wrote:
>> Have you kept trace of what filesystems are in use ?
>> 
>
> Almost everything I do on Linux has been with ext3.  I had a previous 
> diversion into VxFS and an upcoming one into XFS that may shed more light on 
> all this.

it would be nice if you could try ext4 when doing your tests.

It's new enough that I won't trust it for production data yet, but a lot 
of people are jumping on it as if it was just a minor update to ext3 
instead of an almost entirely new filesystem.

David Lang

> And, yes, the whole I/O scheduling approach in Linux was just completely 
> redesigned for a very recent kernel update.  So even what we think we know is 
> already obsolete in some respects.
>
>



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
@ 2010-02-09 19:14           ` Jeff <threshar@torgo.978.org>
  2010-02-10 06:37             ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  0 siblings, 1 reply; 20+ messages in thread

From: Jeff @ 2010-02-09 19:14 UTC (permalink / raw)
  To: david@lang.hm; +Cc: Greg Smith <greg@2ndquadrant.com>; Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>


On Feb 8, 2010, at 11:35 PM, david@lang.hm wrote:
>
>> And, yes, the whole I/O scheduling approach in Linux was just  
>> completely redesigned for a very recent kernel update.  So even  
>> what we think we know is already obsolete in some respects.
>>

I'd done some testing a while ago on the schedulers and at the time  
deadline or noop smashed cfq.  Now, it is 100% possible since then  
that they've made vast improvements to cfq and or the VM to get better  
or similar performance.  I recall a vintage of 2.6 where they severely  
messed up the VM. Glad I didn't upgrade to that one :)

Here's the old post: http://archives.postgresql.org/pgsql-performance/2008-04/msg00155.php


--
Jeff Trout <jeff@jefftrout.com>
http://www.stuarthamm.net/
http://www.dellsmartexitin.com/






^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
@ 2010-02-10 06:37             ` Greg Smith <greg@2ndquadrant.com>
  2010-02-10 07:11               ` Re: Linux I/O tuning: CFQ vs. deadline Scott Marlowe <scott.marlowe@gmail.com>
  2010-02-10 17:22               ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  2010-02-10 18:04               ` Re: Linux I/O tuning:  CFQ vs. deadline Scott Carey <scott@richrelevance.com>
  0 siblings, 3 replies; 20+ messages in thread

From: Greg Smith @ 2010-02-10 06:37 UTC (permalink / raw)
  To: Jeff <threshar@torgo.978.org>; +Cc: david@lang.hm, Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

Jeff wrote:
> I'd done some testing a while ago on the schedulers and at the time 
> deadline or noop smashed cfq.  Now, it is 100% possible since then 
> that they've made vast improvements to cfq and or the VM to get better 
> or similar performance.  I recall a vintage of 2.6 where they severely 
> messed up the VM. Glad I didn't upgrade to that one :)
>
> Here's the old post: 
> http://archives.postgresql.org/pgsql-performance/2008-04/msg00155.php

pgiosim doesn't really mix writes into there though, does it?  The mixed 
read/write situations are the ones where the scheduler stuff gets messy.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning: CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  2010-02-10 06:37             ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-10 07:11               ` Scott Marlowe <scott.marlowe@gmail.com>
  2010-02-11 01:46                 ` Re: Linux I/O tuning: CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2 siblings, 1 reply; 20+ messages in thread

From: Scott Marlowe @ 2010-02-10 07:11 UTC (permalink / raw)
  To: Greg Smith <greg@2ndquadrant.com>; +Cc: Jeff <threshar@torgo.978.org>; david@lang.hm, Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

On Tue, Feb 9, 2010 at 11:37 PM, Greg Smith <greg@2ndquadrant.com> wrote:
> Jeff wrote:
>>
>> I'd done some testing a while ago on the schedulers and at the time
>> deadline or noop smashed cfq.  Now, it is 100% possible since then that
>> they've made vast improvements to cfq and or the VM to get better or similar
>> performance.  I recall a vintage of 2.6 where they severely messed up the
>> VM. Glad I didn't upgrade to that one :)
>>
>> Here's the old post:
>> http://archives.postgresql.org/pgsql-performance/2008-04/msg00155.php
>
> pgiosim doesn't really mix writes into there though, does it?  The mixed
> read/write situations are the ones where the scheduler stuff gets messy.

I agree. I think the only way to really test it is by testing it
against the system it's got to run under.  I'd love to see someone do
a comparison of early to mid 2.6 kernels (2.6.18 like RHEL5) to very
up to date 2.6 kernels.  On fast hardware.  What it does on a laptop
isn't that interesting and I don't have a big machine idle to test it
on.



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning: CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  2010-02-10 06:37             ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-10 07:11               ` Re: Linux I/O tuning: CFQ vs. deadline Scott Marlowe <scott.marlowe@gmail.com>
@ 2010-02-11 01:46                 ` Greg Smith <greg@2ndquadrant.com>
  2010-02-11 01:52                   ` Re: Linux I/O tuning: CFQ vs. deadline david@lang.hm
  0 siblings, 1 reply; 20+ messages in thread

From: Greg Smith @ 2010-02-11 01:46 UTC (permalink / raw)
  To: Scott Marlowe <scott.marlowe@gmail.com>; +Cc: Jeff <threshar@torgo.978.org>; david@lang.hm, Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

Scott Marlowe wrote:
> I'd love to see someone do a comparison of early to mid 2.6 kernels (2.6.18 like RHEL5) to very
> up to date 2.6 kernels.  On fast hardware.

I'd be happy just to find fast hardware that works on every kernel from 
the RHEL5 2.6.18 up to the latest one without issues.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning: CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  2010-02-10 06:37             ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-10 07:11               ` Re: Linux I/O tuning: CFQ vs. deadline Scott Marlowe <scott.marlowe@gmail.com>
  2010-02-11 01:46                 ` Re: Linux I/O tuning: CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-11 01:52                   ` david@lang.hm
  2010-02-11 02:09                     ` Re: Linux I/O tuning: CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  0 siblings, 1 reply; 20+ messages in thread

From: david@lang.hm @ 2010-02-11 01:52 UTC (permalink / raw)
  To: Greg Smith <greg@2ndquadrant.com>; +Cc: Scott Marlowe <scott.marlowe@gmail.com>; Jeff <threshar@threshar.is-a-geek.com>; Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

On Wed, 10 Feb 2010, Greg Smith wrote:

> Scott Marlowe wrote:
>> I'd love to see someone do a comparison of early to mid 2.6 kernels (2.6.18 
>> like RHEL5) to very
>> up to date 2.6 kernels.  On fast hardware.
>
> I'd be happy just to find fast hardware that works on every kernel from the 
> RHEL5 2.6.18 up to the latest one without issues.

it depends on your definition of 'fast hardware'

I have boxes that were very fast at the time that work on all these 
kernels, but they wouldn't be considered fast by todays's standards.

remember that there is a point release about every 3 months, 2.6.33 is 
about to be released, so this is a 3 x (33-18) = ~45 month old kernel.

hardware progresses a LOT on 4 years.

most of my new hardware has no problems with the old kernels as well, but 
once in a while I run into something that doesn't work.

David Lang



^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning: CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  2010-02-10 06:37             ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-10 07:11               ` Re: Linux I/O tuning: CFQ vs. deadline Scott Marlowe <scott.marlowe@gmail.com>
  2010-02-11 01:46                 ` Re: Linux I/O tuning: CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-11 01:52                   ` Re: Linux I/O tuning: CFQ vs. deadline david@lang.hm
@ 2010-02-11 02:09                     ` Greg Smith <greg@2ndquadrant.com>
  0 siblings, 0 replies; 20+ messages in thread

From: Greg Smith @ 2010-02-11 02:09 UTC (permalink / raw)
  To: david@lang.hm; +Cc: Scott Marlowe <scott.marlowe@gmail.com>; Jeff <threshar@threshar.is-a-geek.com>; Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>

david@lang.hm wrote:
> most of my new hardware has no problems with the old kernels as well, 
> but once in a while I run into something that doesn't work.

Quick survey just of what's within 20 feet of me:
-Primary desktop:  2 years old, requires 2.6.23 or later for SATA to work
-Server:  3 years old, requires 2.6.22 or later for the Areca card not 
to panic under load
-Laptops:  both about 2 years old, and require 2.6.28 to work at all; 
mostly wireless issues, but some power management ones that impact the 
processor working right too, occasional SATA ones too.

I'm looking into a new primary desktop to step up to 8 HT cores; I fully 
expect it won't boot anything older than 2.6.28 and may take an even 
newer kernel just for basic processor and disks parts to work.

We're kind of at a worst-case point right now for this sort of thing, on 
the tail side of the almost 3 year old RHEL5 using a 3.5 year old kernel 
as the standard for so many Linux server deployments.  Until RHEL6 is 
ready to go, there's little motivation for the people who make server 
hardware to get all their drivers perfect in the newer kernels.  Just 
after that ships will probably be a good time to do that sort of 
comparison, like it was possible to easily compare RHEL4 using 2.6.9 and 
RHEL5 with 2.6.18 easily in mid to late 2007 with many bits of 
high-performance hardware known to work well on each.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com




^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  2010-02-10 06:37             ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-10 17:22               ` Jeff <threshar@torgo.978.org>
  2 siblings, 0 replies; 20+ messages in thread

From: Jeff @ 2010-02-10 17:22 UTC (permalink / raw)
  To: Greg Smith <greg@2ndquadrant.com>; +Cc: david@lang.hm, Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance@postgresql.org, Albe Laurenz <laurenz.albe@wien.gv.at>


On Feb 10, 2010, at 1:37 AM, Greg Smith wrote:

> Jeff wrote:
>> I'd done some testing a while ago on the schedulers and at the time  
>> deadline or noop smashed cfq.  Now, it is 100% possible since then  
>> that they've made vast improvements to cfq and or the VM to get  
>> better or similar performance.  I recall a vintage of 2.6 where  
>> they severely messed up the VM. Glad I didn't upgrade to that one :)
>>
>> Here's the old post: http://archives.postgresql.org/pgsql-performance/2008-04/msg00155.php
>
> pgiosim doesn't really mix writes into there though, does it?  The  
> mixed read/write situations are the ones where the scheduler stuff  
> gets messy.
>

It has the abillity to rewrite blocks randomly as well - but I  
honestly don't remember if I did that during my cfq/deadline test.   
I'd wager I didn't.  Maybe I'll get some time to run some more tests  
on it in the next couple days

> -- 
> Greg Smith    2ndQuadrant   Baltimore, MD
> PostgreSQL Training, Services and Support
> greg@2ndQuadrant.com  www.2ndQuadrant.com
>

--
Jeff Trout <jeff@jefftrout.com>
http://www.stuarthamm.net/
http://www.dellsmartexitin.com/






^ permalink  raw  reply  [nested|flat] 20+ messages in thread

* Re: Linux I/O tuning:  CFQ vs. deadline
  2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-08 14:57 ` Re: Linux I/O tuning:  CFQ vs. deadline Albe Laurenz <laurenz.albe@wien.gv.at>
  2010-02-08 15:24   ` Re: Linux I/O tuning:  CFQ vs. deadline Kevin Grittner <Kevin.Grittner@wicourts.gov>
  2010-02-08 15:30     ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:16       ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
  2010-02-09 04:35         ` Re: Linux I/O tuning:  CFQ vs. deadline david@lang.hm
  2010-02-09 19:14           ` Re: Linux I/O tuning:  CFQ vs. deadline Jeff <threshar@torgo.978.org>
  2010-02-10 06:37             ` Re: Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
@ 2010-02-10 18:04               ` Scott Carey <scott@richrelevance.com>
  2 siblings, 0 replies; 20+ messages in thread

From: Scott Carey @ 2010-02-10 18:04 UTC (permalink / raw)
  To: Greg Smith <greg@2ndquadrant.com>; +Cc: Jeff <threshar@torgo.978.org>; david@lang.hm <david@lang.hm>; Hannu Krosing <hannu@2ndquadrant.com>; Kevin Grittner <Kevin.Grittner@wicourts.gov>; pgsql-performance; Albe Laurenz <laurenz.albe@wien.gv.at>


On Feb 9, 2010, at 10:37 PM, Greg Smith wrote:

> Jeff wrote:
>> I'd done some testing a while ago on the schedulers and at the time 
>> deadline or noop smashed cfq.  Now, it is 100% possible since then 
>> that they've made vast improvements to cfq and or the VM to get better 
>> or similar performance.  I recall a vintage of 2.6 where they severely 
>> messed up the VM. Glad I didn't upgrade to that one :)
>> 
>> Here's the old post: 
>> http://archives.postgresql.org/pgsql-performance/2008-04/msg00155.php
> 
> pgiosim doesn't really mix writes into there though, does it?  The mixed 
> read/write situations are the ones where the scheduler stuff gets messy.
> 

Also, read/write mix performance depend on the file system not just the scheduler.
The block device readahead parameter can have a big impact too.

If you test xfs, make sure you configure the 'allocsize' mount parameter properly as well.  If there are any sequential reads or writes in there mixed with other reads/writes, that can have a big impact on how fragmented the filesystem gets.

Ext3 has several characteristics for writes that might favor cfq that other file systems do not.  Features like delayed allocation, extents, and write barriers significantly change the pattern of writes seen by the I/O scheduler.

In short, one scheduler may be best for one filesystem, but not a good idea for others.

And then on top of that, it all depends on what type of DB you're running.  Lots of small fast mostly read queries?  Large number of small writes?  Large bulk writes?  Large reporting queries?  Different configurations and tuning is required to maximize performance on each.

There is no single rule for Postgres on Linux that I can think of other than "never have ext3 in 'ordered' or 'journal' mode for your WAL on the same filesystem as your data".

> -- 
> Greg Smith    2ndQuadrant   Baltimore, MD
> PostgreSQL Training, Services and Support
> greg@2ndQuadrant.com  www.2ndQuadrant.com
> 
> 
> -- 
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance




^ permalink  raw  reply  [nested|flat] 20+ messages in thread


end of thread, other threads:[~2010-02-16 21:34 UTC | newest]

Thread overview: 20+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-02-08 09:45 Linux I/O tuning:  CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
2010-02-08 14:57 ` Albe Laurenz <laurenz.albe@wien.gv.at>
2010-02-08 15:24   ` Kevin Grittner <Kevin.Grittner@wicourts.gov>
2010-02-08 15:30     ` Greg Smith <greg@2ndquadrant.com>
2010-02-08 17:49       ` Josh Berkus <josh@agliodbs.com>
2010-02-08 18:59         ` Re: Linux I/O tuning: CFQ vs. deadline Scott Marlowe <scott.marlowe@gmail.com>
2010-02-09 01:14         ` Re: Linux I/O tuning: CFQ vs. deadline Mark Wong <markwkm@gmail.com>
2010-02-09 03:50         ` Scott Carey <scott@richrelevance.com>
2010-02-09 04:14         ` Greg Smith <greg@2ndquadrant.com>
2010-02-16 21:34         ` Jeff Davis <pgsql@j-davis.com>
2010-02-09 04:16       ` Greg Smith <greg@2ndquadrant.com>
2010-02-09 04:35         ` david@lang.hm
2010-02-09 19:14           ` Jeff <threshar@torgo.978.org>
2010-02-10 06:37             ` Greg Smith <greg@2ndquadrant.com>
2010-02-10 07:11               ` Re: Linux I/O tuning: CFQ vs. deadline Scott Marlowe <scott.marlowe@gmail.com>
2010-02-11 01:46                 ` Re: Linux I/O tuning: CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
2010-02-11 01:52                   ` Re: Linux I/O tuning: CFQ vs. deadline david@lang.hm
2010-02-11 02:09                     ` Re: Linux I/O tuning: CFQ vs. deadline Greg Smith <greg@2ndquadrant.com>
2010-02-10 17:22               ` Jeff <threshar@torgo.978.org>
2010-02-10 18:04               ` Scott Carey <scott@richrelevance.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox