pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Ron Peacetree <rjpeace@earthlink.net>
To: josh@agliodbs.com
To: pgsql-hackers@postgresql.org
To: pgsql-performance@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
Date: Mon, 3 Oct 2005 20:07:02 -0400 (EDT)
Message-ID: <21602981.1128384422354.JavaMail.root@elwamui-polski.atl.sa.earthlink.net> (raw)

Let's pretend we get a 24HD HW RAID solution like that J Baker
says he has access to and set it up as a RAID 10.  Assuming
it uses two 64b 133MHz PCI-X busses and has the fastest HDs
available on it,  Jeff says he can hit ~1GBps of XFS FS IO rate
with that set up (12*83.3MBps= 1GBps).

Josh says that pg can't do more than 25MBps of DB level IO
regardless of how fast the physical IO subsystem is because at
25MBps, pg is CPU bound.  

Just how bad is this CPU bound condition?  How powerful a CPU is
needed to attain a DB IO rate of 25MBps?
 
If we replace said CPU with one 2x, 10x, etc faster than that, do we
see any performance increase?

If a modest CPU can drive a DB IO rate of 25MBps, but that rate
does not go up regardless of how much extra CPU we throw at
it...

Ron 

-----Original Message-----
From: Josh Berkus <josh@agliodbs.com>
Sent: Oct 3, 2005 6:03 PM
To: "Jeffrey W. Baker" <jwbaker@acm.org>
Cc: 
Subject: Re: [HACKERS] [PERFORM] A Better External Sort?

Jeffrey,

> I guess database reads are different, but I remain unconvinced that they
> are *fundamentally* different.  After all, a tab-delimited file (my sort
> workload) is a kind of database.

Unfortunately, they are ... because of CPU overheads.   I'm basing what's 
"reasonable" for data writes on the rates which other high-end DBs can 
make.   From that, 25mb/s or even 40mb/s for sorts should be achievable 
but doing 120mb/s would require some kind of breakthrough.

> On a single disk you wouldn't notice, but XFS scales much better when
> you throw disks at it.  I get a 50MB/sec boost from the 24th disk,
> whereas ext3 stops scaling after 16 disks.  For writes both XFS and ext3
> top out around 8 disks, but in this case XFS tops out at 500MB/sec while
> ext3 can't break 350MB/sec.

That would explain it.  I seldom get more than 6 disks (and 2 channels) to 
test with.

-- 
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

               http://archives.postgresql.org




view thread (112+ messages)  latest in thread

Message-ID: <21602981.1128384422354.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>
Permalink:  ../21602981.1128384422354.JavaMail.root@elwamui-polski.atl.sa.earthlink.net/
Also on:    postgresql.org/message-id/21602981.1128384422354.JavaMail.root@elwamui-polski.atl.sa.earthlink.net

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: rjpeace@earthlink.net, josh@agliodbs.com, pgsql-hackers@postgresql.org
  Subject: Re: [HACKERS] A Better External Sort?
  In-Reply-To: <21602981.1128384422354.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>
  References: <21602981.1128384422354.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox