pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
scaling beyond 4 processors
5+ messages / 4 participants
[nested] [flat]

* scaling beyond 4 processors
@ 2004-12-06 22:18  vogler@cipsoft.com
  0 siblings, 4 replies; 5+ messages in thread

From: vogler@cipsoft.com @ 2004-12-06 22:18 UTC (permalink / raw)
  To: pgsql-performance

Hello everyone!

Since our current Postgres server, a quad Xeon system, finally can't keep up with our 
load anymore we're ready to take the next step.

So the question is: Has anyone experiences with running Postgres on systems with 
more than 4 processors in a production environment? Which systems and 
architectures are you using (e.g. IBM xseries, IBM pseries, HP Proliant, Sun Fire, 8-
way Opteron)? How about conflicts between Postgres' shared memory approach and 
the NUMA architecture of most multi-processor machines?

Maybe it's time to switch to Oracle or DB2, but before I give up on Postgres, I wanted 
to hear some other opinions.

Thanks for any hints and suggestions.

Best regards,
Stephan Vogler
CipSoft GmbH



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: scaling beyond 4 processors
@ 2004-12-07 02:52  Jeff <threshar@torgo.978.org>
  parent: vogler@cipsoft.com
  3 siblings, 0 replies; 5+ messages in thread

From: Jeff @ 2004-12-07 02:52 UTC (permalink / raw)
  To: vogler@cipsoft.com; +Cc: pgsql-performance


On Dec 6, 2004, at 5:18 PM, vogler@cipsoft.com wrote:

> Hello everyone!
>
> Since our current Postgres server, a quad Xeon system, finally can't 
> keep up with our
> load anymore we're ready to take the next step.
>

I'm assuming you've already done as much query tweaking as possible.

and are you sure you are CPU bound and not IO bound?
(Symptoms of IO bound are low cpu usage, high load average, poor 
performance. Many processes in "D" state)

> So the question is: Has anyone experiences with running Postgres on 
> systems with
> more than 4 processors in a production environment? Which systems and

Have you also considered a replicated approach?

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/




^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: scaling beyond 4 processors
@ 2004-12-07 04:16  Christopher Browne <cbbrowne@acm.org>
  parent: vogler@cipsoft.com
  3 siblings, 0 replies; 5+ messages in thread

From: Christopher Browne @ 2004-12-07 04:16 UTC (permalink / raw)
  To: pgsql-performance

In the last exciting episode, vogler@cipsoft.com wrote:
> Hello everyone!
>
> Since our current Postgres server, a quad Xeon system, finally can't
> keep up with our load anymore we're ready to take the next step.
>
> So the question is: Has anyone experiences with running Postgres on
> systems with more than 4 processors in a production environment? 
> Which systems and architectures are you using (e.g. IBM xseries, IBM
> pseries, HP Proliant, Sun Fire, 8- way Opteron)? How about conflicts
> between Postgres' shared memory approach and the NUMA architecture
> of most multi-processor machines?

The perhaps odd thing is that just about any alternative to quad-Xeon
is likely to be _way_ better.  There are some context switching
problems that lead to it being remarkably poorer than you'd expect.
Throw in less-than ideal performance of the PAE memory addressing
system and it seems oddly crippled overall.

We've been getting pretty good results with IBM pSeries systems;
they're expensive, but definitely very fast.

Preliminary results with Opterons are also looking very promising.
One process seemed about 25x as fast on a 4-way 8GB Opteron as it was
on a 4-way 8GB Xeon, albeit with enough differences to make the
comparison dangerous.
-- 
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','gmail.com').
http://www.ntlug.org/~cbbrowne/sgml.html
The IETF motto: "Rough consensus *and* working code."



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: scaling beyond 4 processors
@ 2004-12-07 04:17  Christopher Browne <cbbrowne@acm.org>
  parent: vogler@cipsoft.com
  3 siblings, 0 replies; 5+ messages in thread

From: Christopher Browne @ 2004-12-07 04:17 UTC (permalink / raw)
  To: pgsql-performance

The perhaps odd thing is that just about any alternative to quad-Xeon
is likely to be _way_ better.  There are some context switching
problems that lead to it being remarkably poorer than you'd expect.
Throw in less-than ideal performance of the PAE memory addressing
system and it seems oddly crippled overall.

We've been getting pretty good results with IBM pSeries systems;
they're expensive, but definitely very fast.

Preliminary results with Opterons are also looking very promising.
One process seemed about 25x as fast on a 4-way 8GB Opteron as it was
on a 4-way 8GB Xeon, albeit with enough differences to make the
comparison dangerous.
-- 
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','gmail.com').
http://www.ntlug.org/~cbbrowne/sgml.html
The IETF motto: "Rough consensus *and* working code."



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: scaling beyond 4 processors
@ 2004-12-07 17:49  Josh Berkus <josh@agliodbs.com>
  parent: vogler@cipsoft.com
  3 siblings, 0 replies; 5+ messages in thread

From: Josh Berkus @ 2004-12-07 17:49 UTC (permalink / raw)
  To: pgsql-performance; +Cc: vogler@cipsoft.com

Volger,

> Since our current Postgres server, a quad Xeon system, finally can't keep
> up with our load anymore we're ready to take the next step.

There are a lot of reasons this could be happening; Quad Xeon is a problematic 
platform, the more so if you're on Dell hardware.

I've run PostgreSQL on 8-ways, and I know there are a few Sunfire users around 
the community (16-way).   There are definitely specific performance issues 
around specific query loads on multi-way systems; search this list archives 
for "Context Switch Bug".

I will echo others in saying that moving to Opteron on premium hardware should 
jump you at least 2x on performance, and it's a lot cheaper than DB2 or 
Oracle.

And, of course, if you really want help from this list, you'll post more 
specific problems.

-- 
Josh Berkus
Aglio Database Solutions
San Francisco



^ permalink  raw  reply  [nested|flat] 5+ messages in thread


end of thread, other threads:[~2004-12-07 17:49 UTC | newest]

Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-12-06 22:18 scaling beyond 4 processors vogler@cipsoft.com
2004-12-07 02:52 ` Jeff <threshar@torgo.978.org>
2004-12-07 04:16 ` Christopher Browne <cbbrowne@acm.org>
2004-12-07 04:17 ` Christopher Browne <cbbrowne@acm.org>
2004-12-07 17:49 ` Josh Berkus <josh@agliodbs.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox