pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
postgres on a beowulf? (AMD)opteron?
9+ messages / 7 participants
[nested] [flat]

* postgres on a beowulf? (AMD)opteron?
@ 2003-05-19 17:28 george young <gry@ll.mit.edu>
  2003-05-20 06:44 ` Re: postgres on a beowulf? (AMD)opteron? Shridhar Daithankar <shridhar_daithankar@persistent.co.in>
  2003-05-20 08:29 ` Re: postgres on a beowulf? (AMD)opteron? Hannu Krosing <hannu@tm.ee>
  2003-05-20 11:27 ` Re: postgres on a beowulf? (AMD)opteron? Andrew Sullivan <andrew@libertyrms.info>
  2003-05-20 15:37 ` Re: postgres on a beowulf? (AMD)opteron? scott.marlowe <scott.marlowe@ihs.com>
  0 siblings, 4 replies; 9+ messages in thread

From: george young @ 2003-05-19 17:28 UTC (permalink / raw)
  To: pgsql-performance

Has anyone run postgres on a beowulf system? 

I'm shopping for a new server.  One candidate would be a
quad opteron (64-bit AMD "hammer") machine.  Another approach might
be a beowulf of single or dual opterons.  I imagine the beowulf
would be a bit cheaper, and much more expandable, but what about
the shared memory used by the postgres backends?  I gather that
postgres uses shared memory to coordinate (locks?) between backends?

I have a smallish DB (pgdump|bzip2 -> 10MB), with ~45 users logged in
using local X(python/gtk) postgres client apps. 

Will the much slower shared memory access between beowulf nodes be
a performance bottleneck?  

[Next question is: has anyone used postgres on an opteron at all??]

-- George
-- 
 I cannot think why the whole bed of the ocean is
 not one solid mass of oysters, so prolific they seem. Ah,
 I am wandering! Strange how the brain controls the brain!
	-- Sherlock Holmes in "The Dying Detective"



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
@ 2003-05-20 06:44 ` Shridhar Daithankar <shridhar_daithankar@persistent.co.in>
  2003-05-23 16:13   ` Re: postgres on a beowulf? (AMD)opteron? scott.marlowe <scott.marlowe@ihs.com>
  3 siblings, 1 reply; 9+ messages in thread

From: Shridhar Daithankar @ 2003-05-20 06:44 UTC (permalink / raw)
  To: pgsql-performance

On Monday 19 May 2003 22:58, george young wrote:
> Has anyone run postgres on a beowulf system?
>
> I'm shopping for a new server.  One candidate would be a
> quad opteron (64-bit AMD "hammer") machine.  Another approach might
> be a beowulf of single or dual opterons.  I imagine the beowulf
> would be a bit cheaper, and much more expandable, but what about
> the shared memory used by the postgres backends?  I gather that
> postgres uses shared memory to coordinate (locks?) between backends?

Postgresql will not run on beowulf at all since it is an MPI system and 
postgresql can no span a single database across machines (yet). Further it 
won't even run on mosix because mosix does not support shared memory across 
machines.

>
> I have a smallish DB (pgdump|bzip2 -> 10MB), with ~45 users logged in
> using local X(python/gtk) postgres client apps.

Well, you haven't put how many transactions you do, but in general for that 
sort of DB, a P-IV/512MB RAM and SCSI disk would be more than enough unless 
you are doing really exotic things with data..

> [Next question is: has anyone used postgres on an opteron at all??]

Well, if it runs linux as good as anything else, postgresql will run as good 
as anything else..:-)

HTH

 Shridhar



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
  2003-05-20 06:44 ` Re: postgres on a beowulf? (AMD)opteron? Shridhar Daithankar <shridhar_daithankar@persistent.co.in>
@ 2003-05-23 16:13   ` scott.marlowe <scott.marlowe@ihs.com>
  0 siblings, 0 replies; 9+ messages in thread

From: scott.marlowe @ 2003-05-23 16:13 UTC (permalink / raw)
  To: Shridhar Daithankar <shridhar_daithankar@persistent.co.in>; +Cc: pgsql-performance

On Tue, 20 May 2003, Shridhar Daithankar wrote:

> On Monday 19 May 2003 22:58, george young wrote:
> > Has anyone run postgres on a beowulf system?
> >
> > I'm shopping for a new server.  One candidate would be a
> > quad opteron (64-bit AMD "hammer") machine.  Another approach might
> > be a beowulf of single or dual opterons.  I imagine the beowulf
> > would be a bit cheaper, and much more expandable, but what about
> > the shared memory used by the postgres backends?  I gather that
> > postgres uses shared memory to coordinate (locks?) between backends?
> 
> Postgresql will not run on beowulf at all since it is an MPI system and 
> postgresql can no span a single database across machines (yet). Further it 
> won't even run on mosix because mosix does not support shared memory across 
> machines.
> 
> >
> > I have a smallish DB (pgdump|bzip2 -> 10MB), with ~45 users logged in
> > using local X(python/gtk) postgres client apps.
> 
> Well, you haven't put how many transactions you do, but in general for that 
> sort of DB, a P-IV/512MB RAM and SCSI disk would be more than enough unless 
> you are doing really exotic things with data..
> 
> > [Next question is: has anyone used postgres on an opteron at all??]
> 
> Well, if it runs linux as good as anything else, postgresql will run as good 
> as anything else..:-)

Keep in mind, if what you're doing is very memory intensive, then the PIV 
with it's faster memory bandwidth may be the best bet.  If it's CPU 
processing intensive (GIS calculations) or I/O intensive the AMD's should 
be competitive, but for memory I/O speed bound apps, the P IV is still 
faster.

You'll not know which is faster until you've benchmarked it yourself under 
your own load though. :-)




^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
@ 2003-05-20 08:29 ` Hannu Krosing <hannu@tm.ee>
  3 siblings, 0 replies; 9+ messages in thread

From: Hannu Krosing @ 2003-05-20 08:29 UTC (permalink / raw)
  To: gry@ll.mit.edu; +Cc: pgsql-performance

george young kirjutas E, 19.05.2003 kell 20:28:
> Has anyone run postgres on a beowulf system? 

I don't think that postgresql will easyly port to beowulf clusters.

> I'm shopping for a new server.  One candidate would be a
> quad opteron (64-bit AMD "hammer") machine.  Another approach might
> be a beowulf of single or dual opterons.  I imagine the beowulf
> would be a bit cheaper, and much more expandable, but what about
> the shared memory used by the postgres backends?  I gather that
> postgres uses shared memory to coordinate (locks?) between backends?
> 
> I have a smallish DB (pgdump|bzip2 -> 10MB), with ~45 users logged in
> using local X(python/gtk) postgres client apps. 

Why do you want such a monster machine for this smallish DB ?

Are there any special performance requirements you are not telling us
about ?

> Will the much slower shared memory access between beowulf nodes be
> a performance bottleneck?  

I guess that it will not run at all ;(

--------------
Hannu




^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
@ 2003-05-20 11:27 ` Andrew Sullivan <andrew@libertyrms.info>
  2003-05-21 14:54   ` Re: postgres on a beowulf? (AMD)opteron? Ron Johnson <ron.l.johnson@cox.net>
  3 siblings, 1 reply; 9+ messages in thread

From: Andrew Sullivan @ 2003-05-20 11:27 UTC (permalink / raw)
  To: pgsql-performance

On Mon, May 19, 2003 at 01:28:32PM -0400, george young wrote:
> Has anyone run postgres on a beowulf system? 

Can't be done.  None of the cluster systems support cross-machne
shared memory (this is actually a problem for Postgres and NUMA, as
well, BTW).

> I have a smallish DB (pgdump|bzip2 -> 10MB), with ~45 users logged in
> using local X(python/gtk) postgres client apps. 

Seems like you want a sledgehammer to kill a fly here.  That's tiny. 
Why do you want the complications of a cluster for this?

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110




^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
  2003-05-20 11:27 ` Re: postgres on a beowulf? (AMD)opteron? Andrew Sullivan <andrew@libertyrms.info>
@ 2003-05-21 14:54   ` Ron Johnson <ron.l.johnson@cox.net>
  2003-05-21 15:45     ` Re: postgres on a beowulf? (AMD)opteron? scott.marlowe <scott.marlowe@ihs.com>
  0 siblings, 1 reply; 9+ messages in thread

From: Ron Johnson @ 2003-05-21 14:54 UTC (permalink / raw)
  To: pgsql-performance

On Tue, 2003-05-20 at 06:27, Andrew Sullivan wrote:
> On Mon, May 19, 2003 at 01:28:32PM -0400, george young wrote:
> > Has anyone run postgres on a beowulf system? 
> 
> Can't be done.  None of the cluster systems support cross-machne
> shared memory (this is actually a problem for Postgres and NUMA, as
> well, BTW).

VMSclusters and Tru64 clusters do, but, of course, it has to be
programmed for.

And the licensing costs are pretty steep...

-- 
+-----------------------------------------------------------+
| Ron Johnson, Jr.     Home: ron.l.johnson@cox.net          |
| Jefferson, LA  USA   http://members.cox.net/ron.l.johnson |
|                                                           |
| Regarding war zones: "There's nothing sacrosanct about a  |
| hotel with a bunch of journalists in it."                 |
|     Marine Lt. Gen. Bernard E. Trainor (Retired)          |
+-----------------------------------------------------------+




^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
  2003-05-20 11:27 ` Re: postgres on a beowulf? (AMD)opteron? Andrew Sullivan <andrew@libertyrms.info>
  2003-05-21 14:54   ` Re: postgres on a beowulf? (AMD)opteron? Ron Johnson <ron.l.johnson@cox.net>
@ 2003-05-21 15:45     ` scott.marlowe <scott.marlowe@ihs.com>
  2003-05-22 21:23       ` Re: postgres on a beowulf? (AMD)opteron? Bruce Momjian <pgman@candle.pha.pa.us>
  0 siblings, 1 reply; 9+ messages in thread

From: scott.marlowe @ 2003-05-21 15:45 UTC (permalink / raw)
  To: Ron Johnson <ron.l.johnson@cox.net>; +Cc: pgsql-performance

On 21 May 2003, Ron Johnson wrote:

> On Tue, 2003-05-20 at 06:27, Andrew Sullivan wrote:
> > On Mon, May 19, 2003 at 01:28:32PM -0400, george young wrote:
> > > Has anyone run postgres on a beowulf system? 
> > 
> > Can't be done.  None of the cluster systems support cross-machne
> > shared memory (this is actually a problem for Postgres and NUMA, as
> > well, BTW).
> 
> VMSclusters and Tru64 clusters do, but, of course, it has to be
> programmed for.
> 
> And the licensing costs are pretty steep...

someone was on the list last year and had gotten postgresql (sorta) 
working on mosix clusters.  The performance, I recall, was less than 
stellar.




^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
  2003-05-20 11:27 ` Re: postgres on a beowulf? (AMD)opteron? Andrew Sullivan <andrew@libertyrms.info>
  2003-05-21 14:54   ` Re: postgres on a beowulf? (AMD)opteron? Ron Johnson <ron.l.johnson@cox.net>
  2003-05-21 15:45     ` Re: postgres on a beowulf? (AMD)opteron? scott.marlowe <scott.marlowe@ihs.com>
@ 2003-05-22 21:23       ` Bruce Momjian <pgman@candle.pha.pa.us>
  0 siblings, 0 replies; 9+ messages in thread

From: Bruce Momjian @ 2003-05-22 21:23 UTC (permalink / raw)
  To: scott.marlowe <scott.marlowe@ihs.com>; +Cc: Ron Johnson <ron.l.johnson@cox.net>; pgsql-performance

scott.marlowe wrote:
> On 21 May 2003, Ron Johnson wrote:
> 
> > On Tue, 2003-05-20 at 06:27, Andrew Sullivan wrote:
> > > On Mon, May 19, 2003 at 01:28:32PM -0400, george young wrote:
> > > > Has anyone run postgres on a beowulf system? 
> > > 
> > > Can't be done.  None of the cluster systems support cross-machne
> > > shared memory (this is actually a problem for Postgres and NUMA, as
> > > well, BTW).
> > 
> > VMSclusters and Tru64 clusters do, but, of course, it has to be
> > programmed for.
> > 
> > And the licensing costs are pretty steep...
> 
> someone was on the list last year and had gotten postgresql (sorta) 
> working on mosix clusters.  The performance, I recall, was less than 
> stellar.

I think they copied and locked the shared memory for each machine that
needed it.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: postgres on a beowulf? (AMD)opteron?
  2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
@ 2003-05-20 15:37 ` scott.marlowe <scott.marlowe@ihs.com>
  3 siblings, 0 replies; 9+ messages in thread

From: scott.marlowe @ 2003-05-20 15:37 UTC (permalink / raw)
  To: george young <gry@ll.mit.edu>; +Cc: pgsql-performance

On Mon, 19 May 2003, george young wrote:

> Has anyone run postgres on a beowulf system? 
> 
> I'm shopping for a new server.  One candidate would be a
> quad opteron (64-bit AMD "hammer") machine.  Another approach might
> be a beowulf of single or dual opterons.  I imagine the beowulf
> would be a bit cheaper, and much more expandable, but what about
> the shared memory used by the postgres backends?  I gather that
> postgres uses shared memory to coordinate (locks?) between backends?
> 
> I have a smallish DB (pgdump|bzip2 -> 10MB), with ~45 users logged in
> using local X(python/gtk) postgres client apps. 
> 
> Will the much slower shared memory access between beowulf nodes be
> a performance bottleneck?  

Save yourself some money on the big boxes and get a fast drive subsystem 
and lots of memory, those are more important than raw horsepower, and any
dual Opteron / Itanium2 / USparc III / PPC / Xeon machine has plenty of 
CPU ponies to handle the load.

We use dual PIII's for most of our serving, and while our front end web 
servers need to grow a bit to handle all the PHP we're throwing at them, 
the postgresql database on the dual PIII-750 is still plenty fast.  I.e. 
our bottlenecks are elsewhere than pgsql.

I don't know anyone off the top of my head that's running postgresql on an 
Opteron, by the way, but I expect it should work fine.  You're more likely 
to have problems finding a distribution that works well on top of an 
Opteron than to have problems with pgsql.




^ permalink  raw  reply  [nested|flat] 9+ messages in thread


end of thread, other threads:[~2003-05-23 16:13 UTC | newest]

Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-05-19 17:28 postgres on a beowulf? (AMD)opteron? george young <gry@ll.mit.edu>
2003-05-20 06:44 ` Shridhar Daithankar <shridhar_daithankar@persistent.co.in>
2003-05-23 16:13   ` scott.marlowe <scott.marlowe@ihs.com>
2003-05-20 08:29 ` Hannu Krosing <hannu@tm.ee>
2003-05-20 11:27 ` Andrew Sullivan <andrew@libertyrms.info>
2003-05-21 14:54   ` Ron Johnson <ron.l.johnson@cox.net>
2003-05-21 15:45     ` scott.marlowe <scott.marlowe@ihs.com>
2003-05-22 21:23       ` Bruce Momjian <pgman@candle.pha.pa.us>
2003-05-20 15:37 ` scott.marlowe <scott.marlowe@ihs.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox