pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedClustering Postgres
8+ messages / 4 participants
[nested] [flat]
* Clustering Postgres
@ 2004-05-20 18:30 Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
0 siblings, 1 reply; 8+ messages in thread
From: Kris Kiger @ 2004-05-20 18:30 UTC (permalink / raw)
To: pgsql-admin
Has anyone heard of clusgres or, better yet, had any experience with it?
I came across it while searching for active clustering projects.
http://www.linuxlabs.com/clusgres.html
Thanks in advance for the info!
Kris
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Clustering Postgres
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
@ 2004-05-21 20:43 ` Robert Treat <xzilla@users.sourceforge.net>
2004-05-24 13:59 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
0 siblings, 1 reply; 8+ messages in thread
From: Robert Treat @ 2004-05-21 20:43 UTC (permalink / raw)
To: Kris Kiger <kris@musicrebellion.com>; +Cc: pgsql-admin@postgresql.org, "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
On Thu, 2004-05-20 at 14:30, Kris Kiger wrote:
> Has anyone heard of clusgres or, better yet, had any experience with it?
> I came across it while searching for active clustering projects.
>
> http://www.linuxlabs.com/clusgres.html
>
> Thanks in advance for the info!
>
I have heard of it but I've not heard of any reports on these lists of
how well it works. If you (or anyone else) gives it a spin please try to
post an account to the list and/or if you send me the info it could
probably make its way on to techdocs.
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Clustering Postgres
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
@ 2004-05-24 13:59 ` Greg Spiegelberg <gspiegelberg@cranel.com>
2004-05-24 15:42 ` Re: Clustering Postgres Ericson Smith <eric@did-it.com>
2004-05-26 13:51 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
0 siblings, 2 replies; 8+ messages in thread
From: Greg Spiegelberg @ 2004-05-24 13:59 UTC (permalink / raw)
To: Robert Treat <xzilla@users.sourceforge.net>; +Cc: Kris Kiger <kris@musicrebellion.com>; pgsql-admin@postgresql.org, "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
Robert,
I am currently evaluating PolyServe Matrix Server which is a clustering
solution including a clustered file system (mounted read-write
everywhere).
Anything special anyone wants to know? I'm using PostgreSQL 7.4.2 in
a Linux cluster.
Greg
Robert Treat wrote:
> On Thu, 2004-05-20 at 14:30, Kris Kiger wrote:
>
>>Has anyone heard of clusgres or, better yet, had any experience with it?
>> I came across it while searching for active clustering projects.
>>
>>http://www.linuxlabs.com/clusgres.html
>>
>>Thanks in advance for the info!
>>
>
>
> I have heard of it but I've not heard of any reports on these lists of
> how well it works. If you (or anyone else) gives it a spin please try to
> post an account to the list and/or if you send me the info it could
> probably make its way on to techdocs.
>
> Robert Treat
--
Greg Spiegelberg
Product Development Manager
Cranel, Incorporated.
Phone: 614.318.4314
Fax: 614.431.8388
Email: gspiegelberg@cranel.com
Technology. Integrity. Focus. V-Solve!
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Clustering Postgres
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
2004-05-24 13:59 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
@ 2004-05-24 15:42 ` Ericson Smith <eric@did-it.com>
2004-05-24 16:28 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
1 sibling, 1 reply; 8+ messages in thread
From: Ericson Smith @ 2004-05-24 15:42 UTC (permalink / raw)
To: ; +Cc: pgsql-admin@postgresql.org, "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
Do you mean that you are actually testing this product now? If so,
what's the hardware configuration (nodes, storage server, interconnects)
that you are using?
Warmest regards,
Ericson Smith
Tracking Specialist/DBA
+-----------------------+-----------------------------------------+
| http://www.did-it.com | "3 Years ago I was worth $5M on paper!" |
| eric@did-it.com | "Sweet! Can I have fries with that?" |
| 516-255-0500 | - Tom Tomorrow |
+-----------------------+-----------------------------------------+
Greg Spiegelberg wrote:
> Robert,
>
> I am currently evaluating PolyServe Matrix Server which is a clustering
> solution including a clustered file system (mounted read-write
> everywhere).
>
> Anything special anyone wants to know? I'm using PostgreSQL 7.4.2 in
> a Linux cluster.
>
> Greg
>
>
> Robert Treat wrote:
>
>> On Thu, 2004-05-20 at 14:30, Kris Kiger wrote:
>>
>>> Has anyone heard of clusgres or, better yet, had any experience with
>>> it? I came across it while searching for active clustering projects.
>>>
>>> http://www.linuxlabs.com/clusgres.html
>>>
>>> Thanks in advance for the info!
>>>
>>
>>
>> I have heard of it but I've not heard of any reports on these lists of
>> how well it works. If you (or anyone else) gives it a spin please try to
>> post an account to the list and/or if you send me the info it could
>> probably make its way on to techdocs.
>> Robert Treat
>
>
>
Attachments:
eric.vcf (text/x-vcard, 313B) download
begin:vcard
fn:Ericson Smith
n:Smith;Ericson
org:Did-it.com;Programming
adr:#304;;55 Maple Avenue;Rockville Center;NY;11570;USA
email;internet:eric@did-it.com
title:Web Developer
tel;work:516-255-0500
tel;cell:646-483-3420
note:Nothing special!
x-mozilla-html:FALSE
url:http://www.did-it.com
version:2.1
end:vcard
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Clustering Postgres
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
2004-05-24 13:59 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
2004-05-24 15:42 ` Re: Clustering Postgres Ericson Smith <eric@did-it.com>
@ 2004-05-24 16:28 ` Greg Spiegelberg <gspiegelberg@cranel.com>
2004-05-24 17:13 ` Re: Clustering Postgres Ericson Smith <eric@did-it.com>
0 siblings, 1 reply; 8+ messages in thread
From: Greg Spiegelberg @ 2004-05-24 16:28 UTC (permalink / raw)
To: pgsql-admin; +Cc: pgsql-general@postgresql.org <pgsql-general@postgresql.org>
2 dual PIII-1.4GHz Linux RedHat 3.0 servers each with onboard AIC7xxx
and 2 U160 drives (mirrored), 2 GB of ECC memory, 2 onboard Intel Ether
Pro 100Mbps type NIC's, 1 Qlogic 2310 HBA on a 1GBs SAN using a Candera
510 storage cluster with Xyratex SATA on the backend configured with 1+0
LUN's. It will be using a private network and a small disk on the SAN
as the interconnect.
Once I have it running and I've run the necessary tests for myself
I could run a few tests for anyone else who is interested. Deadline
is June 18th when either I scrap it and try a different solution or
we have to buy in which case I'll need it for our production databases.
Below is the config I intend to use for PostgreSQL 7.4.2.
Greg
PostgreSQL 7.4.2 config & compile
configure \
--prefix=/psfs1/apps/pgsql-7.4.2 \
--enable-thread-safety \
--enable-largefiles \
--enable-odbc \
--enable-multibyte \
--enable-shared \
--enable-syslog \
--with-perl \
--with-CXX \
--with-pam \
--with-maxbackends=128 \
--with-tcl \
--with-readline \
--with-zlib
make -j 2 CPPFLAGS="-O3 -DALLOW_ABSOLUTE_DBPATHS -DLOCK_DEBUG" all
Ericson Smith wrote:
> Do you mean that you are actually testing this product now? If so,
> what's the hardware configuration (nodes, storage server, interconnects)
> that you are using?
>
> Warmest regards, Ericson Smith
> Tracking Specialist/DBA
> +-----------------------+-----------------------------------------+
> | http://www.did-it.com | "3 Years ago I was worth $5M on paper!" |
> | eric@did-it.com | "Sweet! Can I have fries with that?" |
> | 516-255-0500 | - Tom Tomorrow |
> +-----------------------+-----------------------------------------+
>
>
> Greg Spiegelberg wrote:
>
>> Robert,
>>
>> I am currently evaluating PolyServe Matrix Server which is a clustering
>> solution including a clustered file system (mounted read-write
>> everywhere).
>>
>> Anything special anyone wants to know? I'm using PostgreSQL 7.4.2 in
>> a Linux cluster.
>>
>> Greg
>>
>>
>> Robert Treat wrote:
>>
>>> On Thu, 2004-05-20 at 14:30, Kris Kiger wrote:
>>>
>>>> Has anyone heard of clusgres or, better yet, had any experience with
>>>> it? I came across it while searching for active clustering projects.
>>>>
>>>> http://www.linuxlabs.com/clusgres.html
>>>>
>>>> Thanks in advance for the info!
>>>>
>>>
>>>
>>> I have heard of it but I've not heard of any reports on these lists of
>>> how well it works. If you (or anyone else) gives it a spin please try to
>>> post an account to the list and/or if you send me the info it could
>>> probably make its way on to techdocs.
>>> Robert Treat
>>
>>
>>
>>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
--
Greg Spiegelberg
Product Development Manager
Cranel, Incorporated.
Phone: 614.318.4314
Fax: 614.431.8388
Email: gspiegelberg@cranel.com
Technology. Integrity. Focus. V-Solve!
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Clustering Postgres
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
2004-05-24 13:59 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
2004-05-24 15:42 ` Re: Clustering Postgres Ericson Smith <eric@did-it.com>
2004-05-24 16:28 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
@ 2004-05-24 17:13 ` Ericson Smith <eric@did-it.com>
0 siblings, 0 replies; 8+ messages in thread
From: Ericson Smith @ 2004-05-24 17:13 UTC (permalink / raw)
To: Greg Spiegelberg <gspiegelberg@cranel.com>; +Cc: pgsql-admin@postgresql.org, "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
We'd be very interested in the performance, especially with the disk
being used as in interconnect (I assume that that is being used to
simulate global shared memory?)
We did have a talk with the folks over at http://www.linuxlabs.com/clustgres
They were honest enough to say that even with the Dolphon interconnects, that writing would be slower than usual. We have not made a complete decision on that yet, but are still looking at alternatives.
It seems that Clustres might be a good solution. They use a SAN, the Dolphin interconnects, and multiple diskless nodes. In theory I am sure it works, but this is one of the chicken/egg problems where we want to see it in general use first before committing :-)
- Ericson
Greg Spiegelberg wrote:
> 2 dual PIII-1.4GHz Linux RedHat 3.0 servers each with onboard AIC7xxx
> and 2 U160 drives (mirrored), 2 GB of ECC memory, 2 onboard Intel Ether
> Pro 100Mbps type NIC's, 1 Qlogic 2310 HBA on a 1GBs SAN using a Candera
> 510 storage cluster with Xyratex SATA on the backend configured with 1+0
> LUN's. It will be using a private network and a small disk on the SAN
> as the interconnect.
>
> Once I have it running and I've run the necessary tests for myself
> I could run a few tests for anyone else who is interested. Deadline
> is June 18th when either I scrap it and try a different solution or
> we have to buy in which case I'll need it for our production databases.
> Below is the config I intend to use for PostgreSQL 7.4.2.
>
> Greg
>
>
> PostgreSQL 7.4.2 config & compile
> configure \
> --prefix=/psfs1/apps/pgsql-7.4.2 \
> --enable-thread-safety \
> --enable-largefiles \
> --enable-odbc \
> --enable-multibyte \
> --enable-shared \
> --enable-syslog \
> --with-perl \
> --with-CXX \
> --with-pam \
> --with-maxbackends=128 \
> --with-tcl \
> --with-readline \
> --with-zlib
> make -j 2 CPPFLAGS="-O3 -DALLOW_ABSOLUTE_DBPATHS -DLOCK_DEBUG" all
>
>
>
> Ericson Smith wrote:
>
>> Do you mean that you are actually testing this product now? If so,
>> what's the hardware configuration (nodes, storage server,
>> interconnects) that you are using?
>>
>> Warmest regards, Ericson Smith
>> Tracking Specialist/DBA
>> +-----------------------+-----------------------------------------+
>> | http://www.did-it.com | "3 Years ago I was worth $5M on paper!" |
>> | eric@did-it.com | "Sweet! Can I have fries with that?" |
>> | 516-255-0500 | - Tom Tomorrow |
>> +-----------------------+-----------------------------------------+
>>
>>
>> Greg Spiegelberg wrote:
>>
>>> Robert,
>>>
>>> I am currently evaluating PolyServe Matrix Server which is a clustering
>>> solution including a clustered file system (mounted read-write
>>> everywhere).
>>>
>>> Anything special anyone wants to know? I'm using PostgreSQL 7.4.2 in
>>> a Linux cluster.
>>>
>>> Greg
>>>
>>>
>>> Robert Treat wrote:
>>>
>>>> On Thu, 2004-05-20 at 14:30, Kris Kiger wrote:
>>>>
>>>>> Has anyone heard of clusgres or, better yet, had any experience
>>>>> with it? I came across it while searching for active clustering
>>>>> projects.
>>>>>
>>>>> http://www.linuxlabs.com/clusgres.html
>>>>>
>>>>> Thanks in advance for the info!
>>>>>
>>>>
>>>>
>>>> I have heard of it but I've not heard of any reports on these lists of
>>>> how well it works. If you (or anyone else) gives it a spin please
>>>> try to
>>>> post an account to the list and/or if you send me the info it could
>>>> probably make its way on to techdocs.
>>>> Robert Treat
>>>
>>>
>>>
>>>
>>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 4: Don't 'kill -9' the postmaster
>
>
>
Attachments:
eric.vcf (text/x-vcard, 313B) download
begin:vcard
fn:Ericson Smith
n:Smith;Ericson
org:Did-it.com;Programming
adr:#304;;55 Maple Avenue;Rockville Center;NY;11570;USA
email;internet:eric@did-it.com
title:Web Developer
tel;work:516-255-0500
tel;cell:646-483-3420
note:Nothing special!
x-mozilla-html:FALSE
url:http://www.did-it.com
version:2.1
end:vcard
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Clustering Postgres
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
2004-05-24 13:59 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
@ 2004-05-26 13:51 ` Robert Treat <xzilla@users.sourceforge.net>
2004-05-27 19:17 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
1 sibling, 1 reply; 8+ messages in thread
From: Robert Treat @ 2004-05-26 13:51 UTC (permalink / raw)
To: Greg Spiegelberg <gspiegelberg@cranel.com>; +Cc: Kris Kiger <kris@musicrebellion.com>; pgsql-admin@postgresql.org, "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
Some basics on how easy it is to install/configure and some before/after type
benchmarks showing how it affected performance. Perferrably you could
measure simple vs. complex queries as well as update vs. insert performance.
If you're really ambitious you could try running one of the tpc style
benchmarks on the system, check the archives or the osdl site for more info
on getting these up and running.
Robert Treat
On Monday 24 May 2004 09:59, Greg Spiegelberg wrote:
> Robert,
>
> I am currently evaluating PolyServe Matrix Server which is a clustering
> solution including a clustered file system (mounted read-write
> everywhere).
>
> Anything special anyone wants to know? I'm using PostgreSQL 7.4.2 in
> a Linux cluster.
>
> Greg
>
> Robert Treat wrote:
> > On Thu, 2004-05-20 at 14:30, Kris Kiger wrote:
> >>Has anyone heard of clusgres or, better yet, had any experience with it?
> >> I came across it while searching for active clustering projects.
> >>
> >>http://www.linuxlabs.com/clusgres.html
> >>
> >>Thanks in advance for the info!
> >
> > I have heard of it but I've not heard of any reports on these lists of
> > how well it works. If you (or anyone else) gives it a spin please try to
> > post an account to the list and/or if you send me the info it could
> > probably make its way on to techdocs.
> >
> > Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Clustering Postgres
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
2004-05-24 13:59 ` Re: Clustering Postgres Greg Spiegelberg <gspiegelberg@cranel.com>
2004-05-26 13:51 ` Re: Clustering Postgres Robert Treat <xzilla@users.sourceforge.net>
@ 2004-05-27 19:17 ` Greg Spiegelberg <gspiegelberg@cranel.com>
0 siblings, 0 replies; 8+ messages in thread
From: Greg Spiegelberg @ 2004-05-27 19:17 UTC (permalink / raw)
To: Robert Treat <xzilla@users.sourceforge.net>; +Cc: Kris Kiger <kris@musicrebellion.com>; pgsql-admin@postgresql.org, "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
Here's the basics basically because I ran out of time and budget to
actually purchase it but here's what I know.
1 day to install. Installed RedHat ES 3.0 from scratch, and the
PolyServe process took perhaps 1.5 hours to install including a
kernel build and system reboot. Roughly 2 hours into PolyServe I
had my first psfs (shared cluster) partition setup on my 1Gb SAN.
Just doing a basic dd to write 1.5GB and 5GB to an ext3 on the
same hardware produced roughly 115 MB/s and 107 MB/s respectively.
It's my understand that PostgreSQL won't create files largers than
2GB though.
Doing the same tests on the PolyServe file system (psfs) accomplished
the tasks in 117 MB/s and 109 MB/s, just slightly faster.
If you're familiar with Veritas Cluster Server it operates similiar
to it though I didn't have a chance to fully test it.
Given the budget I'd definitely take another look at it.
Greg
Robert Treat wrote:
> Some basics on how easy it is to install/configure and some before/after type
> benchmarks showing how it affected performance. Perferrably you could
> measure simple vs. complex queries as well as update vs. insert performance.
> If you're really ambitious you could try running one of the tpc style
> benchmarks on the system, check the archives or the osdl site for more info
> on getting these up and running.
>
> Robert Treat
>
> On Monday 24 May 2004 09:59, Greg Spiegelberg wrote:
>
>>Robert,
>>
>>I am currently evaluating PolyServe Matrix Server which is a clustering
>>solution including a clustered file system (mounted read-write
>>everywhere).
>>
>>Anything special anyone wants to know? I'm using PostgreSQL 7.4.2 in
>>a Linux cluster.
>>
>>Greg
>>
>>Robert Treat wrote:
>>
>>>On Thu, 2004-05-20 at 14:30, Kris Kiger wrote:
>>>
>>>>Has anyone heard of clusgres or, better yet, had any experience with it?
>>>>I came across it while searching for active clustering projects.
>>>>
>>>>http://www.linuxlabs.com/clusgres.html
>>>>
>>>>Thanks in advance for the info!
>>>
>>>I have heard of it but I've not heard of any reports on these lists of
>>>how well it works. If you (or anyone else) gives it a spin please try to
>>>post an account to the list and/or if you send me the info it could
>>>probably make its way on to techdocs.
>>>
>>>Robert Treat
>
>
--
Greg Spiegelberg
Product Development Manager
Cranel, Incorporated.
Phone: 614.318.4314
Fax: 614.431.8388
Email: gspiegelberg@cranel.com
Technology. Integrity. Focus. V-Solve!
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2004-05-27 19:17 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-05-20 18:30 Clustering Postgres Kris Kiger <kris@musicrebellion.com>
2004-05-21 20:43 ` Robert Treat <xzilla@users.sourceforge.net>
2004-05-24 13:59 ` Greg Spiegelberg <gspiegelberg@cranel.com>
2004-05-24 15:42 ` Ericson Smith <eric@did-it.com>
2004-05-24 16:28 ` Greg Spiegelberg <gspiegelberg@cranel.com>
2004-05-24 17:13 ` Ericson Smith <eric@did-it.com>
2004-05-26 13:51 ` Robert Treat <xzilla@users.sourceforge.net>
2004-05-27 19:17 ` Greg Spiegelberg <gspiegelberg@cranel.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox