pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedPostgres and Ingres R3 / SAN
6+ messages / 5 participants
[nested] [flat]
* Postgres and Ingres R3 / SAN
@ 2006-03-06 19:58 Jeremy Haile <jhaile@fastmail.fm>
0 siblings, 1 reply; 6+ messages in thread
From: Jeremy Haile @ 2006-03-06 19:58 UTC (permalink / raw)
To: pgsql-performance
Clustering solutions for PostgreSQL are currently pretty limited. Slony
could be a good option in the future, but it currently only supports
Master-Slave replication (not true clustering) and in my experience is a
pain to set up and administer. Bizgres MPP has a lot of promise,
especially for data warehouses, but it currently doesn't have the best
OLTP database performance.
So, I had a couple of questions:
1) I have heard bad things from people on this list regarding SANs - but
is there a better alternative for a high performance database cluster?
(both for redundancy and performance) I've heard internal storage
touted before, but then you have to do something like master-master
replication to get horizontal scalability and write performance will
suffer.
2) Has anyone on this list had experience using Ingres R3 in a clustered
environment? I am considering using Ingres R3's built-in clustering
support with a SAN, but am interested to know other people's experiences
before we start toying with this possibility. Any experience with the
Ingres support from Computer Associates? Good/bad?
Jeremy
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Postgres and Ingres R3 / SAN
@ 2006-03-07 19:00 Jim C. Nasby <jnasby@pervasive.com>
parent: Jeremy Haile <jhaile@fastmail.fm>
0 siblings, 1 reply; 6+ messages in thread
From: Jim C. Nasby @ 2006-03-07 19:00 UTC (permalink / raw)
To: Jeremy Haile <jhaile@fastmail.fm>; +Cc: pgsql-performance
Please don't steal threds; post a new email rather than replying to an
existing thread.
On Mon, Mar 06, 2006 at 02:58:32PM -0500, Jeremy Haile wrote:
> Clustering solutions for PostgreSQL are currently pretty limited. Slony
> could be a good option in the future, but it currently only supports
> Master-Slave replication (not true clustering) and in my experience is a
> pain to set up and administer. Bizgres MPP has a lot of promise,
> especially for data warehouses, but it currently doesn't have the best
> OLTP database performance.
>
> So, I had a couple of questions:
> 1) I have heard bad things from people on this list regarding SANs - but
> is there a better alternative for a high performance database cluster?
> (both for redundancy and performance) I've heard internal storage
> touted before, but then you have to do something like master-master
> replication to get horizontal scalability and write performance will
> suffer.
PostgreSQL on a SAN won't buy you what I think you think it will. It's
essentially impossible to safely run two PostgreSQL installs off the
same data files without destroying your data. What a SAN can buy you is
disk-level replication, but I've no experience with that.
> 2) Has anyone on this list had experience using Ingres R3 in a clustered
> environment? I am considering using Ingres R3's built-in clustering
> support with a SAN, but am interested to know other people's experiences
> before we start toying with this possibility. Any experience with the
> Ingres support from Computer Associates? Good/bad?
Can you point us at more info about this? I can't even find a website
for Ingress...
I'd be careful about OSS-based clusters. Everyone I've seen has some
limitations, some of which are pretty serious. There are some that are
command-based clustering/replication, but that raises some serious
potential issues with non-deterministic functions among other things.
Continuent seems to have done a good job dealing with this, but there's
still some gotchas you need to be aware of.
Then there's things like MySQL cluster, which requires that the entire
database fits in memory. Well, if the database is in memory, it's going
to be pretty dang fast to begin with, so you're unlikely to need
scaleability across machines.
Basically, truely enterprise-class clustering (and replication) are
extremely hard to do, which is why this is pretty much exclusively the
realm of the 'big 3' at this point. Slony-II could seriously change
things when it comes out, though it still won't give you the data
guarantees that a true syncronous multi-master setup does. But it will
hopefully offer multi-master syncronous type behavior with the
performance of an async database, which would be a huge leap forward.
Perhaps if you posted your performance requirements someone could help
point you to a solution that would meet them.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Postgres and Ingres R3 / SAN
@ 2006-03-07 19:20 Mark Lewis <mark.lewis@mir3.com>
parent: Jim C. Nasby <jnasby@pervasive.com>
0 siblings, 1 reply; 6+ messages in thread
From: Mark Lewis @ 2006-03-07 19:20 UTC (permalink / raw)
To: Jim C. Nasby <jnasby@pervasive.com>; +Cc: Jeremy Haile <jhaile@fastmail.fm>; pgsql-performance
On Tue, 2006-03-07 at 13:00 -0600, Jim C. Nasby wrote:
...
> PostgreSQL on a SAN won't buy you what I think you think it will. It's
> essentially impossible to safely run two PostgreSQL installs off the
> same data files without destroying your data. What a SAN can buy you is
> disk-level replication, but I've no experience with that.
It is possible to run two instances against the same SAN using tools
such as RedHat's Cluster Suite. We use that in-house as a cheap
alternative for Oracle clustering, although we're not using it for our
PostgreSQL servers yet. It's not for load balancing, just
active/passive fault tolerance.
-- Mark Lewis
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Postgres and Ingres R3 / SAN
@ 2006-03-07 19:22 Jim C. Nasby <jnasby@pervasive.com>
parent: Mark Lewis <mark.lewis@mir3.com>
0 siblings, 0 replies; 6+ messages in thread
From: Jim C. Nasby @ 2006-03-07 19:22 UTC (permalink / raw)
To: Mark Lewis <mark.lewis@mir3.com>; +Cc: Jeremy Haile <jhaile@fastmail.fm>; pgsql-performance
On Tue, Mar 07, 2006 at 11:20:50AM -0800, Mark Lewis wrote:
> On Tue, 2006-03-07 at 13:00 -0600, Jim C. Nasby wrote:
>
> ...
>
> > PostgreSQL on a SAN won't buy you what I think you think it will. It's
> > essentially impossible to safely run two PostgreSQL installs off the
> > same data files without destroying your data. What a SAN can buy you is
> > disk-level replication, but I've no experience with that.
>
> It is possible to run two instances against the same SAN using tools
> such as RedHat's Cluster Suite. We use that in-house as a cheap
> alternative for Oracle clustering, although we're not using it for our
> PostgreSQL servers yet. It's not for load balancing, just
> active/passive fault tolerance.
True, but the OP was talking about scaleability, which is not something
you get with this setup.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Postgres and Ingres R3 / SAN
@ 2006-03-08 19:24 Jim Nasby <jnasby@pervasive.com>
0 siblings, 1 reply; 6+ messages in thread
From: Jim Nasby @ 2006-03-08 19:24 UTC (permalink / raw)
To: Jeremy Haile <jhaile@fastmail.fm>; +Cc: pgsql-performance
Adding -performance back; you should do a reply-all if you want to reply to list messages.
> From: Jeremy Haile [mailto:jhaile@fastmail.fm]
> > Can you point us at more info about this? I can't even find
> a website
> > for Ingres...
>
> Ingres is based off of the same original codebase that PostgreSQL was
> based upon (a long time ago) It is owned by Computer
> Associates and was
> open sourced last year. It supports clustering and replication, and
> I've seen an Ingres install set up as a cluster backed by a
> SAN before.
> I just haven't talked to anyone (at least unbiased) who has used this
> type of setup in production, and I'm not fully aware of the
> advantages/disadvantages of this type of setup with Ingres.
> Since this
> group seems pretty knowledgable about performance advantages
> (and we are
> currently running PostgreSQL), I wanted to see if there were any
> experiences or opinions.
>
> Here is a link to their website:
> http://opensource.ca.com/projects/ingres
>
>
> > Perhaps if you posted your performance requirements someone
> could help
> > point you to a solution that would meet them.
>
> This is honestly more of a curiousity question at the moment,
> so I don't
> have any specific numbers. We definitely have a requirement for
> failover in the case of a machine failure, so we at least need
> Master->Slave replication. However, I wanted to solicit
> information on
> clustering alternatives as well, since scalability will likely be a
> future problem for our database.
Ahh, ok... that's likely a much different requirement than true clustering.
What a lot of folks do right now is segregate their application into a read-only stream and the more interactive read-write streams, and then use Slony to replicate data to a number of machines for the read-only work. This way anyone who's hitting the site read-only (and can handle some possible delay) will just hit one of the slave machines. People who are doing interactive work (updating data) will hit the master. Since most applications do far more reading than they do writing, this is a pretty good way to load-balance.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Postgres and Ingres R3 / SAN
@ 2006-03-11 23:06 Josh Berkus <josh@agliodbs.com>
parent: Jim Nasby <jnasby@pervasive.com>
0 siblings, 0 replies; 6+ messages in thread
From: Josh Berkus @ 2006-03-11 23:06 UTC (permalink / raw)
To: pgsql-performance; +Cc: Jim Nasby <jnasby@pervasive.com>; Jeremy Haile <jhaile@fastmail.fm>
Folks,
> > Ingres is based off of the same original codebase that PostgreSQL was
> > based upon (a long time ago)
This is wrong. According to Andrew Yu and others who date back to the
original POSTGRES, development of Postgres involved several of the same
team members as INGRES (most notably Stonebraker himself) but the two
database systems share no code. So the two systems share some ideas and
algorithms, but Postgres is a ground-up rewrite without borrowed code.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2006-03-11 23:06 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-03-06 19:58 Postgres and Ingres R3 / SAN Jeremy Haile <jhaile@fastmail.fm>
2006-03-07 19:00 ` Jim C. Nasby <jnasby@pervasive.com>
2006-03-07 19:20 ` Mark Lewis <mark.lewis@mir3.com>
2006-03-07 19:22 ` Jim C. Nasby <jnasby@pervasive.com>
2006-03-08 19:24 Re: Postgres and Ingres R3 / SAN Jim Nasby <jnasby@pervasive.com>
2006-03-11 23:06 ` Josh Berkus <josh@agliodbs.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox