pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedReplication: Slony-I vs. Mammoth Replicator vs. ?
8+ messages / 5 participants
[nested] [flat]
* Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-14 01:27 Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
0 siblings, 1 reply; 8+ messages in thread
From: Chris Cheston @ 2004-08-14 01:27 UTC (permalink / raw)
To: pgsql-performance
HI all, I'm trying to implement a highly-scalable, high-performance,
real-time database replication system to back-up my Postgres database
as data gets written.
So far, Mammoth Replicator is looking pretty good but it costs $1000+ .
Has anyone tried Slony-I and other replication systems? Slony-I is
pretty new so I'm a little unsure if it's ready for a prime-time
commercial system yet.
So... wanted to put this out to the experts. Has anyone got any
recommendations or had experiences with real-time database replication
solutions that don't rely on RAID? The reason why I don't want to
rely on a hardware solution is because we are renting dedicated
servers and we don't have access to the boxes, only to software that
gets installed on the boxes.
Thanks,
Chris
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
@ 2004-08-14 01:39 ` Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Jan Wieck <JanWieck@Yahoo.com>
0 siblings, 1 reply; 8+ messages in thread
From: Joshua D. Drake @ 2004-08-14 01:39 UTC (permalink / raw)
To: Chris Cheston <ccheston@gmail.com>; +Cc: pgsql-performance
Chris Cheston wrote:
> HI all, I'm trying to implement a highly-scalable, high-performance,
> real-time database replication system to back-up my Postgres database
> as data gets written.
>
> So far, Mammoth Replicator is looking pretty good but it costs $1000+ .
Yes but it includes 30 days of support and 12 months of upgrades/updates :)
> Has anyone tried Slony-I and other replication systems? Slony-I is
> pretty new so I'm a little unsure if it's ready for a prime-time
> commercial system yet.
It really depends on your needs. They are both good systems. Slony-I is
a bit more of a beast to get up and running, and it is a batch
replication system that uses triggers. Once it is up and running it
works well though.
Mammoth Replicator is easy to setup and is integrated into PostgreSQL.
However replicator is 1000+ and doesn't support promoting of slaves
automatically (you can do it by hand) like Slony does. Replicator is
also live replication.
Sincerely,
Joshua D. Drake
>
> So... wanted to put this out to the experts. Has anyone got any
> recommendations or had experiences with real-time database replication
> solutions that don't rely on RAID? The reason why I don't want to
> rely on a hardware solution is because we are renting dedicated
> servers and we don't have access to the boxes, only to software that
> gets installed on the boxes.
>
> Thanks,
> Chris
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if your
> joining column's datatypes do not match
--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachments:
jd.vcf (text/x-vcard, 638B) download
begin:vcard
fn:Joshua D. Drake
n:Drake;Joshua D.
org:Command Prompt, Inc.
adr:;;PO Box 215;Cascade Locks;Oregon;97014;USA
email;internet:jd@commandprompt.com
title:Consultant
tel;work:503-667-4564
tel;fax:503-210-0034
note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl.
x-mozilla-html:FALSE
url:http://www.commandprompt.com/
version:2.1
end:vcard
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
@ 2004-08-14 15:33 ` Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 16:22 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
0 siblings, 1 reply; 8+ messages in thread
From: Jan Wieck @ 2004-08-14 15:33 UTC (permalink / raw)
To: Joshua D. Drake <jd@commandprompt.com>; +Cc: Chris Cheston <ccheston@gmail.com>; pgsql-performance
On 8/13/2004 9:39 PM, Joshua D. Drake wrote:
> Chris Cheston wrote:
>> HI all, I'm trying to implement a highly-scalable, high-performance,
>> real-time database replication system to back-up my Postgres database
>> as data gets written.
>>
>> So far, Mammoth Replicator is looking pretty good but it costs $1000+ .
>
> Yes but it includes 30 days of support and 12 months of upgrades/updates :)
The lead developer of Slony (me) is working for a company that has it
deployed in production already and will move more mission critical
systems to it very shortly. Guess what will be my top work priority if
you spot a bug?
>
>
>> Has anyone tried Slony-I and other replication systems? Slony-I is
>> pretty new so I'm a little unsure if it's ready for a prime-time
>> commercial system yet.
>
> It really depends on your needs. They are both good systems. Slony-I is
> a bit more of a beast to get up and running, and it is a batch
> replication system that uses triggers. Once it is up and running it
> works well though.
>
> Mammoth Replicator is easy to setup and is integrated into PostgreSQL.
> However replicator is 1000+ and doesn't support promoting of slaves
> automatically (you can do it by hand) like Slony does. Replicator is
> also live replication.
Once again, Joshua, would you please explain what you mean with "batch"
and "live" replication system? Slony does group multiple "master"
transactions into one replication transaction to improve performance
(fewer commits on the slaves). The interval of these groups is
configurable and for high volume DBs it is recommended to use about one
second, which means that all commits that fall into an interval of one
second are replicated in one transaction on the slave. On normal running
systems this results in a replication lag of 600 to 800 milliseconds in
average. On overloaded systems the asynchronous nature of course allows
the slaves to fall behind.
What is a usual average replication lag of Mammoth Replicator?
What happens to the other existing slaves when you promote by hand? In
Slony they accept the new master and continue replicating without the
need of rebuilding from scratch. Slony has mechanisms to ensure the new
master will be ahead or equal in the replication process at the time it
takes over and allows client application updates.
The Slony documentation is an issue at the moment and the administrative
tools around it are immature. The replication engine itself exceeds my
own expectations and performs very robust.
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Jan Wieck <JanWieck@Yahoo.com>
@ 2004-08-14 16:22 ` Joshua D. Drake <jd@commandprompt.com>
2004-08-14 17:56 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 19:58 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Christopher Browne <cbbrowne@acm.org>
0 siblings, 2 replies; 8+ messages in thread
From: Joshua D. Drake @ 2004-08-14 16:22 UTC (permalink / raw)
To: Jan Wieck <JanWieck@Yahoo.com>; +Cc: Chris Cheston <ccheston@gmail.com>; pgsql-performance
>
> Once again, Joshua, would you please explain what you mean with
> "batch" and "live" replication system? Slony does group multiple
> "master" transactions into one replication transaction to improve
> performance (fewer commits on the slaves). The interval of these
> groups is configurable and for high volume DBs it is recommended to
> use about one second, which means that all commits that fall into an
> interval of one second are replicated in one transaction on the slave.
> On normal running systems this results in a replication lag of 600 to
> 800 milliseconds in average. On overloaded systems the asynchronous
> nature of course allows the slaves to fall behind.
Your description above is what I considered batch... you are taking a
"batch" of transactions and replicating them versus each transaction. I
am not saying it is bad in any way. I am just saying it is different
that replicator.
> What is a usual average replication lag of Mammoth Replicator?
>
Obviously it depends on the system, the network connectivity between the
systems etc... In our test systems it takes less than 100 ms to
replicate the data. Again it depends on the size of the transaction (the
data being moved).
> What happens to the other existing slaves when you promote by hand?
This is something that Slony has over replicator. Currently the new
master will force a full dump to the slaves. Of course this is already
on the road map, thanks to Slony :) and should be resolved by months end.
> The Slony documentation is an issue at the moment and the
> administrative tools around it are immature. The replication engine
> itself exceeds my own expectations and performs very robust.
>
I have never suggested otherwise. My only comment about maturity is that
their are actually many companies using replicator in production. We
have already dealt with the 1.0 blues as they say.
I hope you understand that I, in no way have ever suggested (purposely)
anything negative about Slony. Only that I believe they serve different
technical solutions.
Sincerely,
Joshua D. Drake
>
> Jan
>
--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 16:22 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
@ 2004-08-14 17:56 ` Jan Wieck <JanWieck@Yahoo.com>
1 sibling, 0 replies; 8+ messages in thread
From: Jan Wieck @ 2004-08-14 17:56 UTC (permalink / raw)
To: Joshua D. Drake <jd@commandprompt.com>; +Cc: Chris Cheston <ccheston@gmail.com>; pgsql-performance
On 8/14/2004 12:22 PM, Joshua D. Drake wrote:
> I hope you understand that I, in no way have ever suggested (purposely)
> anything negative about Slony. Only that I believe they serve different
> technical solutions.
You know I never took anything you said negative. I think People here
need to know that we two have communicated and collaborated outside of
the public mailing lists, that we still do it and that we do not
consider each other as opponents. There is a natural overlap in the
systems we offer and therefore there is some competition. Software
development is a sport. For some professionals this sport happens to be
the main source of income. But that shall not spoil the spirit here.
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 16:22 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
@ 2004-08-14 19:58 ` Christopher Browne <cbbrowne@acm.org>
2004-08-15 00:55 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
1 sibling, 1 reply; 8+ messages in thread
From: Christopher Browne @ 2004-08-14 19:58 UTC (permalink / raw)
To: pgsql-performance
Centuries ago, Nostradamus foresaw when jd@commandprompt.com ("Joshua D. Drake") would write:
> I hope you understand that I, in no way have ever suggested
> (purposely) anything negative about Slony. Only that I believe they
> serve different technical solutions.
Stipulating that I may have some bias ;-), I still don't find it at
all clear what the different situations are "shaped like" that lead to
Mammoth being forcibly preferable to Slony-I.
(Note that I have a pretty decent understanding about how ERS and
Slony work, so I'm not too frightened of technicalities... I set up
instances of both on Thursday, so I'm pretty up to speed :-).)
Win32 support may be true at the moment, although I have to discount
that as we only just got the start of a beta release of native Win32
support for PostgreSQL proper. For that very reason, I had to point
my youngest brother who needed "something better than Access" to
Firebird last Saturday; I played with my niece while he was doing the
install. And there is little reason to think that Slony-I won't be
portable to Win32 given a little interest and effort, particularly
once work to make it play well with "pgxs" gets done.
--
(format nil "~S@~S" "cbbrowne" "ntlug.org")
http://www3.sympatico.ca/cbbrowne/multiplexor.html
"At Microsoft, it doesn't matter which file you're compiling, only
which flags you #define." -- Colin Plumb
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 16:22 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
2004-08-14 19:58 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Christopher Browne <cbbrowne@acm.org>
@ 2004-08-15 00:55 ` Joshua D. Drake <jd@commandprompt.com>
2004-08-16 15:12 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Vivek Khera <khera@kcilink.com>
0 siblings, 1 reply; 8+ messages in thread
From: Joshua D. Drake @ 2004-08-15 00:55 UTC (permalink / raw)
To: Christopher Browne <cbbrowne@acm.org>; +Cc: pgsql-performance
Christopher Browne wrote:
>Centuries ago, Nostradamus foresaw when jd@commandprompt.com ("Joshua D. Drake") would write:
>
>
>>I hope you understand that I, in no way have ever suggested
>>(purposely) anything negative about Slony. Only that I believe they
>>serve different technical solutions.
>>
>>
>
>Stipulating that I may have some bias ;-), I still don't find it at
>all clear what the different situations are "shaped like" that lead to
>Mammoth being forcibly preferable to Slony-I.
>
>
I would choose replicator if:
1. You want ease of setup
2. You want your each transaction to be replicated at time of commit
3. Your database is already laden with triggers
4. You are pushing a very high transactional load*
* Caveat I have no idea how well Slony performs on a system that does
say 200,000 transactions
an hours that are heavily geared toward updates. Replicator performs
very well in this scenario.
5. Replicators administrative tools are more mature than Slony (for
example you know exactly what state your slaves are in with Replicator).
I would choose Slony if:
1. The fact that it is Open Source matters to you
2. The auto promotion of slaves is important*
*This will be fixed in a couple of weeks with Replicator
To be fair, in the real world ---
It doesn't make a bit of difference which one you choose it really comes
down to this:
Replicator is dumb simple to setup. Any halfway talented person can
setup replicator
in 30 minutes with a single master / slave configuration.
Slony is Open Source and thus a little easier on the pocket book initially.
Command Prompt, will support either one -- so the Replicator is
commercially supported
argument is a little weak here.
Sincerely,
Joshua D. Drake
Sincerely,
Joshua D. Drake
>(Note that I have a pretty decent understanding about how ERS and
>Slony work, so I'm not too frightened of technicalities... I set up
>instances of both on Thursday, so I'm pretty up to speed :-).)
>
>Win32 support may be true at the moment, although I have to discount
>that as we only just got the start of a beta release of native Win32
>support for PostgreSQL proper. For that very reason, I had to point
>my youngest brother who needed "something better than Access" to
>Firebird last Saturday; I played with my niece while he was doing the
>install. And there is little reason to think that Slony-I won't be
>portable to Win32 given a little interest and effort, particularly
>once work to make it play well with "pgxs" gets done.
>
>
--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 16:22 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
2004-08-14 19:58 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Christopher Browne <cbbrowne@acm.org>
2004-08-15 00:55 ` Re: Replication: Slony-I vs. Mammoth Replicator vs. ? Joshua D. Drake <jd@commandprompt.com>
@ 2004-08-16 15:12 ` Vivek Khera <khera@kcilink.com>
0 siblings, 0 replies; 8+ messages in thread
From: Vivek Khera @ 2004-08-16 15:12 UTC (permalink / raw)
To: pgsql-performance
One more point for your list:
Choose Slony if Replicator doesn't support your platform. :-)
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: khera@kciLink.com Rockville, MD +1-301-869-4449 x806
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2004-08-16 15:12 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33 ` Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 16:22 ` Joshua D. Drake <jd@commandprompt.com>
2004-08-14 17:56 ` Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 19:58 ` Christopher Browne <cbbrowne@acm.org>
2004-08-15 00:55 ` Joshua D. Drake <jd@commandprompt.com>
2004-08-16 15:12 ` Vivek Khera <khera@kcilink.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox