pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Replication: Slony-I vs. Mammoth Replicator vs. ?
8+ messages / 5 participants
[nested] [flat]

* Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-14 01:27  Chris Cheston <ccheston@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Chris Cheston @ 2004-08-14 01:27 UTC (permalink / raw)
  To: pgsql-performance

HI all, I'm trying to implement a highly-scalable, high-performance,
real-time database replication system to back-up my Postgres database
as data gets written.

So far, Mammoth Replicator is looking pretty good but it costs $1000+ .  

Has anyone tried Slony-I and other replication systems?  Slony-I is
pretty new so I'm a little unsure if it's ready for a prime-time
commercial system yet.

So... wanted to put this out to the experts.  Has anyone got any
recommendations or had experiences with real-time database replication
solutions that don't rely on RAID?  The reason why I don't want to
rely on a hardware solution is because we are renting dedicated
servers and we don't have access to the boxes, only to software that
gets installed on the boxes.

Thanks,
Chris



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-14 01:39  Joshua D. Drake <jd@commandprompt.com>
  parent: Chris Cheston <ccheston@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Joshua D. Drake @ 2004-08-14 01:39 UTC (permalink / raw)
  To: Chris Cheston <ccheston@gmail.com>; +Cc: pgsql-performance

Chris Cheston wrote:
> HI all, I'm trying to implement a highly-scalable, high-performance,
> real-time database replication system to back-up my Postgres database
> as data gets written.
> 
> So far, Mammoth Replicator is looking pretty good but it costs $1000+ .  

Yes but it includes 30 days of support and 12 months of upgrades/updates :)


> Has anyone tried Slony-I and other replication systems?  Slony-I is
> pretty new so I'm a little unsure if it's ready for a prime-time
> commercial system yet.

It really depends on your needs. They are both good systems. Slony-I is 
a bit more of a beast to get up and running, and it is a batch 
replication system that uses triggers. Once it is up and running it 
works well though.

Mammoth Replicator is easy to setup and is integrated into PostgreSQL.
However replicator is 1000+ and doesn't support promoting of slaves 
automatically (you can do it by hand) like Slony does. Replicator is
also live replication.

Sincerely,

Joshua D. Drake



> 
> So... wanted to put this out to the experts.  Has anyone got any
> recommendations or had experiences with real-time database replication
> solutions that don't rely on RAID?  The reason why I don't want to
> rely on a hardware solution is because we are renting dedicated
> servers and we don't have access to the boxes, only to software that
> gets installed on the boxes.
> 
> Thanks,
> Chris
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if your
>       joining column's datatypes do not match


-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL

Attachments:

jd.vcf (text/x-vcard, 638B) download begin:vcard fn:Joshua D. Drake n:Drake;Joshua D. org:Command Prompt, Inc. adr:;;PO Box 215;Cascade Locks;Oregon;97014;USA email;internet:jd@commandprompt.com title:Consultant tel;work:503-667-4564 tel;fax:503-210-0034 note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl. x-mozilla-html:FALSE url:http://www.commandprompt.com/ version:2.1 end:vcard
^ permalink raw reply [nested|flat] 8+ messages in thread

* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-14 15:33  Jan Wieck <JanWieck@Yahoo.com>
  parent: Joshua D. Drake <jd@commandprompt.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Jan Wieck @ 2004-08-14 15:33 UTC (permalink / raw)
  To: Joshua D. Drake <jd@commandprompt.com>; +Cc: Chris Cheston <ccheston@gmail.com>; pgsql-performance

On 8/13/2004 9:39 PM, Joshua D. Drake wrote:

> Chris Cheston wrote:
>> HI all, I'm trying to implement a highly-scalable, high-performance,
>> real-time database replication system to back-up my Postgres database
>> as data gets written.
>> 
>> So far, Mammoth Replicator is looking pretty good but it costs $1000+ .  
> 
> Yes but it includes 30 days of support and 12 months of upgrades/updates :)

The lead developer of Slony (me) is working for a company that has it 
deployed in production already and will move more mission critical 
systems to it very shortly. Guess what will be my top work priority if 
you spot a bug?

> 
> 
>> Has anyone tried Slony-I and other replication systems?  Slony-I is
>> pretty new so I'm a little unsure if it's ready for a prime-time
>> commercial system yet.
> 
> It really depends on your needs. They are both good systems. Slony-I is 
> a bit more of a beast to get up and running, and it is a batch 
> replication system that uses triggers. Once it is up and running it 
> works well though.
> 
> Mammoth Replicator is easy to setup and is integrated into PostgreSQL.
> However replicator is 1000+ and doesn't support promoting of slaves 
> automatically (you can do it by hand) like Slony does. Replicator is
> also live replication.

Once again, Joshua, would you please explain what you mean with "batch" 
and "live" replication system? Slony does group multiple "master" 
transactions into one replication transaction to improve performance 
(fewer commits on the slaves). The interval of these groups is 
configurable and for high volume DBs it is recommended to use about one 
second, which means that all commits that fall into an interval of one 
second are replicated in one transaction on the slave. On normal running 
systems this results in a replication lag of 600 to 800 milliseconds in 
average. On overloaded systems the asynchronous nature of course allows 
the slaves to fall behind.

What is a usual average replication lag of Mammoth Replicator?

What happens to the other existing slaves when you promote by hand? In 
Slony they accept the new master and continue replicating without the 
need of rebuilding from scratch. Slony has mechanisms to ensure the new 
master will be ahead or equal in the replication process at the time it 
takes over and allows client application updates.

The Slony documentation is an issue at the moment and the administrative 
tools around it are immature. The replication engine itself exceeds my 
own expectations and performs very robust.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-14 16:22  Joshua D. Drake <jd@commandprompt.com>
  parent: Jan Wieck <JanWieck@Yahoo.com>
  0 siblings, 2 replies; 8+ messages in thread

From: Joshua D. Drake @ 2004-08-14 16:22 UTC (permalink / raw)
  To: Jan Wieck <JanWieck@Yahoo.com>; +Cc: Chris Cheston <ccheston@gmail.com>; pgsql-performance


>
> Once again, Joshua, would you please explain what you mean with 
> "batch" and "live" replication system? Slony does group multiple 
> "master" transactions into one replication transaction to improve 
> performance (fewer commits on the slaves). The interval of these 
> groups is configurable and for high volume DBs it is recommended to 
> use about one second, which means that all commits that fall into an 
> interval of one second are replicated in one transaction on the slave. 
> On normal running systems this results in a replication lag of 600 to 
> 800 milliseconds in average. On overloaded systems the asynchronous 
> nature of course allows the slaves to fall behind.


Your description above is what I considered batch... you are taking a 
"batch" of transactions and replicating them versus each transaction. I 
am not saying it is bad in any way. I am just saying it is different 
that replicator.

> What is a usual average replication lag of Mammoth Replicator?
>
Obviously it depends on the system, the network connectivity between the 
systems etc... In our test systems it takes less than 100 ms to 
replicate the data. Again it depends on the size of the transaction (the 
data being moved).

> What happens to the other existing slaves when you promote by hand? 

This is something that Slony has over replicator. Currently the new 
master will force a full dump to the slaves. Of course this is already 
on the road map, thanks to Slony :) and should be resolved by months end.

> The Slony documentation is an issue at the moment and the 
> administrative tools around it are immature. The replication engine 
> itself exceeds my own expectations and performs very robust.
>
I have never suggested otherwise. My only comment about maturity is that 
their are actually many companies using replicator in production. We 
have already dealt with the 1.0 blues as they say.

I hope you understand that I, in no way have ever suggested (purposely) 
anything negative about Slony. Only that I believe they serve different 
technical solutions.

Sincerely,

Joshua D. Drake



>
> Jan
>


-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL




^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-14 17:56  Jan Wieck <JanWieck@Yahoo.com>
  parent: Joshua D. Drake <jd@commandprompt.com>
  1 sibling, 0 replies; 8+ messages in thread

From: Jan Wieck @ 2004-08-14 17:56 UTC (permalink / raw)
  To: Joshua D. Drake <jd@commandprompt.com>; +Cc: Chris Cheston <ccheston@gmail.com>; pgsql-performance

On 8/14/2004 12:22 PM, Joshua D. Drake wrote:

> I hope you understand that I, in no way have ever suggested (purposely) 
> anything negative about Slony. Only that I believe they serve different 
> technical solutions.

You know I never took anything you said negative. I think People here 
need to know that we two have communicated and collaborated outside of 
the public mailing lists, that we still do it and that we do not 
consider each other as opponents. There is a natural overlap in the 
systems we offer and therefore there is some competition. Software 
development is a sport. For some professionals this sport happens to be 
the main source of income. But that shall not spoil the spirit here.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-14 19:58  Christopher Browne <cbbrowne@acm.org>
  parent: Joshua D. Drake <jd@commandprompt.com>
  1 sibling, 1 reply; 8+ messages in thread

From: Christopher Browne @ 2004-08-14 19:58 UTC (permalink / raw)
  To: pgsql-performance

Centuries ago, Nostradamus foresaw when jd@commandprompt.com ("Joshua D. Drake") would write:
> I hope you understand that I, in no way have ever suggested
> (purposely) anything negative about Slony. Only that I believe they
> serve different technical solutions.

Stipulating that I may have some bias ;-), I still don't find it at
all clear what the different situations are "shaped like" that lead to
Mammoth being forcibly preferable to Slony-I.

(Note that I have a pretty decent understanding about how ERS and
Slony work, so I'm not too frightened of technicalities...  I set up
instances of both on Thursday, so I'm pretty up to speed :-).)

Win32 support may be true at the moment, although I have to discount
that as we only just got the start of a beta release of native Win32
support for PostgreSQL proper.  For that very reason, I had to point
my youngest brother who needed "something better than Access" to
Firebird last Saturday; I played with my niece while he was doing the
install.  And there is little reason to think that Slony-I won't be
portable to Win32 given a little interest and effort, particularly
once work to make it play well with "pgxs" gets done.
-- 
(format nil "~S@~S" "cbbrowne" "ntlug.org")
http://www3.sympatico.ca/cbbrowne/multiplexor.html
"At  Microsoft, it doesn't  matter which  file you're  compiling, only
which flags you #define."  -- Colin Plumb



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-15 00:55  Joshua D. Drake <jd@commandprompt.com>
  parent: Christopher Browne <cbbrowne@acm.org>
  0 siblings, 1 reply; 8+ messages in thread

From: Joshua D. Drake @ 2004-08-15 00:55 UTC (permalink / raw)
  To: Christopher Browne <cbbrowne@acm.org>; +Cc: pgsql-performance

Christopher Browne wrote:

>Centuries ago, Nostradamus foresaw when jd@commandprompt.com ("Joshua D. Drake") would write:
>  
>
>>I hope you understand that I, in no way have ever suggested
>>(purposely) anything negative about Slony. Only that I believe they
>>serve different technical solutions.
>>    
>>
>
>Stipulating that I may have some bias ;-), I still don't find it at
>all clear what the different situations are "shaped like" that lead to
>Mammoth being forcibly preferable to Slony-I.
>  
>
I would choose replicator if:

1. You want ease of setup
2. You want your each transaction to be replicated at time of commit
3. Your database is already laden with triggers
4. You are pushing a very high transactional load*

* Caveat I have no idea how well Slony performs on a system that does 
say 200,000 transactions
an hours that are heavily geared toward updates. Replicator performs 
very well in this scenario.

5. Replicators administrative tools are more mature than Slony (for 
example you know exactly what state your slaves are in with Replicator).

I would choose Slony if:

1. The fact that it is Open Source matters to you
2. The auto promotion of slaves is important*

*This will be fixed in a couple of weeks with Replicator

To be fair, in the real world ---

It doesn't make a bit of difference which one you choose it really comes 
down to this:

Replicator is dumb simple to setup. Any halfway talented person can 
setup replicator
in 30 minutes with a single master / slave configuration.

Slony is Open Source and thus a little easier on the pocket book initially.

Command Prompt, will support either one -- so the Replicator is 
commercially supported
argument is a little weak here.

Sincerely,

Joshua D. Drake




Sincerely,

Joshua D. Drake




>(Note that I have a pretty decent understanding about how ERS and
>Slony work, so I'm not too frightened of technicalities...  I set up
>instances of both on Thursday, so I'm pretty up to speed :-).)
>
>Win32 support may be true at the moment, although I have to discount
>that as we only just got the start of a beta release of native Win32
>support for PostgreSQL proper.  For that very reason, I had to point
>my youngest brother who needed "something better than Access" to
>Firebird last Saturday; I played with my niece while he was doing the
>install.  And there is little reason to think that Slony-I won't be
>portable to Win32 given a little interest and effort, particularly
>once work to make it play well with "pgxs" gets done.
>  
>


-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Replication: Slony-I vs. Mammoth Replicator vs. ?
@ 2004-08-16 15:12  Vivek Khera <khera@kcilink.com>
  parent: Joshua D. Drake <jd@commandprompt.com>
  0 siblings, 0 replies; 8+ messages in thread

From: Vivek Khera @ 2004-08-16 15:12 UTC (permalink / raw)
  To: pgsql-performance

One more point for your list:

Choose Slony if Replicator doesn't support your platform. :-)


-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D.                Khera Communications, Inc.
Internet: khera@kciLink.com       Rockville, MD  +1-301-869-4449 x806
AIM: vivekkhera Y!: vivek_khera   http://www.khera.org/~vivek/



^ permalink  raw  reply  [nested|flat] 8+ messages in thread


end of thread, other threads:[~2004-08-16 15:12 UTC | newest]

Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-08-14 01:27 Replication: Slony-I vs. Mammoth Replicator vs. ? Chris Cheston <ccheston@gmail.com>
2004-08-14 01:39 ` Joshua D. Drake <jd@commandprompt.com>
2004-08-14 15:33   ` Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 16:22     ` Joshua D. Drake <jd@commandprompt.com>
2004-08-14 17:56       ` Jan Wieck <JanWieck@Yahoo.com>
2004-08-14 19:58       ` Christopher Browne <cbbrowne@acm.org>
2004-08-15 00:55         ` Joshua D. Drake <jd@commandprompt.com>
2004-08-16 15:12           ` Vivek Khera <khera@kcilink.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox