pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedReplication solutions high volume db
7+ messages / 4 participants
[nested] [flat]
* Replication solutions high volume db
@ 2005-03-29 23:34 David B <davidb999@gmail.com>
0 siblings, 3 replies; 7+ messages in thread
From: David B @ 2005-03-29 23:34 UTC (permalink / raw)
To: pgsql-admin
Folks,
I'm thinking of replication tools for high volume site.
Postgres on dual Opteron box. 1/2 GB of data.
Big overnight load of data.
Some inserts during the data but mostly selects.
I think I've narrowed it down to Slony and Mammoth.
Any pro's or con's on one vs the other?
There is the obvious price difference but thats managable if Mammoth
substancially better.
Some area where our environement might stretch a solution.
Some records can be large. Limitied to one table but they can be 40K+
of chraracters in a column.
High volume of inserts coming during nightly load (millions of rows).
These loads are coming in commited blocks of 100.
That cannot slow down the replication.
Our call center agents needs access to them asap.
Any locking issues on the master during the replication process?
Anything else I should be concerned about or things I need to consider?
Personal experiences you have had running either.
tia
David
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Replication solutions high volume db
@ 2005-03-29 23:50 Joshua D. Drake <jd@commandprompt.com>
parent: David B <davidb999@gmail.com>
2 siblings, 1 reply; 7+ messages in thread
From: Joshua D. Drake @ 2005-03-29 23:50 UTC (permalink / raw)
To: David B <davidb999@gmail.com>; +Cc: pgsql-admin
>
> I think I've narrowed it down to Slony and Mammoth.
> Any pro's or con's on one vs the other?
Slony is more complicated to setup.
Slony requires triggers and cron.
Slony is asynchronous
Slony is Open Source
If Slony breaks you can call Command Prompt
It used to be that Slony was timed replication. I do not know if that is
still the case.
Mammoth is about as easy as it can be to setup.
Mammoth does not require triggers or cron
Mammoth is asynchronous
Mammoth costs $$$
Mammoth is set to be optionally synchronous in June.
Mammoth replicates on a per comitted transaction basis.
Mammoth Replicator is PostgreSQL with integrated replication.
If Mammoth breaks you can call Command Prompt ;)
>
> Some area where our environement might stretch a solution.
> Some records can be large. Limitied to one table but they can be 40K+
> of chraracters in a column.
As long as that is not the primary key, there wouldn't be a problem.
The length of the primary key can not be greater than 8k with Mammoth.
>
> High volume of inserts coming during nightly load (millions of rows).
> These loads are coming in commited blocks of 100.
No problem.
>
> That cannot slow down the replication.
> Our call center agents needs access to them asap.
>
> Any locking issues on the master during the replication process?
Neither Mammoth or Slony require any exclusive locks to replicate the
data.
Sincerely,
Joshua D. Drake
> Anything else I should be concerned about or things I need to consider?
> Personal experiences you have had running either.
>
> tia
> David
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings
--
Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564
Custom programming, 24x7 support, managed services, and hosting
Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG
Reliable replication, Mammoth Replicator - http://www.commandprompt.com/
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Replication solutions high volume db
@ 2005-03-29 23:55 Scott Marlowe <smarlowe@g2switchworks.com>
parent: David B <davidb999@gmail.com>
2 siblings, 1 reply; 7+ messages in thread
From: Scott Marlowe @ 2005-03-29 23:55 UTC (permalink / raw)
To: David B <davidb999@gmail.com>; +Cc: pgsql-admin
On Tue, 2005-03-29 at 17:34, David B wrote:
> Folks,
>
> I'm thinking of replication tools for high volume site.
> Postgres on dual Opteron box. 1/2 GB of data.
>
> Big overnight load of data.
> Some inserts during the data but mostly selects.
>
> I think I've narrowed it down to Slony and Mammoth.
> Any pro's or con's on one vs the other?
>
> There is the obvious price difference but thats managable if Mammoth
> substancially better.
>
> Some area where our environement might stretch a solution.
> Some records can be large. Limitied to one table but they can be 40K+
> of chraracters in a column.
>
> High volume of inserts coming during nightly load (millions of rows).
> These loads are coming in commited blocks of 100.
>
> That cannot slow down the replication.
> Our call center agents needs access to them asap.
>
> Any locking issues on the master during the replication process?
>
>
> Anything else I should be concerned about or things I need to consider?
> Personal experiences you have had running either.
I don't see any obvious problems for either replication engine here.
Since slony is free and pretty easy to setup, I'd make a test setup and
see how it handles the load.
Generally speaking I'd expect Mammoth to handle high loads a little
better since it's a log shipper and not trigger based. So, if slony
proves too slow for this situation, then I'd recommend asking the folks
at Command Prompt for a demo of their system to see if it can.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Replication solutions high volume db
@ 2005-03-29 23:57 Roderick A. Anderson <raa@mailporter.net>
parent: David B <davidb999@gmail.com>
2 siblings, 0 replies; 7+ messages in thread
From: Roderick A. Anderson @ 2005-03-29 23:57 UTC (permalink / raw)
To: pgsql-admin
David B wrote:
> Folks,
>
> I'm thinking of replication tools for high volume site.
> Postgres on dual Opteron box. 1/2 GB of data.
Did you really mean 1/2 GB? Seems like a mighty small database.
Maybe 1/2 TB or if you were really wild and crazy 1/2 PB.
Sorry but I couldn't resist. I've funny fingered too many times myself.
Rod
--
---
[This E-mail scanned for viruses by Declude Virus]
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Replication solutions high volume db
@ 2005-03-30 00:12 Scott Marlowe <smarlowe@g2switchworks.com>
parent: Joshua D. Drake <jd@commandprompt.com>
0 siblings, 1 reply; 7+ messages in thread
From: Scott Marlowe @ 2005-03-30 00:12 UTC (permalink / raw)
To: jd@commandprompt.com; +Cc: David B <davidb999@gmail.com>; pgsql-admin
On Tue, 2005-03-29 at 17:50, Joshua D. Drake wrote:
> >
> > I think I've narrowed it down to Slony and Mammoth.
> > Any pro's or con's on one vs the other?
>
> Slony is more complicated to setup.
> Slony requires triggers and cron.
Triggers yes, cron no...
> It used to be that Slony was timed replication. I do not know if that is
> still the case.
Not the case now. I didn't use it "back in the day" (you know, last
year :)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Replication solutions high volume db
@ 2005-03-30 00:16 Joshua D. Drake <jd@commandprompt.com>
parent: Scott Marlowe <smarlowe@g2switchworks.com>
0 siblings, 0 replies; 7+ messages in thread
From: Joshua D. Drake @ 2005-03-30 00:16 UTC (permalink / raw)
To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: David B <davidb999@gmail.com>; pgsql-admin
On Tue, 2005-03-29 at 18:12 -0600, Scott Marlowe wrote:
> On Tue, 2005-03-29 at 17:50, Joshua D. Drake wrote:
> > >
> > > I think I've narrowed it down to Slony and Mammoth.
> > > Any pro's or con's on one vs the other?
> >
> > Slony is more complicated to setup.
> > Slony requires triggers and cron.
>
> Triggers yes, cron no...
Oh? Good to know.
>
> > It used to be that Slony was timed replication. I do not know if that is
> > still the case.
>
> Not the case now. I didn't use it "back in the day" (you know, last
> year :)
I know.. that is so 2004 :)
Sincerely,
Joshua D. Drake
>
--
Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564
Custom programming, 24x7 support, managed services, and hosting
Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG
Reliable replication, Mammoth Replicator - http://www.commandprompt.com/
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Replication solutions high volume db
@ 2005-03-30 00:16 Joshua D. Drake <jd@commandprompt.com>
parent: Scott Marlowe <smarlowe@g2switchworks.com>
0 siblings, 0 replies; 7+ messages in thread
From: Joshua D. Drake @ 2005-03-30 00:16 UTC (permalink / raw)
To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: David B <davidb999@gmail.com>; pgsql-admin
>
> Generally speaking I'd expect Mammoth to handle high loads a little
> better since it's a log shipper and not trigger based. So, if slony
> proves too slow for this situation, then I'd recommend asking the folks
> at Command Prompt for a demo of their system to see if it can.
http://www.commandprompt.com/download
Sincerely,
Joshua D. Drake
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
--
Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564
Custom programming, 24x7 support, managed services, and hosting
Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG
Reliable replication, Mammoth Replicator - http://www.commandprompt.com/
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2005-03-30 00:16 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-03-29 23:34 Replication solutions high volume db David B <davidb999@gmail.com>
2005-03-29 23:50 ` Joshua D. Drake <jd@commandprompt.com>
2005-03-30 00:12 ` Scott Marlowe <smarlowe@g2switchworks.com>
2005-03-30 00:16 ` Joshua D. Drake <jd@commandprompt.com>
2005-03-29 23:55 ` Scott Marlowe <smarlowe@g2switchworks.com>
2005-03-30 00:16 ` Joshua D. Drake <jd@commandprompt.com>
2005-03-29 23:57 ` Roderick A. Anderson <raa@mailporter.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox