pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: William Yu <wyu@talisys.com>
To: Greg Stark <gsstark@mit.edu>
Subject: Re: PostgreSQL clustering VS MySQL clustering
Date: Sat, 29 Jan 2005 00:22:14 -0800
Message-ID: <41FB47B6.3010507@talisys.com> (raw)
In-Reply-To: <874qh0u1yx.fsf@stark.xeocode.com>
References: <200501201503.31241.herve@elma.fr>
	<200501201542.06645.herve@elma.fr>
	<41EFC569.6030800@familyhealth.com.au>
	<200501201602.39568.herve@elma.fr>
	<ctect8$et4$1@news.hub.org>
	<874qh0u1yx.fsf@stark.xeocode.com>

>>I know what I would choose. I'd get the mega server w/ a ton of RAM and skip
>>all the trickyness of partitioning a DB over multiple servers. Yes your data
>>will grow to a point where even the XXGB can't cache everything. On the
>>otherhand, memory prices drop just as fast. By that time, you can ebay your
>>original 16/32GB and get 64/128GB.
> 
> 
> a) What do you do when your calculations show you need 256G of ram? [Yes such
> machines exist but you're not longer in the realm of simply "add more RAM".
> Administering such machines is nigh as complex as clustering]

If you need that much memory, you've got enough customers paying you 
cash to pay for anything. :) Technology always increase -- 8X Opterons 
would double your memory capacity, higher capacity DIMMs, etc.

> b) What do you do when you find you need multiple machines anyways to divide
> the CPU or I/O or network load up. Now you need n big beefy servers when n
> servers 1/nth as large would really have sufficed. This is a big difference
> when you're talking about the difference between colocating 16 1U boxen with
> 4G of ram vs 16 4U opterons with 64G of RAM...
> 
> All that said, yes, speaking as a user I think the path of least resistance is
> to build n complete slaves using Slony and then just divide the workload.
> That's how I'm picturing going when I get to that point.

Replication is good for uptime and high read systems. The problem is 
that if your system has a high volume of writes and you need near 
realtime data syncing, clusters don't get you anything. A write on one 
server means a write on every server. Spreading out the damage over 
multiple machines doesn't help a bit.

Plus the fact that we don't have multi-master replication yet is quite a 
bugaboo. That requires writing quite extensive code if you can't afford 
to have 1 server be your single point of failure. We wrote our own 
multi-master replication code at the client app level and it's quite a 
chore making sure the replication act logically. Every table needs to 
have separate logic to parse situations like "voucher was posted on 
server 1 but voided after on server 2, what's the correct action here?" 
So I've got a slew of complicated if-then-else statements that not only 
have to take into account type of update being made but the sequence.

And yes, I tried doing realtime locks over a VPN link over our servers 
in SF and VA. Ugh...latency was absolutely horrible and made 
transactions run 1000X slower.



view thread (131+ messages)  latest in thread

Message-ID: <41FB47B6.3010507@talisys.com>
Permalink:  ../41FB47B6.3010507@talisys.com/
Also on:    postgresql.org/message-id/41FB47B6.3010507@talisys.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: wyu@talisys.com, gsstark@mit.edu
  Subject: Re: PostgreSQL clustering VS MySQL clustering
  In-Reply-To: <41FB47B6.3010507@talisys.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox