From: Jim C. Nasby <decibel@decibel.org>
To: David Roussel <pgsql-performance@diroussel.xsmail.com>
Cc: Alex Stapleton <alexs@advfn.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Partitioning / Clustering
Date: Wed, 11 May 2005 16:28:19 -0500
Message-ID: <20050511212819.GY31103@decibel.org> (raw)
In-Reply-To: <1115798277.29223.233869472@webmail.messagingengine.com>
References: <5F94B6EB-3925-4CD3-A67F-4310890B438A@advfn.com>
<1115798277.29223.233869472@webmail.messagingengine.com>
On Wed, May 11, 2005 at 08:57:57AM +0100, David Roussel wrote:
> For an interesting look at scalability, clustering, caching, etc for a
> large site have a look at how livejournal did it.
> http://www.danga.com/words/2004_lisa/lisa04.pdf
>
> They have 2.6 Million active users, posting 200 new blog entries per
> minute, plus many comments and countless page views.
Neither of which is that horribly impressive. 200 TPM is less than 4TPS.
While I haven't run high transaction rate databases under PostgreSQL, I
suspect others who have will say that 4TPS isn't that big of a deal.
> Although this system is of a different sort to the type I work on it's
> interesting to see how they've made it scale.
>
> They use mysql on dell hardware! And found single master replication did
> not scale. There's a section on multimaster replication, not sure if
Probably didn't scale because they used to use MyISAM.
> they use it. The main approach they use is to parition users into
> spefic database clusters. Caching is done using memcached at the
Which means they've got a huge amount of additional code complexity, not
to mention how many times you can't post something because 'that cluster
is down for maintenance'.
> application level to avoid hitting the db for rendered pageviews.
Memcached is about the only good thing I've seen come out of
livejournal.
> It's interesting that the solution livejournal have arrived at is quite
> similar in ways to the way google is set up.
Except that unlike LJ, google stays up and it's fast. Though granted, LJ
is quite a bit faster than it was 6 months ago.
--
Jim C. Nasby, Database Consultant decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828
Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: decibel@decibel.org, pgsql-performance@diroussel.xsmail.com, alexs@advfn.com
Subject: Re: Partitioning / Clustering
In-Reply-To: <20050511212819.GY31103@decibel.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox