From: David Roussel <pgsql-performance@diroussel.xsmail.com>
To: Alex Stapleton <alexs@advfn.com>
To: pgsql-performance@postgresql.org
Subject: Re: Partitioning / Clustering
Date: Wed, 11 May 2005 08:57:57 +0100
Message-ID: <1115798277.29223.233869472@webmail.messagingengine.com> (raw)
In-Reply-To: <5F94B6EB-3925-4CD3-A67F-4310890B438A@advfn.com>
References: <5F94B6EB-3925-4CD3-A67F-4310890B438A@advfn.com>
For an interesting look at scalability, clustering, caching, etc for a
large site have a look at how livejournal did it.
http://www.danga.com/words/2004_lisa/lisa04.pdf
They have 2.6 Million active users, posting 200 new blog entries per
minute, plus many comments and countless page views.
Although this system is of a different sort to the type I work on it's
interesting to see how they've made it scale.
They use mysql on dell hardware! And found single master replication did
not scale. There's a section on multimaster replication, not sure if
they use it. The main approach they use is to parition users into
spefic database clusters. Caching is done using memcached at the
application level to avoid hitting the db for rendered pageviews.
It's interesting that the solution livejournal have arrived at is quite
similar in ways to the way google is set up.
David
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: pgsql-performance@diroussel.xsmail.com, alexs@advfn.com
Subject: Re: Partitioning / Clustering
In-Reply-To: <1115798277.29223.233869472@webmail.messagingengine.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox