Received: from magus.postgresql.org ([87.238.57.229]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TCEBV-0005Ub-F9 for pgsql-performance@postgresql.org; Thu, 13 Sep 2012 18:30:41 +0000 Received: from mx3.uci.cu ([200.55.140.180]) by magus.postgresql.org with smtp (Exim 4.72) (envelope-from ) id 1TCEBN-00062H-Ry for pgsql-performance@postgresql.org; Thu, 13 Sep 2012 18:30:41 +0000 Received: (qmail 26310 invoked by uid 507); 13 Sep 2012 18:30:22 -0000 Received: from 10.0.0.188 by ns3.uci.cu (envelope-from , uid 501) with qmail-scanner-2.01st (avp: 5.0.2.0. spamassassin: 3.0.6. perlscan: 2.01st. Clear:RC:1(10.0.0.188):. Processed in 1.123635 secs); 13 Sep 2012 18:30:22 -0000 Received: from unknown (HELO ucimail5.uci.cu) (10.0.0.188) by 0 with SMTP; 13 Sep 2012 18:30:20 -0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by ucimail5.uci.cu (Postfix) with ESMTP id A1A1122C004 for ; Thu, 13 Sep 2012 14:30:20 -0400 (CDT) X-Virus-Scanned: amavisd-new at uci.cu Received: from ucimail5.uci.cu ([127.0.0.1]) by localhost (ucimail5.uci.cu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fF9QYDuuTMWi for ; Thu, 13 Sep 2012 14:30:19 -0400 (CDT) Received: by ucimail5.uci.cu (Postfix, from userid 101) id 6489E22C003; Thu, 13 Sep 2012 14:30:19 -0400 (CDT) Received: from [10.56.18.124] (unknown [10.56.18.124]) (Authenticated sender: mlortiz@uci.cu) by ucimail5.uci.cu (Postfix) with ESMTPSA id 56C5922C002; Thu, 13 Sep 2012 14:30:19 -0400 (CDT) Message-ID: <50522638.3090008@uci.cu> Date: Thu, 13 Sep 2012 14:30:16 -0400 From: Marcos Ortiz User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120615 Thunderbird/13.0.1 MIME-Version: 1.0 To: Ross Reedstrom CC: pgsql-performance@postgresql.org Subject: Re: AppScale backend datastore (NoSQL again kind of) References: <20120913181113.GB26745@rice.edu> In-Reply-To: <20120913181113.GB26745@rice.edu> Content-Type: multipart/alternative; boundary="------------070303010508010106070700" X-Pg-Spam-Score: -2.4 (--) X-Archive-Number: 201209/60 X-Sequence-Number: 47806 This is a multi-part message in MIME format. --------------070303010508010106070700 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Regards, Ross. Dimitri Fontaine gave a excellent talk in the last PgCon about the migration of Fotolog from MySQL to PostgreSQL with amazing advices around this, so you can contact him for his advice. On 09/13/2012 02:11 PM, Ross Reedstrom wrote: > Hey PostgreSQL speed demons - > At work, we're considering an AppScale deployment (that's the Google App Engine > roll-your-own http://appscale.cs.ucsb.edu/). It supports multiple technologies > to back the datastore part of the platform (HBase, Hypertable, MySQL Cluster, > Cassandra, Voldemort, MongoDB, MemcacheDB, Redis). Oddly enough, in performance > tests, the MySQL Cluster seems like the general winner (their tests, we haven't > done any yet) > > So, my immediate thought was "How hard would it be to replace the MySQL > Cluster bit w/ PostgreSQL?" I'm thinking hot standby/streaming rep. May or may > not need a pooling solution in there as well (I need to look at the AppScale > abstraction code, it may already be doing the pooling/direction bit.) It depends of many factors: - size of the MySQL cluster - size of the involving data, etc For the pooling solution, I recommend you to see PgBouncer, it´s a great project widely used for this topic. > > Any thoughts from those with more experience using/building PostgreSQL clusters? > Replacing MySQL Cluster? Clearly they must be using a subset of functionality, > since they support so many different backend stores. I'll probably have to > set up an instance of all this, run some example apps, and see what's actually > stored to get a handle on it. The GAE api for the datastore is sort of a ORM, > w/ yet another query language, that seems to map to SQL better than to NoSQL, > in any case. There seems to be a fairly explicit exposure of a table==class > sort of mapping. > > Ross Best wishes -- Marcos Luis Ortíz Valmaseda *Data Engineer && Sr. System Administrator at UCI* about.me/marcosortiz My Blog Tumblr's blog @marcosluis2186 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS... CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION http://www.uci.cu http://www.facebook.com/universidad.uci http://www.flickr.com/photos/universidad_uci --------------070303010508010106070700 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Regards, Ross.
Dimitri Fontaine gave a excellent talk in the last PgCon about the migration of Fotolog from MySQL to
PostgreSQL with amazing advices around this, so you can contact him for his advice.

On 09/13/2012 02:11 PM, Ross Reedstrom wrote:
Hey PostgreSQL speed demons -
At work, we're considering an AppScale deployment (that's the Google App Engine
roll-your-own http://appscale.cs.ucsb.edu/). It supports multiple technologies
to back the datastore part of the platform (HBase, Hypertable, MySQL Cluster,
Cassandra, Voldemort, MongoDB, MemcacheDB, Redis). Oddly enough, in performance
tests, the MySQL Cluster seems like the general winner (their tests, we haven't
done any yet)

So, my immediate thought was "How hard would it be to replace the MySQL
Cluster bit w/ PostgreSQL?" I'm thinking hot standby/streaming rep. May or may
not need a pooling solution in there as well (I need to look at the AppScale
abstraction code, it may already be doing the pooling/direction bit.)
It depends of many factors:
- size of the MySQL cluster
- size of the involving data, etc

For the pooling solution, I recommend you to see PgBouncer, it´s a great
project widely used for this topic.

Any thoughts from those with more experience using/building PostgreSQL clusters?
Replacing MySQL Cluster? Clearly they must be using a subset of functionality,
since they support so many different backend stores. I'll probably have to
set up an instance of all this, run some example apps, and see what's actually
stored to get a handle on it. The GAE api for the datastore is sort of a ORM,
w/ yet another query language, that seems to map to SQL better than to NoSQL,
in any case. There seems to be a fairly explicit exposure of a table==class
sort of mapping.

Ross
Best wishes
--




--------------070303010508010106070700--