Received: from magus.postgresql.org ([87.238.57.229]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TCHhN-0007Nw-Eu for pgsql-performance@postgresql.org; Thu, 13 Sep 2012 22:15:49 +0000 Received: from [2404:130:0:10::40:2] (helo=mail.catalyst.net.nz) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TCHhJ-0001B2-Gt for pgsql-performance@postgresql.org; Thu, 13 Sep 2012 22:15:48 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.catalyst.net.nz (Postfix) with ESMTP id 771BD33376; Fri, 14 Sep 2012 10:15:39 +1200 (NZST) X-Virus-Scanned: Debian amavisd-new at catalyst.net.nz Received: from mail.catalyst.net.nz ([127.0.0.1]) by localhost (bertrand.catalyst.net.nz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EDTSnoOu1awn; Fri, 14 Sep 2012 10:15:37 +1200 (NZST) Received: from [IPv6:2404:130:0:1000:44ec:8da7:9e71:4c6] (unknown [IPv6:2404:130:0:1000:44ec:8da7:9e71:4c6]) (Authenticated sender: mark.kirkwood) by mail.catalyst.net.nz (Postfix) with ESMTPSA id 9ACA8323C9; Fri, 14 Sep 2012 10:15:37 +1200 (NZST) Message-ID: <50525B09.9010107@catalyst.net.nz> Date: Fri, 14 Sep 2012 10:15:37 +1200 From: Mark Kirkwood User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Ross Reedstrom CC: pgsql-performance@postgresql.org Subject: Re: AppScale backend datastore (NoSQL again kind of) References: <20120913181113.GB26745@rice.edu> In-Reply-To: <20120913181113.GB26745@rice.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Host-Lookup-Failed: Reverse DNS lookup failed for 2404:130:0:10::40:2 (failed) X-Pg-Spam-Score: -1.1 (-) X-Archive-Number: 201209/62 X-Sequence-Number: 47808 On 14/09/12 06:11, Ross Reedstrom wrote: > Hey PostgreSQL speed demons - > At work, we're considering an AppScale deployment (that's the Google App Engine > roll-your-own http://appscale.cs.ucsb.edu/). It supports multiple technologies > to back the datastore part of the platform (HBase, Hypertable, MySQL Cluster, > Cassandra, Voldemort, MongoDB, MemcacheDB, Redis). Oddly enough, in performance > tests, the MySQL Cluster seems like the general winner (their tests, we haven't > done any yet) > > So, my immediate thought was "How hard would it be to replace the MySQL > Cluster bit w/ PostgreSQL?" I'm thinking hot standby/streaming rep. May or may > not need a pooling solution in there as well (I need to look at the AppScale > abstraction code, it may already be doing the pooling/direction bit.) > > Any thoughts from those with more experience using/building PostgreSQL clusters? > Replacing MySQL Cluster? Clearly they must be using a subset of functionality, > since they support so many different backend stores. I'll probably have to > set up an instance of all this, run some example apps, and see what's actually > stored to get a handle on it. The GAE api for the datastore is sort of a ORM, > w/ yet another query language, that seems to map to SQL better than to NoSQL, > in any case. There seems to be a fairly explicit exposure of a table==class > sort of mapping. > Postgres-xc might be a good option to consider too. Regards Mark