Received: from localhost (maia-4.hub.org [200.46.204.183]) by postgresql.org (Postfix) with ESMTP id F37C19FBB9E for ; Thu, 15 Feb 2007 09:39:21 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-new, port 10024) with ESMTP id 88464-01 for ; Thu, 15 Feb 2007 09:39:14 -0400 (AST) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.4 Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.175]) by postgresql.org (Postfix) with ESMTP id CDBFD9FBB85 for ; Thu, 15 Feb 2007 09:39:12 -0400 (AST) Received: by ug-out-1314.google.com with SMTP id k3so136898ugf for ; Thu, 15 Feb 2007 05:39:11 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=DiY085cYYvVlx3977/C78jizo3g1bc0TOotxRwk1OpBmBtlZj8DTlIri4Q5kYgTwJdQzUK+gFwE9zy6Oy72XQPdkIQmsMcCsavjAZqyHZDBGcItzrUi4fbrJPXiosYlRbNI8kwbL0uTb3rf6WuLQstBb0IjTOoYAOBreTHN+eJI= Received: by 10.78.150.7 with SMTP id x7mr432408hud.1171546750879; Thu, 15 Feb 2007 05:39:10 -0800 (PST) Received: by 10.78.35.18 with HTTP; Thu, 15 Feb 2007 05:39:10 -0800 (PST) Message-ID: Date: Thu, 15 Feb 2007 08:39:10 -0500 From: "Merlin Moncure" To: "Luke Lonergan" Subject: Re: Benchmarking PGSQL? Cc: "Krishna Kumar" , pgsql-performance@postgresql.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200702/166 X-Sequence-Number: 23220 On 2/14/07, Luke Lonergan wrote: > Hi Merlin, > > On 2/14/07 8:20 AM, "Merlin Moncure" wrote: > > > I am curious what is your take on the maximum insert performance, in > > mb/sec of large bytea columns (toasted), and how much if any greenplum > > was able to advance this over the baseline. I am asking on behalf of > > another interested party. Interested in numbers broken down per core > > on 8 core quad system and also aggreate. > > Our approach is to attach a segment to each core, so we scale INSERT > linearly on number of cores. So the per core limit we live with is the > 10-20MB/s observed here. We'd like to improve that so that we get better > performance with smaller machines. > > We have demonstrated insert performance of 670 MB/s, 2.4TB/hour for > non-toasted columns using 3 load machines against 120 cores. This rate was > load machine limited. > > WRT toasted bytea columns we haven't done any real benchmarking of those. > Do you have a canned benchmark we can run? Interested in how fast you can insert binary objects (images, files, etc). into the database as a file storage system. Ultimately the insertions would all be done via libpq ExecPrepared/Params. A simple benchmark such as insert a 1mb object via pg_bench over 20 or so connections would be fine. Mostly interested in raw throughput per core and especially interested if you can beat stock pg on the same hardware. merlin