Received: from localhost (unknown [200.46.204.183]) by mail.postgresql.org (Postfix) with ESMTP id 9CEC164FE31 for ; Mon, 8 Dec 2008 16:15:47 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 11286-01 for ; Mon, 8 Dec 2008 16:15:42 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from westnet.com (westnet.com [216.187.52.2]) by mail.postgresql.org (Postfix) with ESMTP id 0E88C64FD5F for ; Mon, 8 Dec 2008 16:15:41 -0400 (AST) Received: from westnet.com (localhost [127.0.0.1]) by westnet.com (8.14.0/8.14.0) with ESMTP id mB8KFafT017620; Mon, 8 Dec 2008 15:15:36 -0500 (EST) Received: from localhost (gsmith@localhost) by westnet.com (8.14.0/8.13.2/Submit) with ESMTP id mB8KFaRh017607; Mon, 8 Dec 2008 15:15:36 -0500 (EST) X-Authentication-Warning: westnet.com: gsmith owned process doing -bs Date: Mon, 8 Dec 2008 15:15:36 -0500 (EST) From: Greg Smith X-X-Sender: gsmith@westnet.com To: Scott Marlowe cc: Josh Berkus , pgsql-performance@postgresql.org Subject: Re: Need help with 8.4 Performance Testing In-Reply-To: Message-ID: References: <200812071138.01215.josh@agliodbs.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests=none X-Spam-Level: X-Archive-Number: 200812/58 X-Sequence-Number: 32094 On Sun, 7 Dec 2008, Scott Marlowe wrote: > When I last used pgbench I wanted to test it with an extremely large > dataset, but it maxes out at -s 4xxx or so, and that's only in the > 40Gigabyte range. Is the limit raised for the pgbench included in > contrib in 8.4? I'm guessing it's an arbitrary limit. There's no artificial limit, just ones that result from things like integer overflow. I don't think has been an issue so far because pgbench becomes seek limited and stops producing interesting results once the database exceeds the sum of all available caching, which means you'd need more than 32GB of RAM in the system running pgbench before this is an issue. Which happens to be the largest size system I've ever ran it on... I'd expect this statement around line 1060 of pgbench.c to overflow first: for (i = 0; i < naccounts * scale; i++) Where i is an integer, naccounts=100,000 , and scale is what you input. That will overflow a signed 32-bit integer at a scale of 2147. If you had tests that failed at twice that, I wonder if you were actually executing against a weird data set for scales of (2148..4294). It's not completely trivial to fix (I just tried), the whole loop needs to be restructured to run against scale and 100,000 separately while keeping the current progress report intact. I'll take a shot at fixing it correctly, this is a bug that should be corrected before 8.4 goes out. I guarantee that version will be used on systems with 64GB of RAM where this matters. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD