Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id 8C947133616B for ; Thu, 14 Apr 2011 17:05:42 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 38458-01 for ; Thu, 14 Apr 2011 20:05:35 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from EXHUB018-2.exch018.msoutlookonline.net (exhub018-2.exch018.msoutlookonline.net [64.78.17.17]) by mail.postgresql.org (Postfix) with ESMTP id 9FC941337BD7 for ; Thu, 14 Apr 2011 17:05:34 -0300 (ADT) Received: from EXVMBX018-1.exch018.msoutlookonline.net ([64.78.17.47]) by EXHUB018-2.exch018.msoutlookonline.net ([64.78.17.17]) with mapi; Thu, 14 Apr 2011 13:05:33 -0700 From: Scott Carey To: Greg Smith CC: Glyn Astill , Kevin Grittner , "david@lang.hm" , Steve Clark , "Joshua D. Drake" , Scott Marlowe , "pgsql-performance@postgresql.org" Date: Thu, 14 Apr 2011 13:05:36 -0700 Subject: Re: Linux: more cores = less concurrency. Thread-Topic: [PERFORM] Linux: more cores = less concurrency. Thread-Index: Acv630/kLRDXqetZQQaXEeOXIDU4aQ== Message-ID: In-Reply-To: <4DA676BB.4050501@2ndquadrant.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.10.0.110310 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-10 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Archive-Number: 201104/216 X-Sequence-Number: 43278 On 4/13/11 9:23 PM, "Greg Smith" wrote: >Scott Carey wrote: >> If postgres is memory bandwidth constrained, what can be done to reduce >> its bandwidth use? >> >> Huge Pages could help some, by reducing page table lookups and making >> overall access more efficient. >> Compressed pages (speedy / lzo) in memory can help trade CPU cycles for >> memory usage for certain memory segments/pages -- this could potentially >> save a lot of I/O too if more pages fit in RAM as a result, and also >>make >> caches more effective. >> =20 > >The problem with a lot of these ideas is that they trade the memory >problem for increased disruption to the CPU L1 and L2 caches. I don't >know how much that moves the bottleneck forward. And not every workload >is memory constrained, either, so those that aren't might suffer from >the same optimizations that help in this situation. Compression has this problem, but I'm not sure where the plural "a lot of these ideas" comes from. Huge Pages helps caches. Dual-Pivot quicksort is more cache friendly and is _always_ equal to or faster than traditional quicksort (its a provably improved algorithm). Smaller hash tables help caches. > >I just posted my slides from my MySQL conference talk today at >http://projects.2ndquadrant.com/talks , and those include some graphs of >recent data collected with stream-scaling. The current situation is >really strange in both Intel and AMD's memory architectures. I'm even >seeing situations where lightly loaded big servers are actually >outperformed by small ones running the same workload. The 32 and 48 >core systems using server-class DDR3/1333 just don't have the bandwidth >to a single core that, say, an i7 desktop using triple-channel DDR3-1600 >does. The trade-offs here are extremely hardware and workload >dependent, and it's very easy to tune for one combination while slowing >another. > >--=20 >Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD >PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us >"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books >