X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id CF37ED8C80 for ; Mon, 12 Sep 2005 21:56:41 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 71287-08 for ; Tue, 13 Sep 2005 00:56:37 +0000 (GMT) Received: from unicorn.rentec.com (unicorn.rentec.com [216.223.240.9]) by svr1.postgresql.org (Postfix) with ESMTP id 46030D8C52 for ; Mon, 12 Sep 2005 21:56:35 -0300 (ADT) Received: from ram.rentec.com (mailhost [192.5.35.66]) by unicorn.rentec.com (8.13.1/8.12.1) with ESMTP id j8D0uGIl015500 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 12 Sep 2005 20:56:18 -0400 (EDT) X-Source: non-mednet Received: from [172.16.160.108] (stange-dhcp2.rentec.com [172.16.160.108]) by ram.rentec.com (8.13.1/8.12.1) with ESMTP id j8D0uEiX002501; Mon, 12 Sep 2005 20:56:15 -0400 (EDT) Message-ID: <43261F8A.1060506@rentec.com> Date: Mon, 12 Sep 2005 20:38:34 -0400 From: Alan Stange Organization: Renaissance Technologies Corp. User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050317) X-Accept-Language: en-us, en MIME-Version: 1.0 To: blblack@gmail.com Cc: PFC , pgsql-performance@postgresql.org Subject: Re: Performance considerations for very heavy INSERT traffic References: <84621a6050912140464c380c3@mail.gmail.com> <84621a60509121502603bd56c@mail.gmail.com> In-Reply-To: <84621a60509121502603bd56c@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Logged: Logged by unicorn.rentec.com as j8D0uGIl015500 at Mon Sep 12 20:56:18 2005 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.015 required=5 tests=[AWL=0.015] X-Spam-Level: X-Archive-Number: 200509/151 X-Sequence-Number: 14459 Brandon Black wrote: > > > On 9/12/05, *PFC* > wrote: > > > > - benchmarking something else than ext3 > (xfs ? reiser3 ?) > > > We've had bad experiences under extreme and/or strange workloads with > XFS here in general, although this is the first major postgresql > project - the rest were with other applications writing to XFS. Bad > experiences like XFS filesystems "detecting internal inconsistencies" > at runtime and unmounting themselves from within the kernel module > (much to the dismay of applications with open files on the > filesystem), on machines with validated good hardware. It has made me > leary of using anything other than ext3 for fear of stability > problems. Reiser3 might be worth taking a look at though. Just one tidbit. We tried XFS on a very active system similar to what you describe. Dual opterons, 8GB memory, fiber channel drives, 2.6 kernel, etc. And the reliability was awful. We spent a lot of time making changes one at a time to try and isolate the cause; when we switched out from XFS to ReiserFS our stability problems went away. It may be the case that the XFS problems have all been corrected in newer kernels, but I'm not going to put too much effort into trying that again. I recently built a postgres with 32KB block sizes and have been doing some testing. For our particular workloads it has been a win. -- Alan