X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id AD62A5370A for ; Tue, 5 Apr 2005 05:16:33 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 97358-07 for ; Tue, 5 Apr 2005 04:16:28 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.206]) by svr1.postgresql.org (Postfix) with ESMTP id 0C72F53619 for ; Tue, 5 Apr 2005 05:16:27 +0100 (BST) Received: by rproxy.gmail.com with SMTP id a41so1375825rng for ; Mon, 04 Apr 2005 21:16:27 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=qDZQgDjFMoJXF2XUmzoMm+b95ShersCf5cBESFlRXjfqFQ+Eb4kP9ERA4SL+o9ZmPgiTXtJh9cT3Ehmsahac+8qqgD40Rt8fJAZyQ8vyZSQ4E7qFNVa45TIE85UWZ5EAuw1Ry2MrOOlf6WKJZiG4ZI6n3z5KjuOc96pi1SwKmcM= Received: by 10.38.86.68 with SMTP id j68mr6030020rnb; Mon, 04 Apr 2005 21:16:27 -0700 (PDT) Received: by 10.38.207.41 with HTTP; Mon, 4 Apr 2005 21:16:27 -0700 (PDT) Message-ID: <59d991c405040421164aeea3fb@mail.gmail.com> Date: Tue, 5 Apr 2005 00:16:27 -0400 From: Christopher Petrilli Reply-To: Christopher Petrilli To: Tom Lane Subject: Re: Sustained inserts per sec ... ? Cc: Greg Stark , jd@commandprompt.com, "Marc G. Fournier" , pgsql-performance@postgresql.org In-Reply-To: <7438.1112673460@sss.pgh.pa.us> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit References: <20050401140421.P18194@ganymede.hub.org> <59d991c4050401125956d6069d@mail.gmail.com> <59d991c40504040648457808e9@mail.gmail.com> <26454.1112629979@sss.pgh.pa.us> <59d991c405040409094e7bbd17@mail.gmail.com> <26669.1112631807@sss.pgh.pa.us> <59d991c40504041846bc39a6@mail.gmail.com> <5217.1112668589@sss.pgh.pa.us> <87y8bxophw.fsf@stark.xeocode.com> <7438.1112673460@sss.pgh.pa.us> X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.315 tagged_above=0 required=5 tests=AWL, DNS_FROM_RFC_ABUSE, RCVD_BY_IP X-Spam-Level: X-Archive-Number: 200504/83 X-Sequence-Number: 11480 On Apr 4, 2005 11:57 PM, Tom Lane wrote: > Greg Stark writes: > > All this is happening within a single transaction too, right? So there hasn't > > been an fsync the entire time. It's entirely up to the kernel when to decide > > to start writing data. > > No ... there's a commit every 500 records. However, I think Chris said > he was running with fsync off; so you're right that the kernel is at > liberty to write stuff to disk when it feels like. It could be that > those outlier points are transactions that occurred in the middle of > periodic syncer-driven mass writes. Maybe fsync off is > counterproductive for this situation? Looking at preliminary results from running with shared_buffers at 16000, it seems this may be correct. Performance was flatter for a BIT longer, but slammed right into the wall and started hitting the 3-30 second range per COPY. I've restarted the run, with fsync turned on (fdatasync), and we'll see. My fear is that it's some bizarre situation interacting with both issues, and one that might not be solvable. Does anyone else have much experience with this sort of sustained COPY? Chris -- | Christopher Petrilli | petrilli@gmail.com