Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 0ECE19FB326 for ; Sat, 28 Oct 2006 09:03:16 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 35780-05 for ; Sat, 28 Oct 2006 09:03:13 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.188]) by postgresql.org (Postfix) with ESMTP id AACC79FA491 for ; Sat, 28 Oct 2006 09:03:12 -0300 (ADT) Received: by nf-out-0910.google.com with SMTP id g2so1395410nfe for ; Sat, 28 Oct 2006 05:03:11 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=G2zntiEepmvyS4zItx53wTb60SJU+iEmhAoh8470lJyst1tQsQz9qOA9Wj6/yQIZuvUrv8CKB1H/brDNatV4H1YKvoQWX04fJfUrz9vwQ2FHiDwDuOkAdZXSdb5aL4CBNyzR/FeenfkClHO3ZAkotRjMSPUxMv5x0SDnTbOxsqY= Received: by 10.82.136.4 with SMTP id j4mr131874bud; Sat, 28 Oct 2006 05:03:10 -0700 (PDT) Received: by 10.82.190.6 with HTTP; Sat, 28 Oct 2006 05:03:10 -0700 (PDT) Message-ID: Date: Sat, 28 Oct 2006 08:03:10 -0400 From: "Michael Artz" To: "Luke Lonergan" Subject: Re: Best COPY Performance Cc: "Worky Workerson" , "Merlin Moncure" , pgsql-performance@postgresql.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.199 tagged_above=0 required=5 tests=AWL, DNS_FROM_RFC_ABUSE, SPF_PASS X-Spam-Level: X-Archive-Number: 200610/594 X-Sequence-Number: 21723 > > Are you saying that I should be able to issue multiple COPY commands > > because my I/O wait is low? I was under the impression that I am I/O > > bound, so multiple simeoultaneous loads would have a detrimental > > effect ... > > The reason I asked how many CPUs was to make sense of the 12% usr CPU time > in the above. That means you are CPU bound and are fully using one CPU. So > you aren't being limited by the I/O in this case, it's the CPU. > > I agree with Merlin that you can speed things up by breaking the file up. > Alternately you can use the OSS Bizgres java loader, which lets you specify > the number of I/O threads with the "-n" option on a single file. Thanks, I'll try that on Monday. > The other thing to wonder about though is why you are so CPU bound at 5 > MB/s. What version of Postgres is this? I was wondering about that as well, and the only thing that I can think of is that its the PK btree index creation on the IP4. PG 8.1.3 x86_64. I installed it via a RH rpm for their "Web Services Beta", or something like that. I know I'm a bit behind the times, but getting stuff in (and out) of my isolated lab is a bit of a pain. I'll compile up a 8.2 beta as well and see how that works out.