X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 1046253707 for ; Tue, 3 May 2005 16:17:22 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 69202-05 for ; Tue, 3 May 2005 19:17:19 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.197]) by svr1.postgresql.org (Postfix) with ESMTP id 8B866533C3 for ; Tue, 3 May 2005 16:17:18 -0300 (ADT) Received: by rproxy.gmail.com with SMTP id a41so22126rng for ; Tue, 03 May 2005 12:17:20 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=F8XBdedxne+r66x4oZA5EfpGNztvqBnRrb+Fg2oE4oEeyX5LlZpefGxN8ordDBGUEIl667Z2rBsEpAgIHSuxop8/Iml5LuSpHmk9nDivkZunhB5GvasQXkW3GwVEGUYKCerFeelxO1mdfjXuH1xgjnPJFrOWA2hUh8c/PTTg1io= Received: by 10.38.72.11 with SMTP id u11mr127059rna; Tue, 03 May 2005 12:17:20 -0700 (PDT) Received: by 10.38.207.41 with HTTP; Tue, 3 May 2005 12:17:20 -0700 (PDT) Message-ID: <59d991c4050503121725619c6d@mail.gmail.com> Date: Tue, 3 May 2005 15:17:20 -0400 From: Christopher Petrilli Reply-To: Christopher Petrilli To: tim@se.linux.org Subject: Re: batch inserts are "slow" Cc: pgsql-performance@postgresql.org In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <59d991c405050208174b6ebbc@mail.gmail.com> X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.318 tagged_above=0 required=5 tests=AWL, DNS_FROM_RFC_ABUSE, RCVD_BY_IP X-Spam-Level: X-Archive-Number: 200505/25 X-Sequence-Number: 12189 On 5/3/05, Tim Terleg=E5rd wrote: > > Just as on Oracle you would use SQL*Loader for this application, you > > should use the COPY syntax for PostgreSQL. You will find it a lot > > faster. I have used it by building the input files and executing > > 'psql' with a COPY command, and also by using it with a subprocess, > > both are quite effective. >=20 > I tried this now. Now it's down to 0.45 seconds. It feels a bit hacky to > run /usr/bin/psql from java, but it sure works. Thanks for the hint! It may feel hacky, but I think if you want to use SQL*Loader on Oracle, you have to do the same thing. I know a C++ app that I use that runs SQL*Loader about once per second to deal with a HUGE volume (10K/sec). In fact, moving the load files onto ramdisk has helped a lot. Chris --=20 | Christopher Petrilli | petrilli@gmail.com