Message-ID: <453D443B.20302@modgraph-usa.com>
Date: Mon, 23 Oct 2006 15:37:47 -0700
From: "Craig A. James" <cjames@modgraph-usa.com>
User-Agent: Thunderbird 1.5.0.7 (X11/20060909)
MIME-Version: 1.0
To: "Jim C. Nasby" <jim@nasby.net>
CC: Worky Workerson <worky.workerson@gmail.com>,
	Merlin Moncure <mmoncure@gmail.com>, pgsql-performance@postgresql.org
Subject: Re: Best COPY Performance
References: <ce4072df0610201305u7836dc7bn437ec0aaf1c3e25f@mail.gmail.com>
	<b42b73150610201811m84369f7kc7d58b39aed3ad73@mail.gmail.com>
	<ce4072df0610230810t5d66bbb8xf1e9c0ed71b6b6c0@mail.gmail.com>
	<20061023215913.GW26892@nasby.net>
In-Reply-To: <20061023215913.GW26892@nasby.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Jim C. Nasby wrote:
> http://stats.distributed.net used to use a perl script to do some
> transformations before loading data into the database. IIRC, when we
> switched to using C we saw 100x improvement in speed, so I suspect that
> if you want performance perl isn't the way to go. I think you can
> compile perl into C, so maybe that would help some.

I use Perl extensively, and have never seen a performance problem.  I suspect the perl-to-C "100x improvement" was due to some other factor, like a slight change in the schema, indexes, or the fundamental way the client (C vs Perl) handled the data during the transformation, or just plain bad Perl code.

Modern scripting languages like Perl and Python make programmers far, far more productive than the bad old days of C/C++.  Don't shoot yourself in the foot by reverting to low-level languages like C/C++ until you've exhausted all other possibilities.  I only use C/C++ for intricate scientific algorithms.

In many cases, Perl is *faster* than C/C++ code that I write, because I can't take the time (for example) to write the high-performance string manipulation that have been fine-tuned and extensively optimized in Perl.

Craig