DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=references:from:in-reply-to:mime-version:date:message-id:subject:to
	:cc:content-type;
	b=k+nO7yTLFBeEbzAalyJAUFzji4MjKbPJbMaf/3kVIgmRHmzXeUmmp+gqX4zlgtcJl4
	CrwoiC5cDLDZMq7qaiQoQPRptxRPdephwxdlvzncgtRCkwnVLzZ6QMKp+CKr1YyMoBTx
	6B82KALefCqS5ol8yScbhRT24pzjOZlWl3BXw=
References: <D5DE36B9-17CF-4A55-9119-CDC0BDC4E48D@gmail.com>
	<4DBC004C020000250003D099@gw.wicourts.gov>
From: Joel Reymont <joelr1@gmail.com>
In-Reply-To: <4DBC004C020000250003D099@gw.wicourts.gov>
Mime-Version: 1.0 (iPhone Mail 8H7)
Date: Sat, 30 Apr 2011 18:58:50 +0100
Message-ID: <4881277622113933088@unknownmsgid>
Subject: Re: stored proc and inserting hundreds of thousands of rows
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Cc: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Content-Type: text/plain; charset=ISO-8859-1

Calculating distance involves giving an array of 150 float8 to a pgsql
function, then calling a C function 2 million times (at the moment),
giving it two arrays of 150 float8.

Just calculating distance for 2 million rows and extracting the
distance takes less than a second. I think that includes sorting by
distance and sending 100 rows to the client.

Are you suggesting eliminating the physical linking and calculating
matching documents on the fly?

Is there a way to speed up my C function by giving it all the float
arrays, calling it once and having it return a set of matches? Would
this be faster than calling it from a select, once for each array?

Sent from my comfortable recliner

On 30/04/2011, at 18:28, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote:

> Joel Reymont <joelr1@gmail.com> wrote:
>
>> We have 2 million documents now and linking an ad to all of them
>> takes 5 minutes on my top-of-the-line SSD MacBook Pro.
>
> How long does it take to run just the SELECT part of the INSERT by
> itself?
>
> -Kevin