From: Heikki Linnakangas <heikki@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Jan Urbański <j.urbanski@students.mimuw.edu.pl>
Cc: Postgres - Hackers <pgsql-hackers@postgresql.org>
Subject: Re: gsoc, text search selectivity and dllist enhancments
Date: Fri, 04 Jul 2008 22:20:08 +0300
Message-ID: <486E77E8.6010404@enterprisedb.com> (raw)
In-Reply-To: <23365.1215186836@sss.pgh.pa.us>
References: <48649261.5040703@students.mimuw.edu.pl>
<6233.1215113144@sss.pgh.pa.us>
<486DD210.9030400@enterprisedb.com>
<23365.1215186836@sss.pgh.pa.us>
Tom Lane wrote:
> "Heikki Linnakangas" <heikki@enterprisedb.com> writes:
>> Tom Lane wrote:
>>> The data structure I'd suggest is a simple array of pointers
>>> to the underlying hash table entries. Since you have a predetermined
>>> maximum number of lexemes to track, you can just palloc the array once
>>> --- you don't need the expansibility properties of a list.
>
>> The number of lexemes isn't predetermined. It's 2 * (longest tsvector
>> seen so far), and we don't know beforehand how long the longest tsvector is.
>
> Hmm, I had just assumed without looking too closely that it was stats
> target times a fudge factor. What is the rationale for doing it as
> above? I don't think I like the idea of the limit varying over the
> course of the scan --- that means that lexemes in different places
> in the input will have significantly different probabilities of
> surviving to the final result.
Well, clearly if the list is smaller than the longest tsvector,
inserting all elements of that long tsvector will flush out all other
entries. Or if we throw away the newly inserted entries, some elements
will never have a chance to climb up the list. I'm not sure where the
"times two" figure comes from, maybe it's just a fudge factor, but the
bottom line is that the minimum size needed depends on the size of the
longest tsvector.
(Jan is offline until Saturday...)
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: heikki@enterprisedb.com, tgl@sss.pgh.pa.us, j.urbanski@students.mimuw.edu.pl
Subject: Re: gsoc, text search selectivity and dllist enhancments
In-Reply-To: <486E77E8.6010404@enterprisedb.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox