Message-ID: <52EED0FC.2060503@fuzzy.cz>
Date: Mon, 03 Feb 2014 00:13:00 +0100
From: Tomas Vondra <tv@fuzzy.cz>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.1.1
MIME-Version: 1.0
To: pgsql-hackers@postgresql.org
Subject: Re: GIN improvements part2: fast scan
References: 
 <CAPpHfdvftaJq7www381naLw1=4u0h+qpXgWvNhcEB9HMVywbGg@mail.gmail.com>
 <CAPpHfduRrMOvXNrmJ0+ALk9dDR3Z34HpNdQPkQZtF_oYX58h9A@mail.gmail.com>
 <51D00FB6.9030008@vmware.com>
 <CAPpHfdtsDe-O6oQdCZUh6N1gGOMz0XKTMNQro-fueg-R06PaWw@mail.gmail.com>
 <528533D9.3040501@vmware.com>
 <CAPpHfduOvUFT+GgCrnBOzRR19eTb9mDsDJwU=S0h477fdgr=RA@mail.gmail.com>
 <CAPpHfdtZOhsmz_uf5BfdXcj5W3zk3K-PYxFc8-7eQw-nPXX1DA@mail.gmail.com>
 <CAPpHfdsEKFmrOU51EWSbF-RDESasSVa2YjoyFoG+0jCZOn1qjw@mail.gmail.com>
 <CAPpHfdvs-EHiwmTgfK=uJ4W3OjskVHyfkuehtyrKfKY_KmMpVw@mail.gmail.com>
 <52E141BA.5000302@vmware.com> <52E4141E.60609@fuzzy.cz>
 <52E42AD0.5060600@vmware.com> <52E4AA3A.9000607@fuzzy.cz>
 <52E53458.2010106@vmware.com>
 <CAPpHfdticvY2+RGTxzQdrr4YSQie+vYGRDRBTuG1ZxBmjwGrOA@mail.gmail.com>
 <CAPpHfduivz3MOA8_o+44n_Z0P-n+WCW=K1DrH+-g3gb-cbM2aA@mail.gmail.com>
 <52E72A10.5040908@fuzzy.cz> <52E75C54.9050209@vmware.c om>
 <52E9A274.6040806@fuzzy.c z> <52EE21D1.2080804@vmware.com>
In-Reply-To: <52EE21D1.2080804@vmware.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 2.2.2014 11:45, Heikki Linnakangas wrote:
> On 01/30/2014 01:53 AM, Tomas Vondra wrote:
>> (3) A file with explain plans for 4 queries suffering ~2x slowdown,
>>      and explain plans with 9.4 master and Heikki's patches is available
>>      here:
>>
>>        http://www.fuzzy.cz/tmp/gin/queries.txt
>>
>>      All the queries have 6 common words, and the explain plans look
>>      just fine to me - exactly like the plans for other queries.
>>
>>      Two things now caught my eye. First some of these queries actually
>>      have words repeated - either exactly like "database & database" or
>>      in negated form like "!anything & anything". Second, while
>>      generating the queries, I use "dumb" frequency, where only exact
>>      matches count. I.e. "write != written" etc. But the actual number
>>      of hits may be much higher - for example "write" matches exactly
>>      just 5% documents, but using @@ it matches more than 20%.
>>
>>      I don't know if that's the actual cause though.
> 
> Ok, here's another variant of these patches. Compared to git master, it
> does three things:
> 
> 1. It adds the concept of ternary consistent function internally, but no
> catalog changes. It's implemented by calling the regular boolean
> consistent function "both ways".
> 
> 2. Use a binary heap to get the "next" item from the entries in a scan.
> I'm pretty sure this makes sense, because arguably it makes the code
> more readable, and reduces the number of item pointer comparisons
> significantly for queries with a lot of entries.
> 
> 3. Only perform the pre-consistent check to try skipping entries, if we
> don't already have the next item from the entry loaded in the array.
> This is a tradeoff, you will lose some of the performance gain you might
> get from pre-consistent checks, but it also limits the performance loss
> you might get from doing useless pre-consistent checks.
> 
> So taken together, I would expect this patch to make some of the
> performance gains less impressive, but also limit the loss we saw with
> some of the other patches.
> 
> Tomas, could you run your test suite with this patch, please?

Sure, will do. Do I get it right that this should be applied instead of
the four patches you've posted earlier?

Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers