Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WAK3o-0002du-BU for pgsql-hackers@arkaria.postgresql.org; Mon, 03 Feb 2014 13:59:40 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1WAK3n-0003zj-Na for pgsql-hackers@arkaria.postgresql.org; Mon, 03 Feb 2014 13:59:39 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WAK3m-0003za-KB for pgsql-hackers@postgresql.org; Mon, 03 Feb 2014 13:59:38 +0000 Received: from 2605ds1-ynoe.3.fullrate.dk ([90.185.142.83] helo=shrek.krogh.cc) by magus.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WAK3j-0007YI-37 for pgsql-hackers@postgresql.org; Mon, 03 Feb 2014 13:59:38 +0000 Received: from localhost (localhost [127.0.0.1]) by shrek.krogh.cc (Postfix) with ESMTP id 9F15E116ACD; Mon, 3 Feb 2014 14:59:33 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at shrek.krogh.cc Received: from shrek.krogh.cc ([127.0.0.1]) by localhost (shrek.krogh.cc [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id spNvJl35Jdd5; Mon, 3 Feb 2014 14:59:30 +0100 (CET) Received: from [90.185.142.91] (2605ds1-ynoe.4.fullrate.dk [90.185.142.91]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: jesper@krogh.cc) by shrek.krogh.cc (Postfix) with ESMTPSA id D84F3116ACA; Mon, 3 Feb 2014 14:59:29 +0100 (CET) Message-ID: <52EFA0C1.7060103@krogh.cc> Date: Mon, 03 Feb 2014 14:59:29 +0100 From: Jesper Krogh User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130330 Thunderbird/17.0.5 MIME-Version: 1.0 To: Tomas Vondra CC: pgsql-hackers@postgresql.org Subject: Re: GIN improvements part2: fast scan References: <51D00FB6.9030008@vmware.com> <528533D9.3040501@vmware.com> <52E141BA.5000302@vmware.com> <52E4141E.60609@fuzzy.cz> <52E42AD0.5060600@vmware.com> <52E4AA3A.9000607@fuzzy.cz> <52E53458.2010106@vmware.com> <52E72A10.5040908@fuzzy.cz> <52E75C54.9050209@vmware.c om> <52E9A274.6040806@fuzzy.c z> <52EE21D1.2080804@vmware.com> <52EED0FC.2060503@fuzzy.cz> <52EEF472.9020106@fuzzy.cz> In-Reply-To: <52EEF472.9020106@fuzzy.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Pg-Spam-Score: -1.9 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org On 03/02/14 02:44, Tomas Vondra wrote: > (2) The question is whether the new patch works fine on rare words. See > this for comparison of the patches against HEAD: > > http://www.fuzzy.cz/tmp/gin/3-rare-words.png > http://www.fuzzy.cz/tmp/gin/3-rare-words-new.png > > and this is the comparison of the two patches: > > http://www.fuzzy.cz/tmp/gin/patches-rare-words.png > > That seems fine to me - some queries are slower, but we're talking > about queries taking 1 or 2 ms, so the measurement error is probably > the main cause of the differences. > > (3) With higher numbers of frequent words, the differences (vs. HEAD or > the previous patch) are not that dramatic as in (1) - the new patch > is consistently by ~20% faster. Just thinking, this is about one algorithm is being better or frequent words and another algorithm being better at rare words... we do have this information (at least or tsvector) in the statistics, would it be possible to just call the "consistent" function more often if the statistics gives signs that it actually is a frequent word? Jesper - heavily dependent on tsvector-searches, with both frequent and rare words. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers