From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: WIP: bloom filter in Hash Joins with batches
Date: Thu, 17 Dec 2015 16:34:47 +0100
Message-ID: <5672D617.9060601@2ndquadrant.com> (raw)
In-Reply-To: <CACACo5Q0z9vQE8pHc+b6rzHr_TOTNg4=nc4eiQodtD1poOBCCQ@mail.gmail.com>
References: <5670946E.8070705@2ndquadrant.com>
<CACACo5Q0z9vQE8pHc+b6rzHr_TOTNg4=nc4eiQodtD1poOBCCQ@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Hi,
On 12/17/2015 10:50 AM, Shulgin, Oleksandr wrote:
> On Tue, Dec 15, 2015 at 11:30 PM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
>
>> Attached is a spreadsheet with results for various work_mem
>> values, and also with a smaller data set (just 30M rows in the fact
>> table), which easily fits into memory. Yet it shows similar gains,
>> shaving off ~40% in the best case, suggesting that this is not just
>> thanks to reduction of I/O when forcing the temp files to disk.
>
> A neat idea! Have you possibly tried to also collect statistics
> about actual false-positive rates and filter allocation sizes in
> every of the collected data points?
The patch counts and prints the total number of lookups, and the number
of eliminated rows. The bloom filter is currently sized for 5% false
positives rate, and the numbers I've seen match that.
I think ultimately we'll need to measure the false positive rate, so
that we can use it to dynamically disable the bloom filter if it gets
inefficient. Also maybe put some of that into EXPLAIN ANALYZE.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, oleksandr.shulgin@zalando.de
Subject: Re: WIP: bloom filter in Hash Joins with batches
In-Reply-To: <5672D617.9060601@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox