VBR-Info: md=nordu.net; mc=all; mv=swamid.se
Date: Wed, 26 Aug 2015 11:00:38 +0200 (CEST)
From: Henrik Thostrup Jensen <htj@nordu.net>
To: pgsql-performance <pgsql-performance@postgresql.org>
Subject: Re: Gist indexing performance with cidr types
In-Reply-To: <alpine.DEB.2.11.1508260940060.35330@pyrite>
Message-ID: <alpine.DEB.2.11.1508261031400.35330@pyrite>
References: <alpine.DEB.2.11.1508251504160.31004@pyrite>
 <CAE2gYzyv8zCeKUKyE8EcZcVtdTFeZaUucRjRqmtzu2jT5EQQyQ@mail.gmail.com>
 <alpine.DEB.2.11.1508260940060.35330@pyrite>
User-Agent: Alpine 2.11 (DEB 23 2013-08-11)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
Received-SPF: neutral (e-mailfilter02.sunet.se: 109.105.110.42 is neither
 permitted nor denied by domain htj@nordu.net)
 receiver=e-mailfilter02.sunet.se; client-ip=109.105.110.42;
 envelope-from=<htj@nordu.net>; helo=smtp1.nordu.net;
 identity=mailfrom
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

On Wed, 26 Aug 2015, Henrik Thostrup Jensen wrote:

>> Can you try 9.5 to see if they help?
>
> I'll try installing it and report back.

I upgraded to 9.5 (easier than expected) and ran vacuum analyze.

The query planner now chooses index scan for outer and inner join. This 
seems to cut off roughly a second or so (31s -> 30s, and 17s->16s for when 
using distint on initial route set).

Query:

EXPLAIN (ANALYZE, BUFFERS)
SELECT rv.route, rv.asn, rv.source FROM
     (SELECT DISTINCT route FROM routes_view WHERE asn = %s) r
INNER JOIN routes_view rv ON (r.route && rv.route)
ORDER BY rv.route;

Explain analyze: http://explain.depesz.com/s/L7kZ


9.5 also seems to fix the case with using CTE/WITH was actually slower. 
The fastest I can currently do is this, which finds the minimal set of 
covering routes before joining:

SET enable_bitmapscan = false;
EXPLAIN ANALYZE
WITH
distinct_routes AS (SELECT DISTINCT route FROM routes_view WHERE asn = %s),
minimal_routes  AS (SELECT route FROM distinct_routes
                     EXCEPT
                     SELECT r1.route
                     FROM distinct_routes r1 INNER JOIN distinct_routes r2 ON (r1.route << r2.route))
SELECT rv.route, rv.asn, rv.source
FROM routes_view rv
JOIN minimal_routes ON (rv.route <<= minimal_routes.route);

Explain analyze: http://explain.depesz.com/s/Plx4

The query planner chooses bitmap Index Scan for this query, which adds 
around .5 second the query time, so it isn't that bad of a decision.

Unfortunately it still takes 15 seconds for my test case (a big network, 
but still a factor 10 from the biggest).

Are the coverage operatons just that expensive?


     Best regards, Henrik


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance