pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Heikki Linnakangas <hlinnakangas@vmware.com>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Tomas Vondra <tv@fuzzy.cz>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: GIN improvements part 1: additional information
Date: Tue, 14 Jan 2014 10:34:56 +0200
Message-ID: <52D4F6B0.4000304@vmware.com> (raw)
In-Reply-To: <CAPpHfdskbPbjWYJhkd-FkZCKEUZd03PRAw05NyT0Hd-jxWOyfg@mail.gmail.com>
References: <CAPpHfdutJEC9+6vWX579afZKzdHKCT2wzDkAEom1XmiXEwF3pw@mail.gmail.com>
	<52950F46.2080009@vmware.com>
	<CAPpHfds7rT_+6oTyPOQ+XH7dfb6ucnBVrfJc-mbAFrctNy847Q@mail.gmail.com>
	<5298615C.7060102@vmware.com>
	<5298E834.4090408@vmware.com>
	<CAPpHfdvP3G2k04tpCyEA0mAd2e8xOQyuj=2wwAj0UVhB1_oe+g@mail.gmail.com>
	<52A58AD5.6030808@vmware.com>
	<CAPpHfdvKC9HE_D+BYBUpxqyWSE4UD_=ykZgfSxkAv4P3LBFQjg@mail.gmail.com>
	<52A62771.3070800@vmware.com>
	<CAPpHfdu2nXxxNCmtMBBiS+3CRdgF8AHCLt2w9Z4s2kKisJSPuA@mail.gmail.com>
	<52AEE445.1060206@vmware.com>
	<CAPpHfdvK3eZr8iQJtp9ax9k80MKRKkPKgqcc0SB1bn+_KVq-Xg@mail.gmail.com>
	<52AF837C.3040201@vmware.com>
	<52B2F5B0.1030809@vmware.com>
	<52B49C34.8060300@vmware.com>
	<CA+HiwqGO91OEE0ZBa0q2RqFEKm5dy5VzXrf7uoYObw9BCKSEVA@mail.gmail.com>
	<CAPpHfduBDxQ7Tw5bZCCgxmJzNHDRG6eELAuseuCEvKt=dw0Yfw@mail.gmail.com>
	<52D0A947.6050706@fuzzy.cz>
	<CAPpHfdskbPbjWYJhkd-FkZCKEUZd03PRAw05NyT0Hd-jxWOyfg@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 01/13/2014 07:07 PM, Alexander Korotkov wrote:
> I've fixed this bug and many other bug. Now patch passes test suite that
> I've used earlier. The results are so:
>
> Operations time:
>           event         |     period
> -----------------------+-----------------
>   index_build           | 00:01:47.53915
>   index_build_recovery  | 00:00:04
>   index_update          | 00:05:24.388163
>   index_update_recovery | 00:00:53
>   search_new            | 00:24:02.289384
>   search_updated        | 00:27:09.193343
> (6 rows)
>
> Index sizes:
>       label     |   size
> ---------------+-----------
>   new           | 384761856
>   after_updates | 667942912
> (2 rows)
>
> Also, I made following changes in algorithms:
>
>     - Now, there is a limit to number of uncompressed TIDs in the page.
>     After reaching this limit, they are encoded independent on if they can fit
>     page. That seems to me more desirable behaviour and somehow it accelerates
>     search speed. Before this change times were following:
>
>           event         |     period
> -----------------------+-----------------
>   index_build           | 00:01:51.467888
>   index_build_recovery  | 00:00:04
>   index_update          | 00:05:03.315155
>   index_update_recovery | 00:00:51
>   search_new            | 00:24:43.194882
>   search_updated        | 00:28:36.316784
> (6 rows)

Hmm, that's strange. Any idea why that's happening? One theory is that 
when you re-encode the pages more aggressively, there are fewer pages 
with a mix of packed and unpacked items. Mixed pages are somewhat slower 
to scan than fully packed or fully unpacked pages, because 
GinDataLeafPageGetItems() has to merge the packed and unpacked items 
into a single list. But I wouldn't expect that effect to be large enough 
to explain the results you got.

>     - Page are not fully re-encoded if it's enough to re-encode just last
>     segment.

Great! We should also take advantage of that in the WAL record that's 
written; no point in WAL-logging all the segments, if we know that only 
last one was modified.	

- Heikki


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers



view thread (119+ messages)  latest in thread

Message-ID: <52D4F6B0.4000304@vmware.com>
Permalink:  ../52D4F6B0.4000304@vmware.com/
Also on:    postgresql.org/message-id/52D4F6B0.4000304@vmware.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: hlinnakangas@vmware.com, aekorotkov@gmail.com, tv@fuzzy.cz
  Subject: Re: GIN improvements part 1: additional information
  In-Reply-To: <52D4F6B0.4000304@vmware.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox