pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Heikki Linnakangas <hlinnakangas@vmware.com>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Tomas Vondra <tv@fuzzy.cz>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: GIN improvements part 1: additional information
Date: Mon, 20 Jan 2014 20:30:35 +0200
Message-ID: <52DD6B4B.3050806@vmware.com> (raw)
In-Reply-To: <CAPpHfdsYTRWuR5YkadvB_G=JiBRLsYRAL7CJkpPiz3AitvhD6g@mail.gmail.com>
References: <CAPpHfdutJEC9+6vWX579afZKzdHKCT2wzDkAEom1XmiXEwF3pw@mail.gmail.com>
	<52A58AD5.6030808@vmware.com>
	<CAPpHfdvKC9HE_D+BYBUpxqyWSE4UD_=ykZgfSxkAv4P3LBFQjg@mail.gmail.com>
	<52A62771.3070800@vmware.com>
	<CAPpHfdu2nXxxNCmtMBBiS+3CRdgF8AHCLt2w9Z4s2kKisJSPuA@mail.gmail.com>
	<52AEE445.1060206@vmware.com>
	<CAPpHfdvK3eZr8iQJtp9ax9k80MKRKkPKgqcc0SB1bn+_KVq-Xg@mail.gmail.com>
	<52AF837C.3040201@vmware.com>
	<52B2F5B0.1030809@vmware.com>
	<52B49C34.8060300@vmware.com>
	<CA+HiwqGO91OEE0ZBa0q2RqFEKm5dy5VzXrf7uoYObw9BCKSEVA@mail.gmail.com>
	<CAPpHfduBDxQ7Tw5bZCCgxmJzNHDRG6eELAuseuCEvKt=dw0Yfw@mail.gmail.com>
	<52D0A947.6050706@fuzzy.cz>
	<CAPpHfdskbPbjWYJhkd-FkZCKEUZd03PRAw05NyT0Hd-jxWOyfg@mail.gmail.com>
	<52D478F2.6070005@fuzzy.cz>
	<52D5E1BC.3000208@fuzzy.cz>
	<CAPpHfdv9Pheu5atEEUk75f_S1nf6vCoRqge2yUQ1v3xgZ4UP3g@mail.gmail.com>
	<CAPpHfds0A03zwmYdhKUg6s01Vq1p30n01=3c-tct264QTpV--Q@mail.gmail.com>
	<52D9789E.3030202@vmware.com>
	<CAPpHfdsYTRWuR5YkadvB_G=JiBRLsYRAL7CJkpPiz3AitvhD6g@mail.gmail .com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 01/17/2014 08:49 PM, Alexander Korotkov wrote:
> On Fri, Jan 17, 2014 at 10:38 PM, Heikki Linnakangas <
> hlinnakangas@vmware.com> wrote:
>
>> On 01/17/2014 01:05 PM, Alexander Korotkov wrote:
>>
>>> Seems to be fixed in the attached version of patch.
>>> Another improvement in this version: only left-most segments and further
>>> are re-encoded. Left part of page are left untouched.
>>
>> I'm looking into this now. A few quick notes:
>>
>> * Even when you don't re-encode the whole page, you still WAL-log the
>> whole page. While correct, it'd be a pretty obvious optimization to only
>> WAL-log the modified part.
>
> Oh, I overlooked it. I wrote code in ginRedoInsertData which finds
> appropriate place fro data but didn't notice that I still wrote whole page
> to xlog record.

Yeah. I think ginRedoInsertData was too cute to be bug-free. I added an 
explicit unmodifiedsize field to the WAL record, so that 
ginRedoInsertData doesn't need to calculate it.

>> * When new items are appended to the end of the page so that only the last
>> existing compressed segment is re-encoded, and the page has to be split,
>> the items aren't divided 50/50 on the pages. The loop that moves segments
>> destined for the left page to the right won't move any existing, untouched,
>> segments.
>
> I think this loop will move one segment. However, it's too small.

Implemented this.

I noticed that the gin vacuum redo routine is dead code, except for the 
data-leaf page handling, because we never remove entries or internal 
nodes (page deletion is a separate wal record type). And the data-leaf 
case is functionally equivalent to heap newpage records. I removed the 
dead code and made it more clear that it resembles heap newpage.

Attached is a yet another version, with more bugs fixed and more 
comments added and updated. I would appreciate some heavy-testing of 
this patch now. If you could re-run the tests you've been using, that 
could be great. I've tested the WAL replay by replicating GIN operations 
over streaming replication. That doesn't guarantee it's correct, but 
it's a good smoke test.

- Heikki

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachments:

  [application/x-gzip] gin-packed-postinglists-20140120.patch.gz (32.4K, ../52DD6B4B.3050806@vmware.com/2-gin-packed-postinglists-20140120.patch.gz)
  download

view thread (119+ messages)  latest in thread

Message-ID: <52DD6B4B.3050806@vmware.com>
Permalink:  ../52DD6B4B.3050806@vmware.com/
Also on:    postgresql.org/message-id/52DD6B4B.3050806@vmware.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: hlinnakangas@vmware.com, aekorotkov@gmail.com, tv@fuzzy.cz
  Subject: Re: GIN improvements part 1: additional information
  In-Reply-To: <52DD6B4B.3050806@vmware.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox