From: Tomas Vondra <tv@fuzzy.cz>
To: pgsql-hackers@postgresql.org
Subject: Re: GIN improvements part 1: additional information
Date: Wed, 15 Jan 2014 02:17:48 +0100
Message-ID: <52D5E1BC.3000208@fuzzy.cz> (raw)
In-Reply-To: <52D478F2.6070005@fuzzy.cz>
References: <CAPpHfdutJEC9+6vWX579afZKzdHKCT2wzDkAEom1XmiXEwF3pw@mail.gmail.com>
<52950F46.2080009@vmware.com>
<CAPpHfds7rT_+6oTyPOQ+XH7dfb6ucnBVrfJc-mbAFrctNy847Q@mail.gmail.com>
<5298615C.7060102@vmware.com>
<5298E834.4090408@vmware.com>
<CAPpHfdvP3G2k04tpCyEA0mAd2e8xOQyuj=2wwAj0UVhB1_oe+g@mail.gmail.com>
<52A58AD5.6030808@vmware.com>
<CAPpHfdvKC9HE_D+BYBUpxqyWSE4UD_=ykZgfSxkAv4P3LBFQjg@mail.gmail.com>
<52A62771.3070800@vmware.com>
<CAPpHfdu2nXxxNCmtMBBiS+3CRdgF8AHCLt2w9Z4s2kKisJSPuA@mail.gmail.com>
<52AEE445.1060206@vmware.com>
<CAPpHfdvK3eZr8iQJtp9ax9k80MKRKkPKgqcc0SB1bn+_KVq-Xg@mail.gmail.com>
<52AF837C.3040201@vmware.com>
<52B2F5B0.1030809@vmware.com>
<52B49C34.8060300@vmware.com>
<CA+HiwqGO91OEE0ZBa0q2RqFEKm5dy5VzXrf7uoYObw9BCKSEVA@mail.gmail.com>
<CAPpHfduBDxQ7Tw5bZCCgxmJzNHDRG6eELAuseuCEvKt=dw0Yfw@mail.gmail.com>
<52D0A947.6050706@fuzzy.cz>
<CAPpHfdskbPbjWYJhkd-FkZCKEUZd03PRAw05NyT0Hd-jxWOyfg@mail.gmail.com>
<52D478F2.6070005@fuzzy.cz>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 14.1.2014 00:38, Tomas Vondra wrote:
> On 13.1.2014 18:07, Alexander Korotkov wrote:
>> On Sat, Jan 11, 2014 at 6:15 AM, Tomas Vondra <tv@fuzzy.cz
>> <mailto:tv@fuzzy.cz>> wrote:
>>
>> On 8.1.2014 22:58, Alexander Korotkov wrote:
>> > Thanks for reporting. Fixed version is attached.
>>
>> I've tried to rerun the 'archie' benchmark with the current patch, and
>> once again I got
>>
>> PANIC: could not split GIN page, didn't fit
>>
>> I reran it with '--enable-cassert' and with that I got
>>
>> TRAP: FailedAssertion("!(ginCompareItemPointers(&items[i - 1],
>> &items[i]) < 0)", File: "gindatapage.c", Line: 149)
>> LOG: server process (PID 5364) was terminated by signal 6: Aborted
>> DETAIL: Failed process was running: INSERT INTO messages ...
>>
>> so the assert in GinDataLeafPageGetUncompressed fails for some reason.
>>
>> I can easily reproduce it, but my knowledge in this area is rather
>> limited so I'm not entirely sure what to look for.
>>
>>
>> I've fixed this bug and many other bug. Now patch passes test suite that
>> I've used earlier. The results are so:
>
> OK, it seems the bug is gone. However now there's a memory leak
> somewhere. I'm loading pgsql mailing list archives (~600k messages)
> using this script
>
> https://bitbucket.org/tvondra/archie/src/1bbeb920/bin/load.py
>
> And after loading about 1/5 of the data, all the memory gets filled by
> the pgsql backends (loading the data in parallel) and the DB gets killed
> by the OOM killer.
I've spent a fair amount of time trying to locate the memory leak, but
so far no luck. I'm not sufficiently familiar with the GIN code.
I can however demonstrate that it's there, and I have rather simple test
case to reproduce it - basically just a CREATE INDEX on a table with ~1M
email message bodies (in a tsvector column). The data is available here
(360MB compressed, 1GB raw):
http://www.fuzzy.cz/tmp/message-b.data.gz
Simply create a single-column table, load data and create the index
CREATE TABLE test ( body_tsvector TSVECTOR );
COPY test FROM '/tmp/message-b.data';
CREATE test_idx ON test USING gin test ( body_tsvector );
I'm running this on a machine with 8GB of RAM, with these settings
shared_buffers=1GB
maintenance_work_mem=1GB
According to top, CREATE INDEX from the current HEAD never consumes more
than ~25% of RAM:
PID USER PR NI VIRT RES SHR %CPU %MEM COMMAND
32091 tomas 20 0 2026032 1,817g 1,040g 56,2 23,8 postgres
which is about right, as (shared_buffers + maintenance_work_mem) is
about 1/4 of RAM.
With the v5 patch version applied, the CREATE INDEX process eventually
goes crazy and allocates almost all the available memory (but somesimes
finishes, mostly by pure luck). This is what I was able to get from top
PID USER PR NI VIRT RES SHR S %CPU %MEM COMMAND
14090 tomas 20 0 7913820 6,962g 955036 D 4,3 91,1 postgres
while the system was still reasonably responsive.
regards
Tomas
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tv@fuzzy.cz
Subject: Re: GIN improvements part 1: additional information
In-Reply-To: <52D5E1BC.3000208@fuzzy.cz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox