To: "Heikki Linnakangas" <heikki@enterprisedb.com>
cc: "Craig Ringer" <craig@postnewspapers.com.au>,
	pgsql-performance@postgresql.org
Subject: Re: Very slow (2 tuples/second) sequential scan after bulk insert;
	speed returns to ~500 tuples/second after commit
In-reply-to: <47D5150C.8090802@enterprisedb.com> 
References: <47D505AF.4040101@postnewspapers.com.au>
	<47D5150C.8090802@enterprisedb.com>
Comments: In-reply-to "Heikki Linnakangas" <heikki@enterprisedb.com>
	message dated "Mon, 10 Mar 2008 11:01:32 -0000"
Date: Mon, 10 Mar 2008 10:33:58 -0400
Message-ID: <23588.1205159638@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
> For 8.4, it would be nice to improve that. I tested that on my laptop 
> with a similarly-sized table, inserting each row in a pl/pgsql function 
> with an exception handler, and I got very similar run times. According 
> to oprofile, all the time is spent in TransactionIdIsInProgress. I think 
> it would be pretty straightforward to store the committed subtransaction 
> ids in a sorted array, instead of a linked list, and binary search.

I think the OP is not complaining about the time to run the transaction
that has all the subtransactions; he's complaining about the time to
scan the table that it emitted.  Presumably, each row in the table has a
different (sub)transaction ID and so we are thrashing the clog lookup
mechanism.  It only happens once because after that the XMIN_COMMITTED
hint bits are set.

This probably ties into the recent discussions about eliminating the
fixed-size allocations for SLRU buffers --- I suspect it would've run
better if it could have scaled up the number of pg_clog pages held in
memory.

			regards, tom lane