Subject: Re: CSStorm occurred again by postgreSQL8.2
From: Simon Riggs <simon@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>,
	ITAGAKI Takahiro <itagaki.takahiro@oss.ntt.co.jp>,
	Bruce Momjian <bruce@momjian.us>, pgsql-hackers@postgresql.org
In-Reply-To: <22080.1158198335@sss.pgh.pa.us>
References: <20060808110913.50FA.ITAGAKI.TAKAHIRO@oss.ntt.co.jp>
	<200608260346.k7Q3k2g06954@momjian.us>
	<20060828181312.5638.ITAGAKI.TAKAHIRO@oss.ntt.co.jp>
	<13251.1157299738@sss.pgh.pa.us>  <22080.1158198335@sss.pgh.pa.us>
Content-Type: text/plain
Date: Tue, 11 Sep 2007 10:42:24 +0100
Message-Id: <1189503744.4281.433.camel@ebony.site>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

On Wed, 2006-09-13 at 21:45 -0400, Tom Lane wrote:

> Anyway, given that there's this one nonobvious gotcha, there might be
> others.  My recommendation is that we take this off the open-items list
> for 8.2 and revisit it in the 8.3 cycle when there's more time.

Well, its still 8.3 just...

As discussed in the other thread "Final Thoughts for 8.3 on LWLocking
and Scalability", XidCacheRemoveRunningXids() is now the only holder of
an X lock during normal processing, so I would like to remove it. 
Here's how:

Currently, we take the lock, remove the subxact and then shuffle down
all the other subxactIds so that the subxact cache is contiguous.

I propose that we simply zero out the subxact entry without re-arranging
the cache; this will be atomic, so we need not acquire an X lock. We
then increment ndeletedxids. When we enter a new subxact into the cache,
if ndeletedxids > 0 we scan the cache to find an InvalidTransactionId
that we can use, then decrement ndeletedxids. So ndeletedxids is just a
hint, not an absolute requirement. nxids then becomes the number of
cache entries and never goes down until EOXact. The subxact cache is no
longer in order, but then it doesn't need to be either.

When we take a snapshot we will end up taking a copy of zeroed cache
entries, so the snapshots will be slightly larger than previously.
Though still no larger than the max. The size reduction was not so large
as to make a significant difference across the whole array, so
scalability is the main issue to resolve.

The snapshots will be valid with no change, since InvalidTransactionId
will never match against any recorded Xid.

I would also like to make the size of the subxact cache configurable
with a parameter such as subtransaction_cache_size = 64 (default), valid
range 4-256.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com