Subject: Re: CSStorm occurred again by postgreSQL8.2
From: Simon Riggs <simon@2ndquadrant.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>,
	ITAGAKI Takahiro <itagaki.takahiro@oss.ntt.co.jp>,
	pgsql-hackers@postgresql.org
In-Reply-To: <200803130013.m2D0DK327190@momjian.us>
References: <200803130013.m2D0DK327190@momjian.us>
Content-Type: text/plain
Date: Thu, 13 Mar 2008 08:24:45 +0000
Message-Id: <1205396685.4285.55.camel@ebony.site>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

On Wed, 2008-03-12 at 20:13 -0400, Bruce Momjian wrote:
> Is this a TODO?  Tom's reply was:

The general topic, yes. The caveats still apply.

> > Nonsense.  Main transaction exit also takes an exclusive lock, and is
> > far more likely to be exercised in typical workloads than a
> > subtransaction abort.
> > 
> > In any case: there has still not been any evidence presented by anyone
> > that optimizing XidCacheRemoveRunningXids will help one bit.  Given the
> > difficulty of measuring any benefit from the last couple of
> > optimizations in this general area, I'm thinking that such evidence
> > will be hard to come by.  And we have got way more than enough on our
> > plates already.  Can we let go of this for 8.3, please?
> 
> ---------------------------------------------------------------------------
> 
> Simon Riggs wrote:
> > On Wed, 2006-09-13 at 21:45 -0400, Tom Lane wrote:
> > 
> > > Anyway, given that there's this one nonobvious gotcha, there might be
> > > others.  My recommendation is that we take this off the open-items list
> > > for 8.2 and revisit it in the 8.3 cycle when there's more time.
> > 
> > Well, its still 8.3 just...
> > 
> > As discussed in the other thread "Final Thoughts for 8.3 on LWLocking
> > and Scalability", XidCacheRemoveRunningXids() is now the only holder of
> > an X lock during normal processing, so I would like to remove it. 
> > Here's how:
> > 
> > Currently, we take the lock, remove the subxact and then shuffle down
> > all the other subxactIds so that the subxact cache is contiguous.
> > 
> > I propose that we simply zero out the subxact entry without re-arranging
> > the cache; this will be atomic, so we need not acquire an X lock. We
> > then increment ndeletedxids. When we enter a new subxact into the cache,
> > if ndeletedxids > 0 we scan the cache to find an InvalidTransactionId
> > that we can use, then decrement ndeletedxids. So ndeletedxids is just a
> > hint, not an absolute requirement. nxids then becomes the number of
> > cache entries and never goes down until EOXact. The subxact cache is no
> > longer in order, but then it doesn't need to be either.
> > 
> > When we take a snapshot we will end up taking a copy of zeroed cache
> > entries, so the snapshots will be slightly larger than previously.
> > Though still no larger than the max. The size reduction was not so large
> > as to make a significant difference across the whole array, so
> > scalability is the main issue to resolve.
> > 
> > The snapshots will be valid with no change, since InvalidTransactionId
> > will never match against any recorded Xid.
> > 
> > I would also like to make the size of the subxact cache configurable
> > with a parameter such as subtransaction_cache_size = 64 (default), valid
> > range 4-256.
> > 
> > -- 
> >   Simon Riggs
> >   2ndQuadrant  http://www.2ndQuadrant.com
> > 
> > 
> > ---------------------------(end of broadcast)---------------------------
> > TIP 9: In versions below 8.0, the planner will ignore your desire to
> >        choose an index scan if your joining column's datatypes do not
> >        match
> 
-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com 

  PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk