pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Craig Ringer <ringerc@ringerc.id.au>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Jeff Janes <jeff.janes@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Daniel Farina <daniel@heroku.com>
Cc: "Harold A. Giménez" <harold.gimenez@gmail.com>
Subject: Re: Re: Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)
Date: Tue, 17 Jul 2012 20:31:52 -0400
Message-ID: <11099.1342571512@sss.pgh.pa.us> (raw)
In-Reply-To: <5005FF9F.4040706@ringerc.id.au>
References: <4FFCCAC4.4030503@ringerc.id.au>
	<CAAZKuFYsK+ZQuC=zf0xE=M-LNa02tB7RBy3CHEUyq7LWCkK6jQ@mail.gmail.com>
	<CAMkU=1ydaypgAeZxyaMc=KhJJw5riGMBiLpvSqOO42=qArRZmQ@mail.gmail.com>
	<32078D4B40DD40ECB739D0ECD7D95996@gmail.com>
	<CAMkU=1zJK6WiS9-6eO8rNmRpKSSB4=uw2mTsD=k64mNh__=Dcg@mail.gmail.com>
	<CAMkU=1w7vLA63hf-+Uc_j61CvOdKixsOHph9d0H-aLNBYgfX=w@mail.gmail.com>
	<CAMkU=1yLXvODRZZ_=fgrEeJfk2tvZPTTD-8n8BwrAhNz_WBT0A@mail.gmail.com>
	<19949.1342376967@sss.pgh.pa.us>
	<CA+TgmoaTf=zjP37-oG7fr1VbGfuvHSfpajgLZ65EeSJX2mmquA@mail.gmail.com>
	<9603.1342454919@sss.pgh.pa.us>
	<CA+Tgmoa4ta60Y-rZpEPDRCQKJV8-Uqe0P610Jgj2q7e+fondig@mail.gmail.com>
	<11952.1342456591@sss.pgh.pa.us>
	<CA+TgmoYRXMWQ_DGmmqu3Zp8oW3Lt-9Lc1q3EHaV_ZmjWqaPkuA@mail.gmail.com>
	<13011.1342457859@sss.pgh.pa.us>
	<CA+TgmoaEPLo5fmBX1JSkGK3umAamFVghwY7qDb4_QqzteZgUPA@mail.gmail.com>
	<15955.1342466333@sss.pgh.pa.us>
	<CA+TgmoYnqP! f03rYJo3G6dMrqShQF2iMtfpYBPgWdxtEdacaX-w@mail.gmail.com>
	<9247.1342565810@sss.pgh.pa.us>
	<5005FF9F.4! 040706@ringerc.id.au>

Craig Ringer <ringerc@ringerc.id.au> writes:
> On 07/18/2012 06:56 AM, Tom Lane wrote:
>> This implies that nobody has done pull-the-plug testing on either HEAD
>> or 9.2 since the checkpointer split went in (2011-11-01)

> That makes me wonder if on top of the buildfarm, extending some 
> buildfarm machines into a "crashfarm" is needed:

Not sure if we need a whole "farm", but certainly having at least one
machine testing this sort of stuff on a regular basis would make me feel
a lot better.

> The main challenge would be coming up with suitable tests to run, ones 
> that could then be checked to make sure nothing was broken.

One fairly simple test scenario could go like this:

	* run the regression tests
	* pg_dump the regression database
	* run the regression tests again
	* hard-kill immediately upon completion
	* restart database, allow it to perform recovery
	* pg_dump the regression database
	* diff previous and new dumps; should be the same

The main thing this wouldn't cover is discrepancies in user indexes,
since pg_dump doesn't do anything that's likely to result in indexscans
on user tables.  It ought to be enough to detect the sort of system-wide
problem we're talking about here, though.

In general I think the hard part is automated reproduction of an
OS-crash scenario, but your ideas about how to do that sound promising.
Once we have that going, it shouldn't be hard to come up with tests
of the form "do X, hard-crash, recover, check X still looks sane".

> What else should be checked? The main thing that comes to mind for me is 
> something I've worried about for a while: that Pg might not always 
> handle out-of-disk-space anywhere near as gracefully as it's often 
> claimed to.

+1

			regards, tom lane



view thread (21+ messages)  latest in thread

Message-ID: <11099.1342571512@sss.pgh.pa.us>
Permalink:  ../11099.1342571512@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/11099.1342571512@sss.pgh.pa.us

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: tgl@sss.pgh.pa.us, ringerc@ringerc.id.au, robertmhaas@gmail.com, jeff.janes@gmail.com, pgsql-hackers@postgresql.org, daniel@heroku.com, harold.gimenez@gmail.com
  Subject: Re: Re: Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)
  In-Reply-To: <11099.1342571512@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox