Message-ID: <5571C660.3030404@BlueTreble.com>
Date: Fri, 5 Jun 2015 10:55:12 -0500
From: Jim Nasby <Jim.Nasby@BlueTreble.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
 rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: Michael Paquier <michael.paquier@gmail.com>, Craig Ringer
 <craig@2ndquadrant.com>
CC: Stephen Frost <sfrost@snowman.net>, Josh Berkus <josh@agliodbs.com>, Noah
 Misch <noah@leadboat.com>, Bruce Momjian <bruce@momjian.us>, Andres Freund
 <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Tom Lane
 <tgl@sss.pgh.pa.us>, Magnus Hagander <magnus@hagander.net>,
 "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, pgsql-core
 <pgsql-core@postgresql.org>
Subject: Re: [CORE] Restore-reliability mode
References: <20150529223321.GB6535@alap3.anarazel.de>
	<CA+TgmoYSO0a9YDe2bhpZoPgO4m62CyQL26LRkCQ8=vrQ8ZwdXA@mail.gmail.com>
	<12510.1432958887@sss.pgh.pa.us>
	<CA+TgmoYovkev0xXnxaTD8vnt=M9Cn+RmOkVEouqwj6_9JLkzLw@mail.gmail.com>
	<20150530154559.GA25053@momjian.us>
	<20150530204727.GJ13944@alap3.anarazel.de>
	<20150531024845.GB13697@momjian.us>
	<20150603135049.GA74753@tornado.leadboat.com>
	<WM!4ae914f9770e3cb63aa899dbc2db60d7ff4a3ad580e90f3dc1cd72315fff26fd1935dbe8884fd2e9f5370d881036c3bd!@asav-3.01.com>
	<556F3798.6010707@agliodbs.com>	<20150604144317.GT26667@tamriel.snowman.net>
	<CAMsr+YEA_YwGMTeG-zGDNL1_RwxN4dJYQ9xQfg8np3CoF4bQ1A@mail.gmail.com>
 <CAB7nPqQsBAKuCWSqd834LwC0T+g8=yzD1GnPts6oMe4Ewrpjbg@mail.gmail.com>
In-Reply-To: 
 <CAB7nPqQsBAKuCWSqd834LwC0T+g8=yzD1GnPts6oMe4Ewrpjbg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2015 15:55:15.2805 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR11MB0677
X-Pg-Spam-Score: -3.0 (---)
List-Archive: <http://archives.postgresql.org/pgsql-hackers>
List-Help: <mailto:majordomo@postgresql.org?body=help>
List-ID: <pgsql-hackers.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@postgresql.org>
List-Post: <mailto:pgsql-hackers@postgresql.org>
List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-hackers>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 6/4/15 11:28 PM, Michael Paquier wrote:
<list of things to test>
* More configuration variations; ./configure, initdb options, and *.conf
* More edge-case testing. (ie: what happens to each varlena as it 
approaches 1GB? 1B tables test. Etc.)
* More race-condition testing, like the tool Peter used heavily during 
ON CONFLICT development (written by Jeff Janes?)
* More non-SQL testing. For example, the logic in HeapTupleSatisfies* is 
quite complicated yet there's no tests dedicated to ensuring the logic 
is correct because it'd be extremely difficult (if not impossible) to 
construct those tests at a SQL level. Testing them with direct test 
calls to HeapTupleSatisfies* wouldn't be difficult, but we have no 
machinery to do C level testing.

>> Is pg_tap a reasonable starting point for this sort of testing?
> IMO, using the TAP machinery would be a good base for that. What lacks
> is a basic set of perl routines that one can easily use to set of test
> scenarios.

I think Stephen was referring specifically to pgTap (http://pgtap.org/).

Isn't our TAP framework just different output from pg_regress? Is there 
documentation on our TAP stuff?

>> >How would a test that would've caught the multixact issues look?
> I have not followed closely those discussions, not sure about that.

I've thought about this and unfortunately I think this may be a scenario 
that's just too complex to completely protect against with a test. What 
might help though is having better testing of edge cases (such as MXID 
wrap) and then combining that with other forms of testing, such as 
pg_upgrade and streaming rep. testing. Test things like "What happens if 
we pg_upgrade a cluster that's in danger of wraparound?"
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers