Subject: Re: Reviewing freeze map code
To: Peter Geoghegan <pg@heroku.com>, Andres Freund <andres@anarazel.de>
References: <20160506212958.wwwc22hfesvj76af@alap3.anarazel.de>
 <20160506213652.GA288015@alvherre.pgsql>
 <20160506214943.d44r4glxgw2ff2sg@alap3.anarazel.de>
 <CAM3SWZQf7VsUkMtSnzDrA0ECsZgjV07r3kD6XvRR-qJGNLWVnQ@mail.gmail.com>
 <1e78ce70-db1e-e639-f758-dc202f56ab5e@BlueTreble.com>
CC: Alvaro Herrera <alvherre@2ndquadrant.com>, "Joshua D. Drake"
	<jd@commandprompt.com>, Josh berkus <josh@agliodbs.com>, Robert Haas
	<robertmhaas@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Jeff Janes
	<jeff.janes@gmail.com>
From: Jim Nasby <Jim.Nasby@BlueTreble.com>
Message-ID: <f8a22607-19fb-15a9-0bc9-2aefbc59bef6@BlueTreble.com>
Date: Tue, 10 May 2016 23:54:33 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0)
 Gecko/20100101 Thunderbird/45.0
MIME-Version: 1.0
In-Reply-To: <1e78ce70-db1e-e639-f758-dc202f56ab5e@BlueTreble.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
SpamDiagnosticOutput: 1:23
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2016 04:54:38.2223 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR11MB0679
X-Pg-Spam-Score: -1.9 (-)
List-Archive: <http://archives.postgresql.org/pgsql-hackers>
List-Help: <mailto:majordomo@postgresql.org?body=help>
List-ID: <pgsql-hackers.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@postgresql.org>
List-Post: <mailto:pgsql-hackers@postgresql.org>
List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-hackers>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 5/10/16 11:42 PM, Jim Nasby wrote:
> On 5/6/16 4:55 PM, Peter Geoghegan wrote:
>> On Fri, May 6, 2016 at 2:49 PM, Andres Freund <andres@anarazel.de> wrote:
>>>> Jeff Janes has done astounding work in these matters.  (I don't think
>>>> we credit him enough for that.)
>>>
>>> +many.
>>
>> Agreed. I'm a huge fan of what Jeff has been able to do in this area.
>> I often say so. It would be even better if Jeff's approach to testing
>> was followed as an example by other people, but I wouldn't bet on it
>> ever happening. It requires real persistence and deep understanding to
>> do well.
>
> It takes deep understanding to *design* the tests, not to write them.
> There's a lot of folks out there that will never understand enough to
> design tests meant to expose data corruption but who could easily code
> someone else's design, especially if we provided tools/ways to tweak a
> cluster to make testing easier/faster (such as artificially advancing
> XID/MXID).

Speaking of which, another email in the thread made me realize that 
there's a test condition no one has mentioned: verifying we don't lose 
tuples after wraparound.

To test this, you'd want a table that's mostly frozen. Ideally, dirty a 
single tuple on a bunch of frozen pages, with committed updates, 
deletes, and un-committed inserts. Advance XID far enough to get you 
close to wrap-around. Do a vacuum, SELECT count(*), advance XID past 
wraparound, SELECT count(*) again and you should get the same number.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)   mobile: 512-569-9461


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers