Message-ID: <492D4460.1000809@enterprisedb.com>
Date: Wed, 26 Nov 2008 14:43:12 +0200
Organization: EnterpriseDB
User-Agent: Mozilla-Thunderbird 2.0.0.17 (X11/20081018)
MIME-Version: 1.0
To: Tom Lane <tgl@sss.pgh.pa.us>
CC: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Visibility map, partial vacuums
References: <4905AE17.7090305@enterprisedb.com>
	<491D376B.9000608@enterprisedb.com>
	<491D7F52.6070908@enterprisedb.com>
	<4925664C.3090605@enterprisedb.com>
	<26361.1227467112@sss.pgh.pa.us>
	<492A6032.6080000@enterprisedb.com>
	<18086.1227537479@sss.pgh.pa.us>
In-Reply-To: <18086.1227537479@sss.pgh.pa.us>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
From: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>

Tom Lane wrote:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
>> The visibility map won't be inquired unless you vacuum. This is a bit 
>> tricky. In vacuum, we only know whether we can set a bit or not, after 
>> we've acquired a cleanup lock on the page, and scanned all the tuples. 
>> While we're holding a cleanup lock, we don't want to do I/O, which could 
>> potentially block out other processes for a long time. So it's too late 
>> to extend the visibility map at that point.
> 
> This is no good; I think you've made the wrong tradeoffs.  In
> particular, even though only vacuum *currently* uses the map, you want
> to extend it to be used by indexscans.  So it's going to uselessly
> spring into being even without vacuums.
> 
> I'm not convinced that I/O while holding cleanup lock is so bad that we
> should break other aspects of the system to avoid it.  However, if you
> want to stick to that, how about
> 	* vacuum page, possibly set its header bit
> 	* release page lock (but not pin)
> 	* if we need to set the bit, fetch the corresponding map page
> 	  (I/O might happen here)
> 	* get share lock on heap page, then recheck its header bit;
> 	  if still set, set the map bit

Yeah, could do that.

There is another problem, though, if the map is frequently probed for 
pages that don't exist in the map, or the map doesn't exist at all. 
Currently, the size of the map file is kept in relcache, in the 
rd_vm_nblocks_cache variable. Whenever a page is accessed that's > 
rd_vm_nblocks_cache, smgrnblocks is called to see if the page exists, 
and rd_vm_nblocks_cache is updated. That means that every probe to a 
non-existing page causes an lseek(), which isn't free.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com