To: "Steven Flatt" <steven.flatt@gmail.com>
cc: "Gregory Stark" <stark@enterprisedb.com>,
	"Bill Moran" <wmoran@collaborativefusion.com>,
	"Vivek Khera" <vivek@khera.org>,
	"Pgsql performance" <pgsql-performance@postgresql.org>
Subject: Re: When/if to Reindex 
In-reply-to: <357fa7590708231525h2f48ce4dw3e48ecca5ccddf72@mail.gmail.com> 
References: <357fa7590707181008k65ab2f20l55bebb3c14f52770@mail.gmail.com>
	<38118C09-FC0A-4CCB-9461-FC6D29D81EE4@khera.org>
	<357fa7590708081212p7b19cbbbn39dbadae8603edd6@mail.gmail.com>
	<20070808152757.9eb9bbcc.wmoran@collaborativefusion.com>
	<11801.1186624308@sss.pgh.pa.us>
	<357fa7590708090651p71b9bc47n66739f8dbbb4ff57@mail.gmail.com>
	<357fa7590708220955h441cd75ap4c0f0342a1c30a14@mail.gmail.com>
	<873aybw2le.fsf@oxford.xeocode.com>
	<357fa7590708231525h2f48ce4dw3e48ecca5ccddf72@mail.gmail.com>
Comments: In-reply-to "Steven Flatt" <steven.flatt@gmail.com>
	message dated "Thu, 23 Aug 2007 18:25:25 -0400"
Date: Thu, 23 Aug 2007 23:59:02 -0400
Message-ID: <17370.1187927942@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Steven Flatt" <steven.flatt@gmail.com> writes:
> Interestingly enough, the example you've given does not work for me either.
> The select count(*) from test blocks until the reindex completes.  Are we
> using the same pg version?

Seems like a fair question, because Greg's example blocks for me too,
in plancat.c where the planner is trying to acquire information on each
index.  This seems to be an unwanted side effect of this 8.2-era patch
http://archives.postgresql.org/pgsql-committers/2006-07/msg00356.php
specifically, note here
http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/optimizer/util/plancat.c.diff?r1=1.121;r2=1.122;f=h
how the new planner coding takes at least AccessShareLock on each index,
where the old coding took no lock at all.

I think that the new coding rule of "you *must* take some lock when
opening the relation" is essential for tables, but it might not be
necessary for indexes if you've got a lock on the parent table.
We don't allow any schema changes on an index to be made without holding
exclusive lock on the parent, so plancat.c's basic purpose of finding
out the properties of the index could be done safely without any index
lock.

The fly in the ointment is that after collecting the pg_index definition
of the index, plancat.c also wants to know how big it is --- it calls
RelationGetNumberOfBlocks.  And that absolutely does look at the
physical storage, which means it absolutely is unsafe to do in parallel
with a REINDEX that will be dropping the old physical storage at some
point.

So maybe we are stuck and we have to say "that doesn't work anymore".
But it feels like we might not be too far away from letting it still
work.  Thoughts, ideas?

			regards, tom lane