Message-ID: <5465B770.70604@BlueTreble.com>
Date: Fri, 14 Nov 2014 02:04:00 -0600
From: Jim Nasby <Jim.Nasby@BlueTreble.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
 rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: Andres Freund <andres@2ndquadrant.com>, Peter Eisentraut
 <peter_e@gmx.net>, Robert Haas <robertmhaas@gmail.com>
CC: Michael Paquier <michael.paquier@gmail.com>, Jim Nasby <jim@nasby.net>,
 PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Subject: Re: REINDEX CONCURRENTLY 2.0
References: 
 <CAB7nPqTZfx20Ra2Wf4Gm0a=NqF4q2aC8Q6OS8vKN6fhnD=cUxQ@mail.gmail.com>
 <CAB7nPqRkwKFgn4BFUybqU-Oo-=Gcbq0K-8H93Gr6fX-GGRPDXg@mail.gmail.com>
 <20140827064120.GZ21544@awork2.anarazel.de>
 <CAB7nPqSdSMeQMCvtt9P+ndfao5ALpVPDEMmaBPVjNiifLzb=3Q@mail.gmail.com>
 <CAB7nPqSyZhdc-gvwm3KyueoceFJSX5ypQAx_rsJ2DWr5nHtixA@mail.gmail.com>
 <545B8AD2.8030203@gmx.net>
 <CA+TgmoZtd=sa8LD0tnWgRDzx-Zzt_g_FnPf9jtj61z_rbuse6w@mail.gmail.com>
 <CA+TgmoZ=0NMfMVjemYjO0zWFi8PS3OMZCEsqimcgc34U8UKQeg@mail.gmail.com>
 <20141112213912.GH13473@awork2.anarazel.de>
 <CA+TgmoZkJu-R58mBUVCJzFJvgeiQyJ2_1RfrZh6gy5LSzyCSag@mail.gmail.com>
 <20141113003106.GJ13473@awork2.anarazel.de> <5465215D.2060505@gmx.net>
 <61E955E2-61D7-45E8-BC66-F021EA0CE3CE@2ndquadrant.com>
In-Reply-To: <61E955E2-61D7-45E8-BC66-F021EA0CE3CE@2ndquadrant.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 11/13/14, 3:50 PM, Andres Freund wrote:
> On November 13, 2014 10:23:41 PM CET, Peter Eisentraut <peter_e@gmx.net> wrote:
>> On 11/12/14 7:31 PM, Andres Freund wrote:
>>> Yes, it sucks. But it beats not being able to reindex a relation with
>> a
>>> primary key (referenced by a fkey) without waiting several hours by a
>>> couple magnitudes. And that's the current situation.
>>
>> That's fine, but we have, for better or worse, defined CONCURRENTLY :=
>> does not take exclusive locks.  Use a different adverb for an
>> in-between
>> facility.
>
> I think that's not actually a service to our users. They'll have to adapt their scripts and knowledge when we get around to the more concurrent version. What exactly CONCURRENTLY means is already not strictly defined and differs between the actions.

It also means that if we ever found a way to get rid of the exclusive lock we'd then have an inconsistency anyway. Or we'd also create REINDEX CONCURRENT at that time, and then have 2 command syntaxes to support.

> I'll note that DROP INDEX CONCURRENTLY actually already  internally acquires an AEL lock. Although it's a bit harder to see the consequences of that.

Having been responsible for a site where downtime was a 6 figure dollar amount per hour, I've spent a LOT of time worrying about lock problems. The really big issue here isn't grabbing an exclusive lock; it's grabbing one at some random time when no one is there to actively monitor what's happening. (If you can't handle *any* exclusive locks, that also means you can never do an ALTER TABLE ADD COLUMN either.) With that in mind, would it be possible to set this up so that the time-consuming process of building the new index file happens first, and then (optionally) some sort of DBA action is required to actually do the relfilenode swap? I realize that's not the most elegant solution, but it's WAY better than this feature not hitting 9.5 and people having to hand-code a solution.

Possible syntax:
REINDEX CONCURRENTLY -- Does what current patch does
REINDEX CONCURRENT BUILD -- Builds new files
REINDEX CONCURRENT SWAP -- Swaps new files in

This suffers from the syntax problems I mentioned above, but at least this way it's all limited to one command, and it probably allows a lot more people to use it.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers