Subject: Re: Logical replication in the same cluster

From: Peter Geoghegan <[email-masked]>
Date: Mon, 1 May 2017 11:22:47 -0700

On Fri, Apr 28, 2017 at 9:28 AM, Robert Haas <[email-masked]> wrote:
> On Thu, Apr 27, 2017 at 4:08 AM, Petr Jelinek
> <[email-masked]> wrote:
>> Back when writing the original patch set, I was also playing with the
>> idea of having CREATE SUBSCRIPTION do multiple committed steps in
>> similar fashion to CREATE INDEX CONCURRENTLY but that leaves mess behind
>> on failure which also wasn't very popular outcome.

There is no inherent reason why the CREATE INDEX CONCURRENTLY style of
using multiple transactions makes it necessary to leave a mess behind
in the event of an error or hard crash. Is someone going to get around
to fixing the problem for CREATE INDEX CONCURRENTLY (e.g., having
extra steps to drop the useless index during recovery)? IIRC, this was
always the plan.

-- 
Peter Geoghegan

VMware vCenter Server
https://www.vmware.com/

-- 
Sent via pgsql-hackers mailing list ([email-masked])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

---

From: Andres Freund <[email-masked]>
Date: Mon, 1 May 2017 11:24:56 -0700

> extra steps to drop the useless index during recovery)? IIRC, this was

Doing catalog changes in recovery is frought with problems. Essentially
requires starting one worker per database, before allowing access.

- Andres

---

From: Peter Geoghegan <[email-masked]>
Date: Mon, 1 May 2017 11:31:53 -0700

> requires starting one worker per database, before allowing access.

Do you think it's worth just covering the case where you get an error,
such as a duplicate violation? I imagine that that's the much more
common case.

-- 
Peter Geoghegan

VMware vCenter Server
https://www.vmware.com/

---

From: Andres Freund <[email-masked]>
Date: Mon, 1 May 2017 11:37:25 -0700

> such as a duplicate violation? I imagine that that's the much more

What exactly are you proposing to do?  You mean catching errors in the
creating backend, if it didn't crash?  That doesn't strike me as a good
idea, because it'll push down the likelihood of the issue below where
people will see it, but it'll still be likely enough for it to create
problems.

- Andres

---

From: Peter Geoghegan <[email-masked]>
Date: Mon, 1 May 2017 12:04:13 -0700

On Mon, May 1, 2017 at 11:37 AM, Andres Freund <[email-masked]> wrote:
> creating backend, if it didn't crash?

That is what I meant, though I'm not actually proposing to do anything.

> That doesn't strike me as a good

I was concerned about that too. I have a hard time defending changes
like this to myself, but it doesn't hurt to ask.

-- 
Peter Geoghegan

VMware vCenter Server
https://www.vmware.com/

---

From: Greg Stark <[email-masked]>
Date: Mon, 1 May 2017 22:20:47 +0100

> requires starting one worker per database, before allowing access.

The "plan" was to add more layers PG_TRY and transactions so that if
there was an error during building the index all the remnants of the
failed index build got cleaned up. But when I went tried to actually
do it the problem seemed to metastatize and it was going to require
two or three layers of messy nested PG_TRY and extra transactions.
Perhaps there's a cleaner way to structure it and I should look again.

I don't recall ever having a plan to do anything in recovery. I think
we did talk about why it was hard to mark hash indexes invalid during
recovery which was probably the same problem.

-- 
greg