From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
cc: Michael Paquier <michael@paquier.xyz>,
        Postgres hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: REINDEX INDEX results in a crash for an index of pg_class since
 9.6
In-reply-to: <20190501013638.7u7fwr4zktbyyng5@alap3.anarazel.de>
References: <20190430173457.pg6i6urvevse5jer@alap3.anarazel.de>
 <29546.1556647550@sss.pgh.pa.us>
 <20190430182741.s6o7mo5ebmgxxi5x@alap3.anarazel.de>
 <16277.1556649660@sss.pgh.pa.us>
 <20190430190308.kga2z5oz7vab7w22@alap3.anarazel.de>
 <17996.1556651503@sss.pgh.pa.us>
 <20190430221045.6slpi2vm2noda5jb@alap3.anarazel.de>
 <28313.1556663080@sss.pgh.pa.us> <28926.1556664156@sss.pgh.pa.us>
 <20190430225307.lvfvcf2776iywhw4@alap3.anarazel.de>
 <20190501013638.7u7fwr4zktbyyng5@alap3.anarazel.de>
Comments: In-reply-to Andres Freund <andres@anarazel.de>
	message dated "Tue, 30 Apr 2019 18:36:38 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8748.1556685814.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Wed, 01 May 2019 00:43:34 -0400
Message-ID: <8749.1556685814@sss.pgh.pa.us>
Precedence: bulk

Andres Freund <andres@anarazel.de> writes:
> On 2019-04-30 15:53:07 -0700, Andres Freund wrote:
>> I'll move the test into a new "reindex_catalog" test, with a comment
>> explaining that the failure cases necessitating that are somewhere
>> between bugs, ugly warts, an hard to fix edge cases.

> Just pushed that.

locust is kind of unimpressed:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dlocust&dt=3D2019=
-05-01%2003%3A12%3A13

The relevant bit of log is

2019-05-01 05:24:47.527 CEST [97690:429] pg_regress/create_view LOG:  stat=
ement: DROP SCHEMA temp_view_test CASCADE;
2019-05-01 05:24:47.605 CEST [97690:430] pg_regress/create_view LOG:  stat=
ement: DROP SCHEMA testviewschm2 CASCADE;
2019-05-01 05:24:47.858 CEST [97694:1] [unknown] LOG:  connection received=
: host=3D[local]
2019-05-01 05:24:47.863 CEST [97694:2] [unknown] LOG:  connection authoriz=
ed: user=3Dpgbuildfarm database=3Dregression
2019-05-01 05:24:47.878 CEST [97694:3] pg_regress/reindex_catalog LOG:  st=
atement: REINDEX TABLE pg_class;
2019-05-01 05:24:48.887 CEST [97694:4] pg_regress/reindex_catalog ERROR:  =
deadlock detected
2019-05-01 05:24:48.887 CEST [97694:5] pg_regress/reindex_catalog DETAIL: =
 Process 97694 waits for ShareLock on transaction 2559; blocked by process=
 97690.
	Process 97690 waits for RowExclusiveLock on relation 1259 of database 163=
87; blocked by process 97694.
	Process 97694: REINDEX TABLE pg_class;
	Process 97690: DROP SCHEMA testviewschm2 CASCADE;
2019-05-01 05:24:48.887 CEST [97694:6] pg_regress/reindex_catalog HINT:  S=
ee server log for query details.
2019-05-01 05:24:48.887 CEST [97694:7] pg_regress/reindex_catalog CONTEXT:=
  while checking uniqueness of tuple (12,71) in relation "pg_class"
2019-05-01 05:24:48.887 CEST [97694:8] pg_regress/reindex_catalog STATEMEN=
T:  REINDEX TABLE pg_class;
2019-05-01 05:24:48.904 CEST [97690:431] pg_regress/create_view LOG:  disc=
onnection: session time: 0:00:03.748 user=3Dpgbuildfarm database=3Dregress=
ion host=3D[local]

which is mighty confusing at first glance, but I think the explanation is
that what the postmaster is reporting is process 97690's *latest* query,
not what it's currently doing.  What it's really currently doing at the
moment of the deadlock is cleaning out its temporary schema after the
client disconnected.  So this says you were careless about where to insert
the reindex_catalog test in the test schedule: it can't be after anything
that creates any temp objects.  That seems like kind of a problem :-(.
We could put it second, after the tablespace test, but that would mean
that we're reindexing after very little churn has happened in the
catalogs, which doesn't seem like much of a stress test.

Another fairly interesting thing is that this log includes the telltale

2019-05-01 05:24:48.887 CEST [97694:7] pg_regress/reindex_catalog CONTEXT:=
  while checking uniqueness of tuple (12,71) in relation "pg_class"

Why did I have to dig to find that information in HEAD?  Have we lost
some useful context reporting?  (Note this run is in the v10 branch.)

			regards, tom lane