pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Kyle Kingsbury <aphyr@jepsen.io>
To: Peter Geoghegan <pg@bowt.ie>
Cc: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: Potential G2-item cycles under serializable isolation
Date: Wed, 3 Jun 2020 10:20:22 -0400
Message-ID: <4e63a8e6-41c9-2a62-b780-2b56005d2253@jepsen.io> (raw)
In-Reply-To: <CAH2-Wz=cYfUSXEUJkWf=xgXdmCTCgqNRKGg1LvbiKLmRvtQTNw@mail.gmail.com>
References: <db7b729d-0226-d162-a126-8a8ab2dc4443@jepsen.io>
	<CAH2-WzkmEz0ax0AG_nc788uRSe_P_RTQwgkveY5VCjDy5y4eUw@mail.gmail.com>
	<11bb3199-c685-1cec-63bb-f92848edbe10@jepsen.io>
	<CAH2-Wzne1fr47aayESfDdfMvjiB+jSfW+xEWggdZuRYa8CL8Nw@mail.gmail.com>
	<CAMotZ_w2-bdQms+rFa-DA_vH+y7A=ky0Gmwu9Y81LT=2ge5TtQ@mail.gmail.com>
	<CAH2-WzkBwwDj38NvdiP4J3ia_POC68C1-FCtXvnDvnzwMASv5w@mail.gmail.com>
	<54c6c503-b939-c4ce-0d54-4f998ac04168@jepsen.io>
	<CAH2-Wzm2F+pNUVeonrS3cZfttJDBmp0nMUiHYG4qAoGQO-rCvA@mail.gmail.com>
	<3150072e-13b8-7655-211b-17898f894e45@jepsen.io>
	<CAH2-WzmD__iOD-zBNDU07ZDQkha36cJ5aXdxB8BYW6=bWR2oqg@mail.gmail.com>
	<b1187ab1-1946-ff78-163d-ff76974e1902@jepsen.io>
	<CAH2-WzkR4qpHOcirynHd1=xoo05T6vYLSzYHV_GWU0wQysJU_A@mail.gmail.com>
	<CAH2-Wz=cYfUSXEUJkWf=xgXdmCTCgqNRKGg1LvbiKLmRvtQTNw@mail.gmail.com>

On 6/2/20 7:17 PM, Peter Geoghegan wrote:
> On Tue, Jun 2, 2020 at 10:24 AM Peter Geoghegan <pg@bowt.ie> wrote:
>> I'd be surprised if your existing test cases needed any adjustment. My
>> guess is that this won't take long.
> You said it takes about a minute in your opening e-mail; how
> consistent is this? I note from the Postgres logs you provided that
> Postgres starts accepting connections at 2020-05-31 18:50:27.580, and
> shows its last log message at  2020-05-31 18:51:29.781 PDT. So it's
> suspiciously close to *exactly* one minute.
I set the test duration to 60 seconds for those runs, but it'll break in as 
little as 10. That's less of a sure thing though. :)
>   Note that
> autovacuum_naptime has as its default '1min'. Your workload probably
> generates a lot of index bloat, which may tend to cause autovacuum to
> want to delete whole B-Tree leaf pages, which impacts predicate
> locking.
With the default (debian) postgresql.conf, which has autovacuum_naptime 
commented out (default 1min), I see anomalies at (just picking a random recent 
test) 8.16 seconds, 9.76 seconds, and 19.6 seconds.  Another run: 28.0 seconds, 
32.3 seconds.
> Could you check what happens when you reduce autovacuum_naptime to
> (say) '5s' in postgresql.conf? Does that change make the G2-item cycle
> issue manifest itself earlier? And can you discern any pattern like
> that yourself?
It doesn't look like setting autovacuum_naptime makes a difference.
> It seems kind of inconvenient to run Jepsen -- I suppose I could use
> Docker or something like that, but I don't have experience with it.
> What do you think is the simplest workflow for somebody that just
> wants to recreate your result on a Debian system?

I am really sorry about that--I know it's not convenient. Jepsen's built for 
testing whole distributed systems, and is probably a bit overkill for testing a 
single Postgres process. I don't have any experience with Docker, but I think 
Docker Compose might be a good option for a single-node system? I apologize--I 
*just* started writing this test against Debian Buster a few days ago, and the 
existing AWS Marketplace and Docker Compose environments are still on Stretch, 
so on top of setting up a Jepsen environment you also gotta do a Debian upgrade. 
:'-O

I'll see about writing a version of the test that doesn't use any of the 
automation, so you can point it at a local postgres instance. Then all you'll 
need is lein and a jdk.

--Kyle






view thread (77+ messages)  latest in thread

Message-ID: <4e63a8e6-41c9-2a62-b780-2b56005d2253@jepsen.io>
Permalink:  ../4e63a8e6-41c9-2a62-b780-2b56005d2253@jepsen.io/
Also on:    postgresql.org/message-id/4e63a8e6-41c9-2a62-b780-2b56005d2253@jepsen.io

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: aphyr@jepsen.io, pg@bowt.ie
  Subject: Re: Potential G2-item cycles under serializable isolation
  In-Reply-To: <4e63a8e6-41c9-2a62-b780-2b56005d2253@jepsen.io>
  References: <db7b729d-0226-d162-a126-8a8ab2dc4443@jepsen.io> <CAH2-WzkmEz0ax0AG_nc788uRSe_P_RTQwgkveY5VCjDy5y4eUw@mail.gmail.com> <11bb3199-c685-1cec-63bb-f92848edbe10@jepsen.io> <CAH2-Wzne1fr47aayESfDdfMvjiB+jSfW+xEWggdZuRYa8CL8Nw@mail.gmail.com> <CAMotZ_w2-bdQms+rFa-DA_vH+y7A=ky0Gmwu9Y81LT=2ge5TtQ@mail.gmail.com> <CAH2-WzkBwwDj38NvdiP4J3ia_POC68C1-FCtXvnDvnzwMASv5w@mail.gmail.com> <54c6c503-b939-c4ce-0d54-4f998ac04168@jepsen.io> <CAH2-Wzm2F+pNUVeonrS3cZfttJDBmp0nMUiHYG4qAoGQO-rCvA@mail.gmail.com> <3150072e-13b8-7655-211b-17898f894e45@jepsen.io> <CAH2-WzmD__iOD-zBNDU07ZDQkha36cJ5aXdxB8BYW6=bWR2oqg@mail.gmail.com> <b1187ab1-1946-ff78-163d-ff76974e1902@jepsen.io> <CAH2-WzkR4qpHOcirynHd1=xoo05T6vYLSzYHV_GWU0wQysJU_A@mail.gmail.com> <CAH2-Wz=cYfUSXEUJkWf=xgXdmCTCgqNRKGg1LvbiKLmRvtQTNw@mail.gmail.com> <4e63a8e6-41c9-2a62-b780-2b56005d2253@jepsen.io>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox