pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Avoid spurious deadlocks when upgrading a tuple lock
32+ messages / 6 participants
[nested] [flat]
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-13 21:32 Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-13 21:32 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for X
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for X
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once X releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after X finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/de87a084c0a5ac927017cd0834b33a932651cfc9
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 84 +++++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 150 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 57 ++++++++
5 files changed, 281 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2019-06-14 14:10 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 1 reply; 32+ messages in thread
From: Tom Lane @ 2019-06-14 14:10 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: pgsql-committers@lists.postgresql.org
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> Avoid spurious deadlocks when upgrading a tuple lock
I'm now getting
heapam.c: In function 'heap_lock_tuple':
heapam.c:4041: warning: 'skip_tuple_lock' may be used uninitialized in this function
Please fix.
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-06-14 15:11 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-14 15:11 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org, Etsuro Fujita <etsuro.fujita@gmail.com>
On 2019-Jun-14, Tom Lane wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> > Avoid spurious deadlocks when upgrading a tuple lock
>
> I'm now getting
>
> heapam.c: In function 'heap_lock_tuple':
> heapam.c:4041: warning: 'skip_tuple_lock' may be used uninitialized in this function
Hm, I don't get that warning. Does this patch silence it, please?
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-14 15:28 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 32+ messages in thread
From: Tom Lane @ 2019-06-14 15:28 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: pgsql-committers@lists.postgresql.org, Etsuro Fujita <etsuro.fujita@gmail.com>
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2019-Jun-14, Tom Lane wrote:
>> I'm now getting
>> heapam.c: In function 'heap_lock_tuple':
>> heapam.c:4041: warning: 'skip_tuple_lock' may be used uninitialized in this function
> Hm, I don't get that warning. Does this patch silence it, please?
Uh, no patch attached? But initializing the variable where it's
declared would certainly silence it.
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-06-14 15:32 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:37 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 2 replies; 32+ messages in thread
From: Tom Lane @ 2019-06-14 15:32 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: pgsql-committers@lists.postgresql.org, Etsuro Fujita <etsuro.fujita@gmail.com>
I wrote:
>> Hm, I don't get that warning. Does this patch silence it, please?
> Uh, no patch attached? But initializing the variable where it's
> declared would certainly silence it.
BTW, after looking around a bit I wonder if this complaint isn't
exposing an actual logic bug. Shouldn't skip_tuple_lock have
a lifetime similar to first_time?
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-06-14 15:37 ` Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-14 15:37 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org, Etsuro Fujita <etsuro.fujita@gmail.com>
On 2019-Jun-14, Tom Lane wrote:
> I wrote:
> >> Hm, I don't get that warning. Does this patch silence it, please?
>
> > Uh, no patch attached? But initializing the variable where it's
> > declared would certainly silence it.
>
> BTW, after looking around a bit I wonder if this complaint isn't
> exposing an actual logic bug. Shouldn't skip_tuple_lock have
> a lifetime similar to first_time?
I think you're right. I should come up with a test case that exercises
that case.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-06-15 03:43 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 16:25 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 2 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-15 03:43 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@lists.postgresql.org; alexk@hintbits.com
On 2019-Jun-14, Tom Lane wrote:
> I wrote:
> >> Hm, I don't get that warning. Does this patch silence it, please?
>
> > Uh, no patch attached? But initializing the variable where it's
> > declared would certainly silence it.
>
> BTW, after looking around a bit I wonder if this complaint isn't
> exposing an actual logic bug. Shouldn't skip_tuple_lock have
> a lifetime similar to first_time?
I think there are worse problems here. I tried the attached isolation
spec. Note that the only difference in the two permutations is that s0
finishes earlier in one than the other; yet the first one works fine and
the second one hangs until killed by the 180s timeout. (s3 isn't
released for a reason I'm not sure I understand.)
I don't think I'm going to have time to investigate this deeply over the
weekend, so I think the safest course of action is to revert this for
next week's set.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
setup
{
drop table if exists tlu_job;
create table tlu_job (id integer primary key, name text);
insert into tlu_job values(1, 'a');
}
teardown
{
drop table tlu_job;
}
session "s0"
setup { begin; }
step "s0_forupdate" { select id from tlu_job where id = 1 for update; }
step "s0_commit" { commit; }
session "s1"
setup { begin; }
step "s1_keyshare" { select id from tlu_job where id = 1 for key share;}
step "s1_share" { select id from tlu_job where id = 1 for share; }
step "s1_update" { update tlu_job set name = 'b' where id = 1; }
step "s1_delete" { delete from tlu_job where id = 1; }
step "s1_rollback" { rollback; }
step "s1_commit" { commit; }
session "s2"
setup { begin; }
step "s2_for_update" { select id from tlu_job where id = 1 for update; }
step "s2_update" { update tlu_job set name = 'b' where id = 1; }
step "s2_delete" { delete from tlu_job where id = 1; }
step "s2_rollback" { rollback; }
step "s2_commit" { commit; }
session "s3"
setup { begin; }
step "s3_keyshare" { select id from tlu_job where id = 1 for key share; }
step "s3_share" { select id from tlu_job where id = 1 for share; }
step "s3_for_update" { select id from tlu_job where id = 1 for update; }
step "s3_update" { update tlu_job set name = 'c' where id = 1; }
step "s3_delete" { delete from tlu_job where id = 1; }
step "s3_rollback" { rollback; }
step "s3_commit" { commit; }
permutation "s0_forupdate" "s1_share" "s2_for_update" "s0_commit" "s3_share" "s3_for_update" "s1_rollback" "s3_rollback" "s2_rollback"
permutation "s0_forupdate" "s1_share" "s2_for_update" "s3_share" "s0_commit" "s3_for_update" "s1_rollback" "s3_rollback" "s2_rollback"
Attachments:
[text/plain] with_s0.spec (1.6K, ../../20190615034337.GA24560@alvherre.pgsql/2-with_s0.spec)
download | inline:
setup
{
drop table if exists tlu_job;
create table tlu_job (id integer primary key, name text);
insert into tlu_job values(1, 'a');
}
teardown
{
drop table tlu_job;
}
session "s0"
setup { begin; }
step "s0_forupdate" { select id from tlu_job where id = 1 for update; }
step "s0_commit" { commit; }
session "s1"
setup { begin; }
step "s1_keyshare" { select id from tlu_job where id = 1 for key share;}
step "s1_share" { select id from tlu_job where id = 1 for share; }
step "s1_update" { update tlu_job set name = 'b' where id = 1; }
step "s1_delete" { delete from tlu_job where id = 1; }
step "s1_rollback" { rollback; }
step "s1_commit" { commit; }
session "s2"
setup { begin; }
step "s2_for_update" { select id from tlu_job where id = 1 for update; }
step "s2_update" { update tlu_job set name = 'b' where id = 1; }
step "s2_delete" { delete from tlu_job where id = 1; }
step "s2_rollback" { rollback; }
step "s2_commit" { commit; }
session "s3"
setup { begin; }
step "s3_keyshare" { select id from tlu_job where id = 1 for key share; }
step "s3_share" { select id from tlu_job where id = 1 for share; }
step "s3_for_update" { select id from tlu_job where id = 1 for update; }
step "s3_update" { update tlu_job set name = 'c' where id = 1; }
step "s3_delete" { delete from tlu_job where id = 1; }
step "s3_rollback" { rollback; }
step "s3_commit" { commit; }
permutation "s0_forupdate" "s1_share" "s2_for_update" "s0_commit" "s3_share" "s3_for_update" "s1_rollback" "s3_rollback" "s2_rollback"
permutation "s0_forupdate" "s1_share" "s2_for_update" "s3_share" "s0_commit" "s3_for_update" "s1_rollback" "s3_rollback" "s2_rollback"
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-15 16:25 ` Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 32+ messages in thread
From: Tom Lane @ 2019-06-15 16:25 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: pgsql-hackers@lists.postgresql.org; alexk@hintbits.com
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2019-Jun-14, Tom Lane wrote:
>> BTW, after looking around a bit I wonder if this complaint isn't
>> exposing an actual logic bug. Shouldn't skip_tuple_lock have
>> a lifetime similar to first_time?
> I think there are worse problems here. I tried the attached isolation
> spec. Note that the only difference in the two permutations is that s0
> finishes earlier in one than the other; yet the first one works fine and
> the second one hangs until killed by the 180s timeout. (s3 isn't
> released for a reason I'm not sure I understand.)
Ugh.
> I don't think I'm going to have time to investigate this deeply over the
> weekend, so I think the safest course of action is to revert this for
> next week's set.
+1. This is an old bug, we don't have to improve it for this release.
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-15 17:01 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
1 sibling, 1 reply; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-15 17:01 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@lists.postgresql.org; alexk@hintbits.com
On 2019-Jun-14, Alvaro Herrera wrote:
> I think there are worse problems here. I tried the attached isolation
> spec. Note that the only difference in the two permutations is that s0
> finishes earlier in one than the other; yet the first one works fine and
> the second one hangs until killed by the 180s timeout. (s3 isn't
> released for a reason I'm not sure I understand.)
Actually, those behaviors both seem correct to me now that I look
closer. So this was a false alarm. In the code before de87a084c0, the
first permutation deadlocks, and the second permutation hangs. The only
behavior change is that the first one no longer deadlocks, which is the
desired change.
I'm still trying to form a case to exercise the case of skip_tuple_lock
having the wrong lifetime.
The fact that both permutations behave differently, even though the
only difference is where s0 commits relative to the s3_share step, is an
artifact of our unusual tuple locking implementation.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-15 22:12 ` Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 1 reply; 32+ messages in thread
From: Oleksii Kliukin @ 2019-06-15 22:12 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> On 2019-Jun-14, Alvaro Herrera wrote:
>
>> I think there are worse problems here. I tried the attached isolation
>> spec. Note that the only difference in the two permutations is that s0
>> finishes earlier in one than the other; yet the first one works fine and
>> the second one hangs until killed by the 180s timeout. (s3 isn't
>> released for a reason I'm not sure I understand.)
>
> Actually, those behaviors both seem correct to me now that I look
> closer. So this was a false alarm. In the code before de87a084c0, the
> first permutation deadlocks, and the second permutation hangs. The only
> behavior change is that the first one no longer deadlocks, which is the
> desired change.
>
> I'm still trying to form a case to exercise the case of skip_tuple_lock
> having the wrong lifetime.
Hm… I think it was an oversight from my part not to give skip_lock_tuple the
same lifetime as have_tuple_lock or first_time (also initializing it to
false at the same time). Even if now it might not break anything in an
obvious way, a backward jump to l3 label will leave skip_lock_tuple
uninitialized, making it very dangerous for any future code that will rely
on its value.
> The fact that both permutations behave differently, even though the
> only difference is where s0 commits relative to the s3_share step, is an
> artifact of our unusual tuple locking implementation.
Cheers,
Oleksii
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
@ 2019-06-15 22:47 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-18 18:13 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
0 siblings, 2 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-15 22:47 UTC (permalink / raw)
To: Oleksii Kliukin <alexk@hintbits.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
On 2019-Jun-16, Oleksii Kliukin wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>
> > On 2019-Jun-14, Alvaro Herrera wrote:
> >
> >> I think there are worse problems here. I tried the attached isolation
> >> spec. Note that the only difference in the two permutations is that s0
> >> finishes earlier in one than the other; yet the first one works fine and
> >> the second one hangs until killed by the 180s timeout. (s3 isn't
> >> released for a reason I'm not sure I understand.)
> >
> > Actually, those behaviors both seem correct to me now that I look
> > closer. So this was a false alarm. In the code before de87a084c0, the
> > first permutation deadlocks, and the second permutation hangs. The only
> > behavior change is that the first one no longer deadlocks, which is the
> > desired change.
> >
> > I'm still trying to form a case to exercise the case of skip_tuple_lock
> > having the wrong lifetime.
>
> Hm… I think it was an oversight from my part not to give skip_lock_tuple the
> same lifetime as have_tuple_lock or first_time (also initializing it to
> false at the same time). Even if now it might not break anything in an
> obvious way, a backward jump to l3 label will leave skip_lock_tuple
> uninitialized, making it very dangerous for any future code that will rely
> on its value.
But that's not the danger ... with the current coding, it's initialized
to false every time through that block; that means the tuple lock will
never be skipped if we jump back to l3. So the danger is that the first
iteration sets the variable, then jumps back; second iteration
initializes the variable again, so instead of skipping the lock, it
takes it, causing a spurious deadlock.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-16 19:04 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-18 16:26 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 2 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-16 19:04 UTC (permalink / raw)
To: Oleksii Kliukin <alexk@hintbits.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
On 2019-Jun-15, Alvaro Herrera wrote:
> But that's not the danger ... with the current coding, it's initialized
> to false every time through that block; that means the tuple lock will
> never be skipped if we jump back to l3. So the danger is that the first
> iteration sets the variable, then jumps back; second iteration
> initializes the variable again, so instead of skipping the lock, it
> takes it, causing a spurious deadlock.
So, I'm too lazy today to generate a case that fully reproduces the
deadlock, because you need to stall 's2' a little bit using the
well-known advisory lock trick, but this one hits the code that would
re-initialize the variable.
I'm going to push the change of lifetime of the variable for now.
setup
{
drop table if exists tlu_job;
create table tlu_job (id integer primary key, name text);
insert into tlu_job values(1, 'a');
}
teardown
{
drop table tlu_job;
}
session "s0"
setup { begin; set deadlock_timeout=1}
step "s0_fornokeyupdate" { select id from tlu_job where id = 1 for no key update; }
step "s0_update" { update tlu_job set name = 's0' where id = 1; }
step "s0_commit" { commit; }
session "s1"
setup { begin; set deadlock_timeout=1}
step "s1_for_key_share" { select id from tlu_job where id = 1 for key share; }
step "s1_for_update" { select id from tlu_job where id = 1 for update; }
step "s1_rollback" { rollback; }
session "s2"
setup { begin; set deadlock_timeout=1}
step "s2_for_key_share" { select id from tlu_job where id = 1 for key share; }
step "s2_for_share" { select id from tlu_job where id = 1 for share; }
step "s2_rollback" { rollback; }
session "s3"
setup { begin; set deadlock_timeout=1}
step "s3_update" { update tlu_job set name = 'c' where id = 1; }
step "s3_rollback" { rollback; }
permutation "s1_for_key_share" "s2_for_key_share" "s0_fornokeyupdate" "s2_for_share" "s0_update" "s0_commit" "s1_rollback" "s2_rollback" "s3_rollback"
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-17 00:02 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 00:40 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 1 reply; 32+ messages in thread
From: Tom Lane @ 2019-06-17 00:02 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Oleksii Kliukin <alexk@hintbits.com>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> I'm going to push the change of lifetime of the variable for now.
If you're going to push anything before tomorrow's wrap, please do it
*now* not later. We're running out of time to get a full sample of
buildfarm results.
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-06-17 00:40 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 01:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-17 00:40 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Oleksii Kliukin <alexk@hintbits.com>; pgsql-hackers@lists.postgresql.org
On 2019-Jun-16, Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > I'm going to push the change of lifetime of the variable for now.
>
> If you're going to push anything before tomorrow's wrap, please do it
> *now* not later. We're running out of time to get a full sample of
> buildfarm results.
Yeah, I had to bail out earlier today, so the only thing I'm confident
pushing now is a revert.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 00:40 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-17 01:10 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 01:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 32+ messages in thread
From: Tom Lane @ 2019-06-17 01:10 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Oleksii Kliukin <alexk@hintbits.com>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2019-Jun-16, Tom Lane wrote:
>> If you're going to push anything before tomorrow's wrap, please do it
>> *now* not later. We're running out of time to get a full sample of
>> buildfarm results.
> Yeah, I had to bail out earlier today, so the only thing I'm confident
> pushing now is a revert.
Yeah, let's do that. I don't want to risk shipping broken code.
We can try again for the next updates.
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 00:40 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 01:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-06-17 01:32 ` Michael Paquier <michael@paquier.xyz>
2019-06-17 02:25 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 1 reply; 32+ messages in thread
From: Michael Paquier @ 2019-06-17 01:32 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Oleksii Kliukin <alexk@hintbits.com>; pgsql-hackers@lists.postgresql.org
On Sun, Jun 16, 2019 at 09:10:13PM -0400, Tom Lane wrote:
> Yeah, let's do that. I don't want to risk shipping broken code.
> We can try again for the next updates.
Could you revert asap please then?
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20190617013211.GB3153@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 00:40 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 01:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 01:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Michael Paquier <michael@paquier.xyz>
@ 2019-06-17 02:25 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 02:27 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-17 02:25 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Oleksii Kliukin <alexk@hintbits.com>; pgsql-hackers@lists.postgresql.org
On 2019-Jun-17, Michael Paquier wrote:
> On Sun, Jun 16, 2019 at 09:10:13PM -0400, Tom Lane wrote:
> > Yeah, let's do that. I don't want to risk shipping broken code.
> > We can try again for the next updates.
>
> Could you revert asap please then?
Done.
I initially thought to keep the test in place, but then realized it
might be unstable, so I removed that too.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 00:40 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 01:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 01:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Michael Paquier <michael@paquier.xyz>
2019-06-17 02:25 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-17 02:27 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 02:44 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 32+ messages in thread
From: Tom Lane @ 2019-06-17 02:27 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Michael Paquier <michael@paquier.xyz>; Oleksii Kliukin <alexk@hintbits.com>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2019-Jun-17, Michael Paquier wrote:
>> Could you revert asap please then?
> Done.
Thanks.
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 00:40 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 01:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 01:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Michael Paquier <michael@paquier.xyz>
2019-06-17 02:25 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 02:27 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-06-17 02:44 ` Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 32+ messages in thread
From: Michael Paquier @ 2019-06-17 02:44 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Oleksii Kliukin <alexk@hintbits.com>; pgsql-hackers@lists.postgresql.org
On Sun, Jun 16, 2019 at 10:27:25PM -0400, Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>> On 2019-Jun-17, Michael Paquier wrote:
>>> Could you revert asap please then?
>
>> Done.
>
> Thanks.
Thanks, Alvaro.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20190617024420.GA18917@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-18 16:26 ` Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-18 16:26 UTC (permalink / raw)
To: Oleksii Kliukin <alexk@hintbits.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
On 2019-Jun-16, Alvaro Herrera wrote:
> So, I'm too lazy today to generate a case that fully reproduces the
> deadlock, because you need to stall 's2' a little bit using the
> well-known advisory lock trick, but this one hits the code that would
> re-initialize the variable.
Here's such a case. I was unable to reproduce the condition with a
smaller sequence of commands. This one does hit the deadlock when used
with the previous code, as expected; with the fixed code (ie.
skip_tuple_lock in the outer scope and same lifetime as "first_time")
then it works fine, no deadlock.
I'm going to push the fixed commit this afternoon, including this as an
additional permutation in the spec file.
setup
{
drop table if exists tlu_job;
create table tlu_job (id integer primary key, name text);
insert into tlu_job values(1, 'a');
}
teardown
{
drop table tlu_job;
}
session "s0"
setup { begin; }
step "s0_keyshare" { select id from tlu_job where id = 1 for key share; }
step "s0_share" { select id from tlu_job where id = 1 for share; }
step "s0_rollback" { rollback; }
session "s1"
setup { begin; }
step "s1_keyshare" { select id from tlu_job where id = 1 for key share; }
step "s1_savept_e" { savepoint s1_e; }
step "s1_share" { select id from tlu_job where id = 1 for share; }
step "s1_savept_f" { savepoint s1_f; }
step "s1_fornokeyupd" { select id from tlu_job where id = 1 for no key update; }
step "s1_rollback_f" { rollback to s1_f; }
step "s1_rollback_e" { rollback to s1_e; }
step "s1_rollback" { rollback; }
session "s2"
setup { begin; }
step "s2_keyshare" { select id from tlu_job where id = 1 for key share; }
step "s2_fornokeyupd" { select id from tlu_job where id = 1 for no key update; }
step "s2_rollback" { rollback; }
session "s3"
setup { begin; }
step "s3_for_update" { select id from tlu_job where id = 1 for update; }
step "s3_rollback" { rollback; }
permutation "s1_keyshare" "s3_for_update" "s2_keyshare" "s1_savept_e" "s1_share" "s1_savept_f" "s1_fornokeyupd" "s2_fornokeyupd" "s0_keyshare" "s1_rollback_f" "s0_share" "s1_rollback_e" "s1_rollback" "s2_rollback" "s0_rollback" "s3_rollback"
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-18 18:13 ` Oleksii Kliukin <alexk@hintbits.com>
2019-06-18 22:25 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 1 reply; 32+ messages in thread
From: Oleksii Kliukin @ 2019-06-18 18:13 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> On 2019-Jun-16, Oleksii Kliukin wrote:
>
>> Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>>
>>> On 2019-Jun-14, Alvaro Herrera wrote:
>>>
>>>> I think there are worse problems here. I tried the attached isolation
>>>> spec. Note that the only difference in the two permutations is that s0
>>>> finishes earlier in one than the other; yet the first one works fine and
>>>> the second one hangs until killed by the 180s timeout. (s3 isn't
>>>> released for a reason I'm not sure I understand.)
>>>
>>> Actually, those behaviors both seem correct to me now that I look
>>> closer. So this was a false alarm. In the code before de87a084c0, the
>>> first permutation deadlocks, and the second permutation hangs. The only
>>> behavior change is that the first one no longer deadlocks, which is the
>>> desired change.
>>>
>>> I'm still trying to form a case to exercise the case of skip_tuple_lock
>>> having the wrong lifetime.
>>
>> Hm… I think it was an oversight from my part not to give skip_lock_tuple the
>> same lifetime as have_tuple_lock or first_time (also initializing it to
>> false at the same time). Even if now it might not break anything in an
>> obvious way, a backward jump to l3 label will leave skip_lock_tuple
>> uninitialized, making it very dangerous for any future code that will rely
>> on its value.
>
> But that's not the danger ... with the current coding, it's initialized
> to false every time through that block; that means the tuple lock will
> never be skipped if we jump back to l3. So the danger is that the first
> iteration sets the variable, then jumps back; second iteration
> initializes the variable again, so instead of skipping the lock, it
> takes it, causing a spurious deadlock.
Sorry, I was confused, as I was looking only at
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=de87a084c0a5ac927017cd0834b33a932...
without taking your subsequent commit that silences compiler warnings at
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=3da73d6839dc47f1f47ca57974bf28e5a...
into consideration. With that commit, the danger is indeed in resetting the
skip mechanism on each jump and potentially causing deadlocks.
Cheers,
Oleksii
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-18 18:13 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
@ 2019-06-18 22:25 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-19 14:08 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
0 siblings, 1 reply; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-18 22:25 UTC (permalink / raw)
To: Oleksii Kliukin <alexk@hintbits.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
On 2019-Jun-18, Oleksii Kliukin wrote:
> Sorry, I was confused, as I was looking only at
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=de87a084c0a5ac927017cd0834b33a932...
>
> without taking your subsequent commit that silences compiler warnings at
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=3da73d6839dc47f1f47ca57974bf28e5a...
> into consideration. With that commit, the danger is indeed in resetting the
> skip mechanism on each jump and potentially causing deadlocks.
Yeah, I understand the confusion.
Anyway, as bugs go, this one seems pretty benign. It would result in a
unexplained deadlock, very rarely, and only for people who use a very
strange locking pattern that includes (row-level) lock upgrades. I
think it also requires aborted savepoints too, though I don't rule out
the possibility that there might be a way to reproduce this without
that.
I pushed the patch again just now, with the new permutation.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 03:43 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 17:01 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-18 18:13 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Oleksii Kliukin <alexk@hintbits.com>
2019-06-18 22:25 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-06-19 14:08 ` Oleksii Kliukin <alexk@hintbits.com>
0 siblings, 0 replies; 32+ messages in thread
From: Oleksii Kliukin @ 2019-06-19 14:08 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> On 2019-Jun-18, Oleksii Kliukin wrote:
>
>> Sorry, I was confused, as I was looking only at
>> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=de87a084c0a5ac927017cd0834b33a932...
>>
>> without taking your subsequent commit that silences compiler warnings at
>> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=3da73d6839dc47f1f47ca57974bf28e5a...
>> into consideration. With that commit, the danger is indeed in resetting the
>> skip mechanism on each jump and potentially causing deadlocks.
>
> Yeah, I understand the confusion.
>
> Anyway, as bugs go, this one seems pretty benign. It would result in a
> unexplained deadlock, very rarely, and only for people who use a very
> strange locking pattern that includes (row-level) lock upgrades. I
> think it also requires aborted savepoints too, though I don't rule out
> the possibility that there might be a way to reproduce this without
> that.
>
> I pushed the patch again just now, with the new permutation.
Thank you very much for working on it and committing the fix!
Cheers,
Oleksii
^ permalink raw reply [nested|flat] 32+ messages in thread
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-13 21:32 Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-13 22:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tatsuo Ishii <ishii@sraoss.co.jp>
0 siblings, 1 reply; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-13 21:32 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for X
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for X
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once X releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after X finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/85600b7b5da42c5166eb1188c173beb3cc356178
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 84 +++++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 150 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 57 ++++++++
5 files changed, 281 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2019-06-13 22:32 ` Tatsuo Ishii <ishii@sraoss.co.jp>
2019-06-14 00:27 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 1 reply; 32+ messages in thread
From: Tatsuo Ishii @ 2019-06-13 22:32 UTC (permalink / raw)
To: alvherre@alvh.no-ip.org; +Cc: pgsql-committers@lists.postgresql.org
I am a little bit confused. Here T1 and X are the same transaction?
Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp
> Avoid spurious deadlocks when upgrading a tuple lock
>
> When two (or more) transactions are waiting for transaction T1 to release a
> tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
> spurious deadlock can be reported among the waiting transactions when T1
> finishes. The simplest example case seems to be:
>
> T1: select id from job where name = 'a' for key share;
> Y: select id from job where name = 'a' for update; -- starts waiting for X
> Z: select id from job where name = 'a' for key share;
> T1: update job set name = 'b' where id = 1;
> Z: update job set name = 'c' where id = 1; -- starts waiting for X
> T1: rollback;
>
> At this point, transaction Y is rolled back on account of a deadlock: Y
> holds the heavyweight tuple lock and is waiting for the Xmax to be released,
> while Z holds part of the multixact and tries to acquire the heavyweight
> lock (per protocol) and goes to sleep; once X releases its part of the
> multixact, Z is awakened only to be put back to sleep on the heavyweight
> lock that Y is holding while sleeping. Kaboom.
>
> This can be avoided by having Z skip the heavyweight lock acquisition. As
> far as I can see, the biggest downside is that if there are multiple Z
> transactions, the order in which they resume after X finishes is not
> guaranteed.
>
> Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
> work there (because isolationtester is not smart enough), so I'm not going
> to risk it.
>
> Author: Oleksii Kliukin
> Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
>
> Branch
> ------
> REL_11_STABLE
>
> Details
> -------
> https://git.postgresql.org/pg/commitdiff/85600b7b5da42c5166eb1188c173beb3cc356178
>
> Modified Files
> --------------
> src/backend/access/heap/README.tuplock | 10 ++
> src/backend/access/heap/heapam.c | 84 +++++++++---
> .../expected/tuplelock-upgrade-no-deadlock.out | 150 +++++++++++++++++++++
> src/test/isolation/isolation_schedule | 1 +
> .../specs/tuplelock-upgrade-no-deadlock.spec | 57 ++++++++
> 5 files changed, 281 insertions(+), 21 deletions(-)
>
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-13 22:32 ` Re: pgsql: Avoid spurious deadlocks when upgrading a tuple lock Tatsuo Ishii <ishii@sraoss.co.jp>
@ 2019-06-14 00:27 ` Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-14 00:27 UTC (permalink / raw)
To: Tatsuo Ishii <ishii@sraoss.co.jp>; +Cc: pgsql-committers@lists.postgresql.org
On 2019-Jun-14, Tatsuo Ishii wrote:
> I am a little bit confused. Here T1 and X are the same transaction?
Argh :-( Yes, X and T1 are the same transaction.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 32+ messages in thread
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-13 21:32 Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-13 21:32 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for X
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for X
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once X releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after X finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Branch
------
REL_10_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/14a91a8fc279e9ea438f73049d93481d99b5b9f9
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 84 +++++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 150 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 57 ++++++++
5 files changed, 281 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-13 21:32 Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-13 21:32 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for X
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for X
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once X releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after X finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Branch
------
REL9_6_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/cb2398d8069ea5c93c005a48b7778338602b658b
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 84 +++++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 150 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 57 ++++++++
5 files changed, 281 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-18 22:24 Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-18 22:24 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
This puts back reverted commit de87a084c0a5, with some bug fixes.
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for T1
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for T1
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once T1 releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after T1 finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Discussion: https://postgr.es/m/2815.1560521451@sss.pgh.pa.us
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/8b21b416ed621501db3be38817c298c57470524f
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 83 ++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 195 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 72 ++++++++
5 files changed, 340 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-18 22:24 Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-18 22:24 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
This puts back reverted commit de87a084c0a5, with some bug fixes.
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for T1
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for T1
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once T1 releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after T1 finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Discussion: https://postgr.es/m/2815.1560521451@sss.pgh.pa.us
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/5246d3e791033267516b66ce3518f9cc6c23c261
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 83 ++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 195 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 72 ++++++++
5 files changed, 340 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-18 22:24 Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-18 22:24 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
This puts back reverted commit de87a084c0a5, with some bug fixes.
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for T1
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for T1
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once T1 releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after T1 finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Discussion: https://postgr.es/m/2815.1560521451@sss.pgh.pa.us
Branch
------
REL_10_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/0772d8a00eb88b50ac81f379955d30660cecdeeb
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 83 ++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 195 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 72 ++++++++
5 files changed, 340 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
* pgsql: Avoid spurious deadlocks when upgrading a tuple lock
@ 2019-06-18 22:24 Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 32+ messages in thread
From: Alvaro Herrera @ 2019-06-18 22:24 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Avoid spurious deadlocks when upgrading a tuple lock
This puts back reverted commit de87a084c0a5, with some bug fixes.
When two (or more) transactions are waiting for transaction T1 to release a
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a
spurious deadlock can be reported among the waiting transactions when T1
finishes. The simplest example case seems to be:
T1: select id from job where name = 'a' for key share;
Y: select id from job where name = 'a' for update; -- starts waiting for T1
Z: select id from job where name = 'a' for key share;
T1: update job set name = 'b' where id = 1;
Z: update job set name = 'c' where id = 1; -- starts waiting for T1
T1: rollback;
At this point, transaction Y is rolled back on account of a deadlock: Y
holds the heavyweight tuple lock and is waiting for the Xmax to be released,
while Z holds part of the multixact and tries to acquire the heavyweight
lock (per protocol) and goes to sleep; once T1 releases its part of the
multixact, Z is awakened only to be put back to sleep on the heavyweight
lock that Y is holding while sleeping. Kaboom.
This can be avoided by having Z skip the heavyweight lock acquisition. As
far as I can see, the biggest downside is that if there are multiple Z
transactions, the order in which they resume after T1 finishes is not
guaranteed.
Backpatch to 9.6. The patch applies cleanly on 9.5, but the new tests don't
work there (because isolationtester is not smart enough), so I'm not going
to risk it.
Author: Oleksii Kliukin
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com
Discussion: https://postgr.es/m/2815.1560521451@sss.pgh.pa.us
Branch
------
REL9_6_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/0ba35c7c9f8e16bdd33893d7236d4fb7fb633b0f
Modified Files
--------------
src/backend/access/heap/README.tuplock | 10 ++
src/backend/access/heap/heapam.c | 83 ++++++---
.../expected/tuplelock-upgrade-no-deadlock.out | 195 +++++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/tuplelock-upgrade-no-deadlock.spec | 72 ++++++++
5 files changed, 340 insertions(+), 21 deletions(-)
^ permalink raw reply [nested|flat] 32+ messages in thread
end of thread, other threads:[~2019-06-19 14:08 UTC | newest]
Thread overview: 32+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-14 14:10 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:11 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-14 15:28 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:32 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-14 15:37 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 03:43 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 16:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-15 17:01 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-15 22:12 ` Oleksii Kliukin <alexk@hintbits.com>
2019-06-15 22:47 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-16 19:04 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 00:02 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 00:40 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 01:10 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 01:32 ` Michael Paquier <michael@paquier.xyz>
2019-06-17 02:25 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-17 02:27 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-06-17 02:44 ` Michael Paquier <michael@paquier.xyz>
2019-06-18 16:26 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-18 18:13 ` Oleksii Kliukin <alexk@hintbits.com>
2019-06-18 22:25 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-19 14:08 ` Oleksii Kliukin <alexk@hintbits.com>
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-13 22:32 ` Tatsuo Ishii <ishii@sraoss.co.jp>
2019-06-14 00:27 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-13 21:32 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-18 22:24 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-18 22:24 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-18 22:24 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-06-18 22:24 pgsql: Avoid spurious deadlocks when upgrading a tuple lock Alvaro Herrera <alvherre@alvh.no-ip.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox