MIME-Version: 1.0
From: Mark Charsley <mcharsley@google.com>
Date: Wed, 14 Oct 2020 15:20:05 +0100
Message-ID: 
 <CAAf4L0eqm6hozAZyKP005a6UEPnnQcQe-=qznbnOr6B-tQSCAw@mail.gmail.com>
Subject: Possible data race in psycopg2
To: psycopg@postgresql.org
Content-Type: multipart/alternative; boundary="0000000000000adb1a05b1a23b7a"
Precedence: bulk

--0000000000000adb1a05b1a23b7a
Content-Type: text/plain; charset="UTF-8"

Hi. I've started a little project that uses psycopg2, and hit a problem
when running tests under ThreadSanitizer:

WARNING: ThreadSanitizer: data race (pid=9435)
  Read of size 4 at 0x7b44000aae38 by thread T7:
    #0 psyco_conn_cursor psycopg2/psycopg/connection_type.c:88:15 (_
psycopg.abi3.so+0x2a434)

  Previous write of size 4 at 0x7b44000aae38 by thread T6 (mutexes: write
M142280395227573736):
    #0 pq_begin_locked psycopg2/psycopg/pqpath.c:377:22 (_psycopg.abi3.so
+0x38766)
    #1 _pq_execute_sync psycopg2/psycopg/pqpath.c:819:22 (_psycopg.abi3.so
+0x3923e)
    #2 pq_execute psycopg2/psycopg/pqpath.c:947:16 (_psycopg.abi3.so
+0x3abd7)
    #3 _psyco_curs_execute psycopg2/psycopg/cursor_type.c:447:11 (_
psycopg.abi3.so+0x32820)
    #4 curs_execute psycopg2/psycopg/cursor_type.c:491:13 (_psycopg.abi3.so
+0x2f92b)

It's possible that this is caused by an underlying failure in the postgres
server, where thread sanitizer panics about a
safe-except-under-the-most-insanely-optimising-compiler issue, discussed
here:
https://www.postgresql.org/message-id/CAAf4L0d5XpgmX%2BnzT2G_F7EPkJb2JODr_QCzOsSvWBnXxLKYVQ%40mail.gmail.com
which may have killed the server (the tests I'm running compile everything,
including the DB server with tsan), and handling the dead connection causes
the issues mentioned above. But can wiser and more experienced heads than
mine cast their eyes over the code above, and see if there is indeed a
problem with a write to conn_status in pq_begin_lockedand a read from it in
psyco_conn_cursor without a memory barrier in-between?

thanks

Mark

--0000000000000adb1a05b1a23b7a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi. I&#39;ve started a little project that uses psycopg2, =
and hit a problem when running tests under ThreadSanitizer:<div><br></div><=
div>WARNING: ThreadSanitizer: data race (pid=3D9435)<br>=C2=A0 Read of size=
 4 at 0x7b44000aae38 by thread T7:<br>=C2=A0 =C2=A0 #0 psyco_conn_cursor ps=
ycopg2/psycopg/connection_type.c:88:15 (_<a href=3D"http://psycopg.abi3.so"=
>psycopg.abi3.so</a>+0x2a434)<br><br>=C2=A0 Previous write of size 4 at 0x7=
b44000aae38 by thread T6 (mutexes: write M142280395227573736):<br>=C2=A0 =
=C2=A0 #0 pq_begin_locked psycopg2/psycopg/pqpath.c:377:22 (_<a href=3D"htt=
p://psycopg.abi3.so">psycopg.abi3.so</a>+0x38766)<br>=C2=A0 =C2=A0 #1 _pq_e=
xecute_sync psycopg2/psycopg/pqpath.c:819:22 (_<a href=3D"http://psycopg.ab=
i3.so">psycopg.abi3.so</a>+0x3923e)<br>=C2=A0 =C2=A0 #2 pq_execute psycopg2=
/psycopg/pqpath.c:947:16 (_<a href=3D"http://psycopg.abi3.so">psycopg.abi3.=
so</a>+0x3abd7)<br>=C2=A0 =C2=A0 #3 _psyco_curs_execute psycopg2/psycopg/cu=
rsor_type.c:447:11 (_<a href=3D"http://psycopg.abi3.so">psycopg.abi3.so</a>=
+0x32820)<br>=C2=A0 =C2=A0 #4 curs_execute psycopg2/psycopg/cursor_type.c:4=
91:13 (_<a href=3D"http://psycopg.abi3.so">psycopg.abi3.so</a>+0x2f92b)<br>=
<br></div><div>It&#39;s possible that this is caused by an underlying failu=
re in the postgres server, where thread sanitizer panics about a safe-excep=
t-under-the-most-insanely-optimising-compiler issue, discussed here:=C2=A0<=
a href=3D"https://www.postgresql.org/message-id/CAAf4L0d5XpgmX%2BnzT2G_F7EP=
kJb2JODr_QCzOsSvWBnXxLKYVQ%40mail.gmail.com">https://www.postgresql.org/mes=
sage-id/CAAf4L0d5XpgmX%2BnzT2G_F7EPkJb2JODr_QCzOsSvWBnXxLKYVQ%40mail.gmail.=
com</a> which may have killed the server (the tests I&#39;m running compile=
 everything, including the DB server with tsan), and handling the dead conn=
ection causes the issues mentioned above. But can wiser and more experience=
d heads than mine cast their eyes over the code above, and see if there is =
indeed a problem with a write to conn_status in=C2=A0<span style=3D"color:r=
gb(0,0,0);font-family:monospace;font-size:medium;white-space:pre">pq_begin_=
locked</span>and a read from it in=C2=A0<span style=3D"color:rgb(0,0,0);fon=
t-family:monospace;font-size:medium;white-space:pre">psyco_conn_cursor</spa=
n>=C2=A0without a memory barrier in-between?</div><div><br></div><div>thank=
s</div><div><br></div><div>Mark</div></div>

--0000000000000adb1a05b1a23b7a--