From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: logical decoding / rewrite map vs. maxAllocatedDescs
Date: Tue, 14 Aug 2018 16:05:29 +0200
Message-ID: <9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com> (raw)
In-Reply-To: <f8221cf4-719c-ac4e-56bf-3be106c41d92@2ndquadrant.com>
References: <bc07e92b-d3b9-fabc-2993-edc0ef89be67@2ndquadrant.com>
<20180813144630.hmzdrbviwao64ube@alvherre.pgsql>
<20180813144946.fn4zpdwpucx43emz@alap3.anarazel.de>
<f8221cf4-719c-ac4e-56bf-3be106c41d92@2ndquadrant.com>
On 08/14/2018 01:49 PM, Tomas Vondra wrote:
> On 08/13/2018 04:49 PM, Andres Freund wrote:
>> Hi,
>>
>> On 2018-08-13 11:46:30 -0300, Alvaro Herrera wrote:
>>> On 2018-Aug-11, Tomas Vondra wrote:
>>>
>>>> Hmmm, it's difficult to compare "bt full" output, but my backtraces
>>>> look
>>>> somewhat different (and all the backtraces I'm seeing are 100% exactly
>>>> the same). Attached for comparison.
>>>
>>> Hmm, looks similar enough to me -- at the bottom you have the executor
>>> doing its thing, then an AcceptInvalidationMessages in the middle
>>> section atop which sit a few more catalog accesses, and further up from
>>> that you have another AcceptInvalidationMessages with more catalog
>>> accesses. AFAICS that's pretty much the same thing Andres was
>>> describing.
>>
>> It's somewhat different because it doesn't seem to involve a reload of a
>> nailed table, which my traces did. I wasn't able to reproduce the crash
>> more than once, so I'm not at all sure how to properly verify the issue.
>> I'd appreciate if Thomas could try to do so again with the small patch I
>> provided.
>>
>
> I'll try in the evening. I've tried reproducing it on my laptop, but I
> can't make that happen for some reason - I know I've seen some crashes
> here, but all the reproducers were from the workstation I have at home.
>
> I wonder if there's some subtle difference between the two boxes, making
> it more likely on one of them ... the whole environment (distribution,
> packages, compiler, ...) should be exactly the same, though. The only
> thing I can think of is different CPU speed, possibly making some race
> conditions more/less likely. No idea.
>
I take that back - I can reproduce the crashes, both with and without
the patch, all the way back to 9.6. Attached is a bunch of backtraces
from various versions. There's a bit of variability depending on which
pgbench script gets started first (insert.sql or vacuum.sql) - in one
case (when vacuum is started before insert) the crash happens in
InitPostgres/RelationCacheInitializePhase3, otherwise it happens in
exec_simple_query.
Another observation is that the failing COPY is not necessary, I can
reproduce the crashes without this (so even with wal_level=replica).
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
[application/gzip] crash-10.log.gz (9.2K, ../9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com/2-crash-10.log.gz)
download
[application/gzip] crash-11.log.gz (9.3K, ../9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com/3-crash-11.log.gz)
download
[application/gzip] crash-11-2.log.gz (13.1K, ../9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com/4-crash-11-2.log.gz)
download
[application/gzip] crash-11-3.log.gz (10.3K, ../9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com/5-crash-11-3.log.gz)
download
[application/gzip] crash-96.log.gz (10.1K, ../9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com/6-crash-96.log.gz)
download
[application/gzip] crash-96-2.log.gz (10.1K, ../9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com/7-crash-96-2.log.gz)
download
[application/gzip] crash-96-logical.log.gz (11.9K, ../9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com/8-crash-96-logical.log.gz)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, andres@anarazel.de, alvherre@2ndquadrant.com
Subject: Re: logical decoding / rewrite map vs. maxAllocatedDescs
In-Reply-To: <9552a9ef-250d-c7bf-abca-0c0533215fee@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox