Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ia1bP-0008Cq-UV for pgsql-bugs@arkaria.postgresql.org; Wed, 27 Nov 2019 17:59:48 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1ia1bO-0005Gk-5E for pgsql-bugs@arkaria.postgresql.org; Wed, 27 Nov 2019 17:59:46 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ia1bN-0005Gd-Md for pgsql-bugs@lists.postgresql.org; Wed, 27 Nov 2019 17:59:45 +0000 Received: from mail-qv1-xf41.google.com ([2607:f8b0:4864:20::f41]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1ia1bI-0006wE-Ij for pgsql-bugs@lists.postgresql.org; Wed, 27 Nov 2019 17:59:44 +0000 Received: by mail-qv1-xf41.google.com with SMTP id n4so9257041qvq.9 for ; Wed, 27 Nov 2019 09:59:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=DIEO7r/VURllpkuziIFWK13tpM0EnCy8Oer1h+wDBoE=; b=glc7ks9YhnDoiSAfSx9+1TwcYYYb+C+dAUhr8Uk+Wj/a1mHxx8PxYEPiyO16IppIfV cejsQVUmoYEpTVk3fMqZEVQgkgqwviiNyxgfzE1sU3Q+079lnCbHNzF7m5ZbsBrea+5r xQBQ9JNY01mvhWkeGMW5LaeCPaUVFulm4mpfdj47ymErpBs5DK7rOEZgzMoaP2fMf/70 br4WlANN0eAquxazrezZmSbo664aIFLlbFvd8uU2gTfpPvNbvxBKRobevEVStpcq5kSm uhfHN3By675K19wdi0g8KgYX1UqWOM4UghSz3MgmLigQom0U1aQ19DUeTq2BOS/Iw02H jKbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=DIEO7r/VURllpkuziIFWK13tpM0EnCy8Oer1h+wDBoE=; b=WhNX3MVwKJZxUD0jz2mf1IpGMx/yjQoIjzxhEUzxeovA26KMEYsCtpu+57KPOm46ja E8fPEeK/N7h5sYKak53t0nCyve71u1BW9HLAsITVRBaoQiYnw6ChbNoNrnE/7wjhm/Xo 9wxCx5I2ZO2AZYBxkr5JUgNU9znnqMGuAx2x5JqW5mN2eYLl1RN22+Odat2nGHU4ZYxU R1qyV9dRqeJOyUZGNaPTy8EzS6tkQllXWVCHQp+gREUuCi/GBpv2LAppdo09SlPOc7Ea eP/jiSagrz41AhlJU5SHoOWEgnQon4MeH3xsXkwVCFH4b5cO0U70fyLY8lTsH7eGJykU cwVw== X-Gm-Message-State: APjAAAUr/RU4DeBE5potEg/+hquTHI04KoBsvDeSmHU5bHZeUUG5PugT XgvWxqncPdt2gG5FQlEmmiBdmw== X-Google-Smtp-Source: APXvYqzBEmKnXBy8MymptH64ArV1tpr/PG4wx72WP+H3FsQ4glnB0UYNuRZ9j2E3CfaTo79MN+DADQ== X-Received: by 2002:a05:6214:86:: with SMTP id n6mr6416113qvr.219.1574877578523; Wed, 27 Nov 2019 09:59:38 -0800 (PST) Received: from nimloth.alvh.no-ip.org ([190.121.29.3]) by smtp.gmail.com with ESMTPSA id c27sm7012632qko.132.2019.11.27.09.59.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Nov 2019 09:59:37 -0800 (PST) Received: by nimloth.alvh.no-ip.org (Postfix, from userid 1000) id 12A77300661; Wed, 27 Nov 2019 14:59:35 -0300 (-03) Date: Wed, 27 Nov 2019 14:59:35 -0300 From: Alvaro Herrera To: Andres Freund Cc: Tomas Vondra , Andrey Salnikov , pgsql-bugs@lists.postgresql.org Subject: Re: BUG #16125: Crash of PostgreSQL's wal sender during logical replication Message-ID: <20191127175935.GA20945@alvherre.pgsql> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20191119232849.qhtuhq4udrqrz4he@alap3.anarazel.de> User-Agent: Mutt/1.10.1 (2018-07-13) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 2019-Nov-19, Andres Freund wrote: > > > 2525:rmgr: Heap len (rec/tot): 1671/ 1671, tx: 1667601530, lsn: 25EE/D66F5508, prev 25EE/D66F54C8, desc: INSERT off 2, blkref #0: rel 1663/13018/88964792 blk 18475 > > > 2532:rmgr: Heap len (rec/tot): 54/ 54, tx: 1667601530, lsn: 25EE/D66F7EF0, prev 25EE/D66F7EA0, desc: DELETE off 2 , blkref #0: rel 1663/13018/88964792 blk 18475 > > So, this DELETE change triggers the issue. If I understand it correctly, > > that means the transaction locked the tuple and then deleted it. But I > > don't see any record about the lock. > > Hm. I don't think it necessarily means that. compute_new_xmax_infomask(), when > called from heap_delete(), will e.g. set HEAP_KEYS_UPDATED when > > if (old_infomask & HEAP_XMAX_INVALID) > > is true. Which is the most common case. I actually find it *more* > curious to *not* see that for a delete. I've not traced fully through > it, yet. > > > Alvaro, it's not clear to me whether the the multixact codepaths in > compute_new_xmax_infomask() actually work as expected for deletes. So, what's really happening here is heap_insert(speculative) followed by heap_abort_speculative(). That seems to explain the above WAL sequence. I *think* the speculative insertion code in reorderbuffer simply delays the tuple insertion until it is confirmed; so this sequence would result in nothing at all happening at ReorderBufferCommit. Unless something has messed up the state somehow. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services