Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iP7pB-0007ry-Ho for pgsql-bugs@arkaria.postgresql.org; Mon, 28 Oct 2019 16:24:57 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1iP7p9-00080E-9i for pgsql-bugs@arkaria.postgresql.org; Mon, 28 Oct 2019 16:24:55 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iP7p8-000807-Ra for pgsql-bugs@lists.postgresql.org; Mon, 28 Oct 2019 16:24:55 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iP7p5-0005hs-Rl for pgsql-bugs@lists.postgresql.org; Mon, 28 Oct 2019 16:24:54 +0000 Received: by mail-wr1-x441.google.com with SMTP id p4so10558868wrm.8 for ; Mon, 28 Oct 2019 09:24:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=dfcehdf9Mzv/L1g3E7+jtkfnBomZ6g3tHZaLN1kgEmM=; b=oE3mwf/OnCx7xxdR5O4824N1K8m1Ms8QwSeBDQpbAaqAJKAHE0gCrd7DcD2crl6gFj T9xVPBek26jQzGqRc3J2aXt/RKhpyUnsMapmW6dU4s0GeXs3Fp3kErnAP3G+Rsic5mo5 5Y4DFZzJzgWMRfbK59Y/MVx5iC25+sBP+HXbl46/B/DaAzrmdkLqgGg5jLv88DclBhqm t2+I1PyhkTs5yV0Fvs7pJOCG6Xfl0jcQGiA2gkpEe5ylsX+ydy2qIExtALyQxrRbz1bG Aljx2K9fEuFxkegjZEWNIm7VHObQnQ8orQt2Hfskxz9LskPQ7n/3kJaidWxjM4wAgY2j 8qRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=dfcehdf9Mzv/L1g3E7+jtkfnBomZ6g3tHZaLN1kgEmM=; b=So8xWKrHvGxmrbr/LDszoqW8jneNAFJgKxusImPoHWzUrsl6ikwBiFnuq8vU5K7tyg RrrnNOr41ivCrmJRReFpKYnAW82iFEedvIiDkd0yxW+638ZMjDY2ASKAbCCnHUTlOefS n7uz3aC9mGw5cs3az6pEGSX7UsNoYfDMGYurpOR/MWeKBfAwIv45s6PVtPFnYZO/CFPA UJ2+4sSAFtIEPJuA1eDuB2fudU7kDPYmNAbmSE8VIGGWvBA5Mgii06KumWwKvgN739VX XrFw+WI27mfS5AGVowJwSJcD72sZj7CDHqkqbcOAWM7gOAGS7dj4l1+6+TilXogn83n8 UNAA== X-Gm-Message-State: APjAAAXRVRmQtFnm1CKpZcTYMh3PAp1WK75EdlF1CdthIqpSf5Go+/eP TMem4+WAgB1VFcPA4NoVlwvZUw== X-Google-Smtp-Source: APXvYqz3bl13g8DDyLvRj0sKtyA9MT/Agv30lppPh7sLkpJdOg5Gq4D6X9BKw8OuXJLbsoYcClztcA== X-Received: by 2002:adf:f54a:: with SMTP id j10mr12546278wrp.337.1572279888720; Mon, 28 Oct 2019 09:24:48 -0700 (PDT) Received: from localhost (ip-86-49-253-183.net.upcbroadband.cz. [86.49.253.183]) by smtp.gmail.com with ESMTPSA id i3sm12112918wrw.69.2019.10.28.09.24.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2019 09:24:47 -0700 (PDT) Date: Mon, 28 Oct 2019 17:24:46 +0100 From: Tomas Vondra To: Josef Machytka Cc: pgsql-bugs@lists.postgresql.org Subject: Re: memory problems and crash of db when deleting data from table with thousands of partitions Message-ID: <20191028162446.wlktskknjpsrj4rt@development> References: <20191023124751.pnzcos7i3ei2vjxc@development> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20180716 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Mon, Oct 28, 2019 at 04:18:59PM +0100, Josef Machytka wrote: >Thank you for your email, FYI - we now did tests with PostgreSQL 12 and >unfortunately it is also not able to handle to our case. Only difference is >that PG 12 is not killed by OOM killer and even does not crash - which is >good. But it reports error "out of memory" and stops the statement. So at >least it looks like much more stable then PG 11. > Hmmm, this seems a bit weird to me: MessageContext: 52197329840 total in 8274 blocks; 75904 free (9 chunks); 52197253936 used That context is generally meant for parse trees and other long-lived stuff, and I wouldn't expect it to grow to 52GB of data, even if there are many many partitions. I wonder if this might be just another manifestation of the memory leak from [1]. Can you provide a self-contained reproducer, i.e. a script I could use to reproduce the issue? [1] https://www.postgresql.org/message-id/20191024221758.vfv2enubnwmy3deu@development regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services