Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iUtyH-0000Au-Gq for pgsql-bugs@arkaria.postgresql.org; Wed, 13 Nov 2019 14:50:14 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1iUtyG-0008Ih-5V for pgsql-bugs@arkaria.postgresql.org; Wed, 13 Nov 2019 14:50:12 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iUtyF-0008IX-OP for pgsql-bugs@lists.postgresql.org; Wed, 13 Nov 2019 14:50:11 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iUtyC-0006Yw-KD for pgsql-bugs@lists.postgresql.org; Wed, 13 Nov 2019 14:50:11 +0000 Received: by mail-wr1-x443.google.com with SMTP id a15so2675837wrf.9 for ; Wed, 13 Nov 2019 06:50:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=JzsydBwUe9SfSTAT8R9dds+XgyppmbUdX+lTiW1CAdA=; b=JE5ZM8duMZZIDYm2JHPrQ8a/fCdTI0EQafslFtfc8La1EWyGjaC7LeCKyr2/xKzkbT kcFYcI5Br8IxNEZuKMBCXyLlTm9JREpluU/WDJb7AxH/AX3U7yoLtlwc6kavg/j+HAl3 Tir618I96jVeZiST4KWZ64XYt3ek2ZK+FOHqM2rjKg+4eihmm74xM3g1nQ6z3YD2KLmW FESwC4Z+0UqfY/MAQeJx/SMXbEeqqBFmMXllvMoOtD8LGNeH0gExvQvj9UzOH1cEKUzt ZzXNpH+yYkXX9tWJP9OyfrUURGytvpQ0I+faHoJyEOM8QMgsG7K2F64qQz06edqGVNAL cpFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=JzsydBwUe9SfSTAT8R9dds+XgyppmbUdX+lTiW1CAdA=; b=RQeClQjylPmWignlgu1zlP0kD80Oe53puhqw/PYR69Rj54wehCR2ZbnJ80tvhoq0g/ b5Tsj8Ofe8NDF0pMKRUpTrGBFf2MT5uhterJ7a4WOhInpmdfrD6FR+ZQ3lP3F3x4++SM xL88AZNoe6onHH8XpAzHEGLrkfgB4J+oJ1J8L3ogvlzS1pQSSn2Uc7vR5DDtUWjbsqq/ EZrEAKddSmmvVCBBVmHY7x3cZA0jwHLXdKnRFLFTllpCbyBS+swb8owTX/ttaRLCo+q5 /wec+a9RAc+NIOdZLhgpwhEMBa2HjITJxttSCyCmOvbN81o5QwVvq74gDita6MOaXi51 76GA== X-Gm-Message-State: APjAAAUsezmPTd/NCjJeQnLHo+bJBUqCiKlo/ugDuRvz3gN2DWNz9Q5W 2xT9sn3mWlwmzBV9Z1vOZKhvCBcMQk4= X-Google-Smtp-Source: APXvYqx7FXfqxVwd2qyPwqsIoXKNkRZM9feeY3v8EG3Lf0LxVDMk9GEJyghmSCm+fIRg7qmuw6hfmg== X-Received: by 2002:adf:dc06:: with SMTP id t6mr3326414wri.378.1573656606487; Wed, 13 Nov 2019 06:50:06 -0800 (PST) Received: from localhost (static-84-42-175-93.net.upcbroadband.cz. [84.42.175.93]) by smtp.gmail.com with ESMTPSA id z11sm4022066wrg.0.2019.11.13.06.50.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Nov 2019 06:50:05 -0800 (PST) Date: Wed, 13 Nov 2019 15:50:04 +0100 From: Tomas Vondra To: ben@lantern.is, pgsql-bugs@lists.postgresql.org Subject: Re: BUG #16112: large, unexpected memory consumption Message-ID: <20191113145004.opxx4vuajyoadbcq@development> References: <16112-4448bbf55a404189@postgresql.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="qr32ioh6tregukqi" Content-Disposition: inline In-Reply-To: <16112-4448bbf55a404189@postgresql.org> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --qr32ioh6tregukqi Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline On Wed, Nov 13, 2019 at 12:22:07PM +0000, PG Bug reporting form wrote: >The following bug has been logged on the website: > >Bug reference: 16112 >Logged by: Ben Cornett >Email address: ben@lantern.is >PostgreSQL version: 12.0 >Operating system: linux 2.6.32 >Description: > >Creation of table t1 in the query below causes the server process to consume >close to 1GB of memory. The amount of memory consumed is proportional to >the value passed to generate_series in the first query. > >CREATE temp TABLE t0 AS >SELECT > i, > ARRAY[1,2,3] a >FROM GENERATE_SERIES(1, 12000000) i >; > >CREATE TEMP TABLE t1 AS >SELECT > i, > x >FROM t0, UNNEST(a) x; > >I observed the same behavior in other test queries that included implicit >lateral joins. > Yeah, I can reproduce this pretty easily. It seems like a memory leak in ExecMakeTableFunctionResult. a9c35cf85ca reworked FunctionCallInfo to be variable-length, but it gets allocated in ExecutorState context directly and so until the end of the query. The attached trivial patch fixes that by adding a pfree() at the end of the function. I wonder if we have the same issue elsewhere ... regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services --qr32ioh6tregukqi Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="memleak-a9c35cf85ca.patch" diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c index c8a3efc365..da19bbfc66 100644 --- a/src/backend/executor/execSRF.c +++ b/src/backend/executor/execSRF.c @@ -412,6 +412,8 @@ no_function_result: MemoryContextSwitchTo(callerContext); + pfree(fcinfo); + /* All done, pass back the tuplestore */ return rsinfo.setResult; } --qr32ioh6tregukqi--