Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ihCJM-0000hU-Rv for pgsql-bugs@arkaria.postgresql.org; Tue, 17 Dec 2019 12:50:49 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1ihCJL-0001Su-Cs for pgsql-bugs@arkaria.postgresql.org; Tue, 17 Dec 2019 12:50:47 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ihCJL-0001Sn-4W for pgsql-bugs@lists.postgresql.org; Tue, 17 Dec 2019 12:50:47 +0000 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1ihCJD-0001nf-CR for pgsql-bugs@lists.postgresql.org; Tue, 17 Dec 2019 12:50:46 +0000 Received: by mail-wr1-x42e.google.com with SMTP id t2so11196901wrr.1 for ; Tue, 17 Dec 2019 04:50:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ci/i5bPxrmaTtk58gfQMAkdsc6QxTnOIsz9IMIs6Ps0=; b=Np7+leDerDvBmyJCmspBsVANSqDCILOZ3eduUs5v1EJs2TcZ6EZqdfo9L9513dEYQC 6h3Q9i5FVIPDvwYlRmNQMAyH3NjS8OmN1h7t9UNN21QZUoR0fAt0gCd7+9XiH0ZDf30a J4BomqqMG3Y73K0zC89m7eGDeXpFnSvttEQ5EXXwMgS7QSu18+OBXAaR/A/3iRNj5dJ8 0HTlpuCWZ132Zok7doBC+WDowD2mu8mPY+NqFkah15ggwAchHvmNUldhq8GDJPUEHkf1 Xt5D+JtG/KNrBv2RadaguXEZBXyRlKH8P/OAHC49epRoydvysRJCRjvj3YLUpIAdX4QL GFUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ci/i5bPxrmaTtk58gfQMAkdsc6QxTnOIsz9IMIs6Ps0=; b=adaKOYC3pwEsKFDoRZv2Su0JAMzCYMh+yNRcfmwuuY6FcXzn+AtIVj6ZDbi727Mlp1 1BeqMSkwlEUTccFR2SDFh/6HazeDfjdYP9kxTIl0MUDaHIJO9kH4lcUl2rv7Cq7RpIve 9eCYgpZk9yFm4IJIu+cJRLw0BJGU2VrN7+BcV9kM7zIvMp2caEsICSy4EWNM6gRW9v1I YUTuNVtzFl0I/xkSfQB/cCZreIZFngnBUUZqw5tHTp1m5vtYH6LOPUnBlBuxbPlY8QOP /I1u6e5pcEzhiJOY4mT8vmNprAGixcexWFBLLUVKqbpS1FCxn05abMKY9OIAZFgBO/FW AqPQ== X-Gm-Message-State: APjAAAVX6xA1XiaiIlS8S71ZK16xCWmtVVpx4bO1YJmTtOCwWehoe3SG dY/czcFPfmbyc0qa6L5SR0RAJQ== X-Google-Smtp-Source: APXvYqxZRCWQ2vl+QsjL+gJ3GinoCmaURvG3BnDdz5tXb5KNtlRjf1q6phY34PJUvztypD/Nqmaxag== X-Received: by 2002:adf:f70b:: with SMTP id r11mr39085493wrp.388.1576587038327; Tue, 17 Dec 2019 04:50:38 -0800 (PST) Received: from localhost (ip-86-49-253-92.net.upcbroadband.cz. [86.49.253.92]) by smtp.gmail.com with ESMTPSA id i16sm2957280wmb.36.2019.12.17.04.50.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Dec 2019 04:50:37 -0800 (PST) Date: Tue, 17 Dec 2019 13:50:35 +0100 From: Tomas Vondra To: Roman Cervenak Cc: Tom Lane , pgsql-bugs@lists.postgresql.org Subject: Re: Memory leak (possibly connected to postgis) leading to server crash Message-ID: <20191217125035.wdplskovliz7d5lh@development> References: <20191206110154.v7yshqsn32zyxx4q@development> <20191206124119.qe2xaffcbkcbmve3@development> <18069.1576082949@sss.pgh.pa.us> <20191213132036.3m27dlquaxoorkja@development> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Tue, Dec 17, 2019 at 12:51:27PM +0100, Roman Cervenak wrote: >Hey guys, >I have reproducer for you. > >Luckily, the issue is not specific for our data, and can be demonstrated >also on OpenStreepMap data. I have imported OSM planet with Imposm, which >creates tables with PostGIS geometry columns. >I have used roads for linestring testing and landuse for polygon testing, >with queries combining several geometric operations (similar to my >production workload). > >The linesting test with 7 concurrent workers increases memory consumption >by about 6 GB per hour on my environment. > >If you don't have OSM database with geometries, I have dumped the roads >table (about 25GB download): >https://sygicpublic.blob.core.windows.net/postgres-bug-data/osm_road.sql.gzip >Here is the tool running queries in parallel workers, written in .NET Core >(just fill in database connection info in Program.cs, build and run): >https://github.com/rouen-sk/PostgisMemoryLeakDemoCore > >Let me know if I can help further. > I'll try reproducing it, but I don't have any environment to run c# stuff. Is it possible to reproduce a single query reproducing the issue, with fixed parameters? Or is the randomization necessary? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services