pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Roman Cervenak <roman@cervenak.info>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: Memory leak (possibly connected to postgis) leading to server crash
Date: Tue, 17 Dec 2019 13:50:35 +0100
Message-ID: <20191217125035.wdplskovliz7d5lh@development> (raw)
In-Reply-To: <CAGjExY2JyROqZpUkOZkvVS7=-MPPM7Te5wgnJ+oD5h0X2V6pTw@mail.gmail.com>
References: <CAGjExY1C40d=cqbf6nfeNDCLWk57MhFu3mgR0+KRPb8+_uW_KA@mail.gmail.com>
	<20191206110154.v7yshqsn32zyxx4q@development>
	<CAGjExY0sFbx+JytMkY-h8=NWqD_f0DOwB8QXrQ4SqJ+BttoX6g@mail.gmail.com>
	<20191206124119.qe2xaffcbkcbmve3@development>
	<CAGjExY0Kb0-h3MqWbiYry3e40nKbYyOW3-=BO7iKLJpo8R66aw@mail.gmail.com>
	<18069.1576082949@sss.pgh.pa.us>
	<CAGjExY2xQa8hxEYBTDDv6Y66V2=D0HEDsmLt42ZVupC3BxsLsQ@mail.gmail.com>
	<20191213132036.3m27dlquaxoorkja@development>
	<CAGjExY2JyROqZpUkOZkvVS7=-MPPM7Te5wgnJ+oD5h0X2V6pTw@mail.gmail.com>

On Tue, Dec 17, 2019 at 12:51:27PM +0100, Roman Cervenak wrote:
>Hey guys,
>I have reproducer for you.
>
>Luckily, the issue is not specific for our data, and can be demonstrated
>also on OpenStreepMap data. I have imported OSM planet with Imposm, which
>creates tables with PostGIS geometry columns.
>I have used roads for linestring testing and landuse for polygon testing,
>with queries combining several geometric operations (similar to my
>production workload).
>
>The linesting test with 7 concurrent workers increases memory consumption
>by about 6 GB per hour on my environment.
>
>If you don't have OSM database with geometries, I have dumped the roads
>table (about 25GB download):
>https://sygicpublic.blob.core.windows.net/postgres-bug-data/osm_road.sql.gzip
>Here is the tool running queries in parallel workers, written in .NET Core
>(just fill in database connection info in Program.cs, build and run):
>https://github.com/rouen-sk/PostgisMemoryLeakDemoCore
>
>Let me know if I can help further.
>

I'll try reproducing it, but I don't have any environment to run c#
stuff. Is it possible to reproduce a single query reproducing the issue,
with fixed parameters? Or is the randomization necessary?

regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 





view thread (16+ messages)  latest in thread

Message-ID: <20191217125035.wdplskovliz7d5lh@development>
Permalink:  ../20191217125035.wdplskovliz7d5lh@development/
Also on:    postgresql.org/message-id/20191217125035.wdplskovliz7d5lh@development

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, roman@cervenak.info, tgl@sss.pgh.pa.us
  Subject: Re: Memory leak (possibly connected to postgis) leading to server crash
  In-Reply-To: <20191217125035.wdplskovliz7d5lh@development>
  References: <CAGjExY1C40d=cqbf6nfeNDCLWk57MhFu3mgR0+KRPb8+_uW_KA@mail.gmail.com> <20191206110154.v7yshqsn32zyxx4q@development> <CAGjExY0sFbx+JytMkY-h8=NWqD_f0DOwB8QXrQ4SqJ+BttoX6g@mail.gmail.com> <20191206124119.qe2xaffcbkcbmve3@development> <CAGjExY0Kb0-h3MqWbiYry3e40nKbYyOW3-=BO7iKLJpo8R66aw@mail.gmail.com> <18069.1576082949@sss.pgh.pa.us> <CAGjExY2xQa8hxEYBTDDv6Y66V2=D0HEDsmLt42ZVupC3BxsLsQ@mail.gmail.com> <20191213132036.3m27dlquaxoorkja@development> <CAGjExY2JyROqZpUkOZkvVS7=-MPPM7Te5wgnJ+oD5h0X2V6pTw@mail.gmail.com> <20191217125035.wdplskovliz7d5lh@development>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox