pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Sergey Koposov <Sergey.Koposov@ed.ac.uk>
To: lepihov@gmail.com <lepihov@gmail.com>
To: tgl@sss.pgh.pa.us <tgl@sss.pgh.pa.us>
Cc: melanieplageman@gmail.com <melanieplageman@gmail.com>
Cc: tomas@vondra.me <tomas@vondra.me>
Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Cc: a.rybakina@postgrespro.ru <a.rybakina@postgrespro.ru>
Subject: Re: BUG #18909: Query creates millions of temporary files and stalls
Date: Sun, 4 May 2025 12:35:08 +0000
Message-ID: <59090d89e656fde93b539e95a23a740d532d4f88.camel@ed.ac.uk> (raw)
In-Reply-To: <fb86d600-09f9-469a-a712-6ab26ff38641@gmail.com>
References: <18909-e5e1b702c9441b8a@postgresql.org>
	<2051617.1746223978@sss.pgh.pa.us>
	<beb051161520fc48248c2f90582eb15c6a71d239.camel@ed.ac.uk>
	<5e98fba645682c40bf5f88ef858963ade9681956.camel@ed.ac.uk>
	<2177960.1746289626@sss.pgh.pa.us>
	<80bab58be182e80e73b5d5f71664ede9ce58f957.camel@ed.ac.uk>
	<8fc51cd7-115a-425a-828e-c99defb5afef@gmail.com>
	<ba927eb7126844c1b964e75cadb774488794dfde.camel@ed.ac.uk>
	<0ce957be37434cfca4820d1e89a70cb2b456d3db.camel@ed.ac.uk>
	<2216372.1746309806@sss.pgh.pa.us>
	<ca371295b353c0306ccab1fcd6a06bd6421f4955.camel@ed.ac.uk>
	<2221240.1746312453@sss.pgh.pa.us>
	<f6fb5c3463006c1d4471bc30f9edb419cb777329.camel@ed.ac.uk>
	<9691212b-86f0-4476-940e-2e4ba0dd0cf9@gmail.com>
	<0a2ccb925ec37c93a143359de6381392779fdde4.camel@ed.ac.uk>
	<fb86d600-09f9-469a-a712-6ab26ff38641@gmail.com>

Thanks,

I don't have a strong opinion on how this is fixed, but I personally think that
is a bug that needs addressing in back branches.

I think I remember hitting OOMs in similar situations with hashjoins in the past
and I just disabled them.

But I'd certainly argue the situation here is more severe with 12 million temp files (possibly more).
Even the cancelling of the query doesn't quite work well,
because the deletion of temp files is not fast enough in the cancellation.
Only the restarting of the server does delete tmp files fast enough (takes a few min still).
And apparently even ls can't quite deal with 12 millions files in a folder.

  Sergey


The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.


view thread (17+ messages)

Message-ID: <59090d89e656fde93b539e95a23a740d532d4f88.camel@ed.ac.uk>
Permalink:  ../59090d89e656fde93b539e95a23a740d532d4f88.camel@ed.ac.uk/
Also on:    postgresql.org/message-id/59090d89e656fde93b539e95a23a740d532d4f88.camel@ed.ac.uk

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: Sergey.Koposov@ed.ac.uk, lepihov@gmail.com, tgl@sss.pgh.pa.us, melanieplageman@gmail.com, tomas@vondra.me, pgsql-bugs@lists.postgresql.org, a.rybakina@postgrespro.ru
  Subject: Re: BUG #18909: Query creates millions of temporary files and stalls
  In-Reply-To: <59090d89e656fde93b539e95a23a740d532d4f88.camel@ed.ac.uk>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox