From: Justin Pryzby <pryzby@telsasoft.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Bossart, Nathan <bossartn@amazon.com>
Subject: Re: shared tempfile was not removed on statement_timeout
Date: Mon, 27 Jul 2020 05:39:02 -0500
Message-ID: <20200727103902.GM4286@telsasoft.com> (raw)
In-Reply-To: <CA+hUKGJPzYo-qz63tp9052i8FKehRyUw4djo8a474xvRCZ+vHQ@mail.gmail.com>
References: <CA+hUKGJf5uPTPz-9FoxnaOfeu74Hi494HG8EyVAmPUatPjPL9A@mail.gmail.com>
<CA+hUKGJStr-3B6qNnFEOpES8HHc3Wwe3wSrYYQJcQhHuTB9SdQ@mail.gmail.com>
<20200721043258.GE5748@telsasoft.com>
<CA+hUKGJPzYo-qz63tp9052i8FKehRyUw4djo8a474xvRCZ+vHQ@mail.gmail.com>
On Mon, Jul 27, 2020 at 08:00:46PM +1200, Thomas Munro wrote:
> On Tue, Jul 21, 2020 at 4:33 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > /*
> > * clean up a spool structure and its substructures.
> > */
> > static void
> > _bt_spooldestroy(BTSpool *btspool)
> > {
> > + void *fileset = tuplesort_shared_fileset(btspool->sortstate);
> > + if (fileset)
> > + SharedFileSetDeleteAll(fileset);
> > tuplesort_end(btspool->sortstate);
> > pfree(btspool);
> > }
>
> Why can't tuplesort_end do it?
Because then I think the parallel workers remove their own files, with tests
failing like:
+ERROR: could not open temporary file "0.0" from BufFile "0": No such file or directory
I look around a bit more and came up with this, which works, but I don't know
enough to say if it's right.
inline patch 1: src/backend/storage/file/fd.c, src/backend/utils/sort/tuplesort.c (+4/-0 lines) download inline-1.patchdiff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.cindex 5f6420efb2..f89d42f475 100644--- a/src/backend/storage/file/fd.c+++ b/src/backend/storage/file/fd.c@@ -3344,6 +3344,7 @@ walkdir(const char *path,
struct stat fst;
int sret;
+ usleep(99999);
CHECK_FOR_INTERRUPTS();
if (strcmp(de->d_name, ".") == 0 ||
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.cindex 3c49476483..c6e5e6d00b 100644--- a/src/backend/utils/sort/tuplesort.c+++ b/src/backend/utils/sort/tuplesort.c@@ -1387,6 +1387,9 @@ tuplesort_free(Tuplesortstate *state)
void
tuplesort_end(Tuplesortstate *state)
{
+ if (state->shared && state->shared->workersFinished == state->nParticipants)+ SharedFileSetDeleteAll(&state->shared->fileset);+
tuplesort_free(state);
/*
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, thomas.munro@gmail.com, bossartn@amazon.com
Subject: Re: shared tempfile was not removed on statement_timeout
In-Reply-To: <20200727103902.GM4286@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox