Date: Mon, 27 Jul 2020 05:39:02 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>,
	"Bossart, Nathan" <bossartn@amazon.com>
Subject: Re: shared tempfile was not removed on statement_timeout
Message-ID: <20200727103902.GM4286@telsasoft.com>
References: 
 <CA+hUKGJf5uPTPz-9FoxnaOfeu74Hi494HG8EyVAmPUatPjPL9A@mail.gmail.com>
 <CA+hUKGJStr-3B6qNnFEOpES8HHc3Wwe3wSrYYQJcQhHuTB9SdQ@mail.gmail.com>
 <20200721043258.GE5748@telsasoft.com>
 <CA+hUKGJPzYo-qz63tp9052i8FKehRyUw4djo8a474xvRCZ+vHQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CA+hUKGJPzYo-qz63tp9052i8FKehRyUw4djo8a474xvRCZ+vHQ@mail.gmail.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

On Mon, Jul 27, 2020 at 08:00:46PM +1200, Thomas Munro wrote:
> On Tue, Jul 21, 2020 at 4:33 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> >  /*
> >   * clean up a spool structure and its substructures.
> >   */
> >  static void
> >  _bt_spooldestroy(BTSpool *btspool)
> >  {
> > +       void *fileset = tuplesort_shared_fileset(btspool->sortstate);
> > +       if (fileset)
> > +               SharedFileSetDeleteAll(fileset);
> >         tuplesort_end(btspool->sortstate);
> >         pfree(btspool);
> >  }
> 
> Why can't tuplesort_end do it?

Because then I think the parallel workers remove their own files, with tests
failing like:

+ERROR:  could not open temporary file "0.0" from BufFile "0": No such file or directory

I look around a bit more and came up with this, which works, but I don't know
enough to say if it's right.

diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index 5f6420efb2..f89d42f475 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -3344,6 +3344,7 @@ walkdir(const char *path,
 		struct stat fst;
 		int			sret;
 
+		usleep(99999);
 		CHECK_FOR_INTERRUPTS();
 
 		if (strcmp(de->d_name, ".") == 0 ||
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3c49476483..c6e5e6d00b 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -1387,6 +1387,9 @@ tuplesort_free(Tuplesortstate *state)
 void
 tuplesort_end(Tuplesortstate *state)
 {
+	if (state->shared && state->shared->workersFinished == state->nParticipants)
+		SharedFileSetDeleteAll(&state->shared->fileset);
+
 	tuplesort_free(state);
 
 	/*