Subject: Re: bug: copy progress reporting of backends which run multiple COPYs

From: Robert Haas <[email-masked]>
Date: Wed, 8 May 2024 10:07:15 -0400

On Tue, May 7, 2024 at 9:12 PM Michael Paquier <[email-masked]> wrote:
> FWIW, I'm rather annoyed by the fact that we rely on the ParseState to
> decide if reporting should happen or not.  file_fdw tells, even if
> that's accidental, that status reporting can be useful if working on a
> single table.  So, shutting down the whole reporting if a caller if
> BeginCopyFrom() does not give a ParseState is too heavy-handed, IMO.

I think you're hoping for too much. The progress reporting
infrastructure is fundamentally designed around the idea that there
can only be one progress-reporting operation in progress at a time.
For COPY, that is, I believe, true, but for file_fdw, it's false. If
we want to do any kind of progress reporting from within plannable
queries, we need some totally different and much more complex
infrastructure that can report progress for, probably, each plan node
individually. I think it's likely a mistake to try to shoehorn cases
like this into the infrastructure
that we have today. It will just encourage people to try to use the
current infrastructure in ways that are less and less like what it was
actually designed to do; whereas what we should be doing if we want
this kind of functionality, at least IMHO, is building infrastructure
that's actually fit for purpose.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

---

From: Michael Paquier <[email-masked]>
Date: Thu, 9 May 2024 08:57:32 +0900

> that's actually fit for purpose.

Hmm.  OK.  I have been looking around for cases out there where
BeginCopyFrom() could be called with a pstate where the reporting
could matter, and could not find anything worth worrying about.  It
still makes me a bit uneasy to not have a separate argument in the
function to control that.  Now, if you, Justin and Matthias agree with
this approach, I won't stand in the way either.
--
Michael