public inbox for [email protected]  

help / color / mirror / Atom feed

From: Justin Pryzby <[email protected]>
To: Tomas Vondra <[email protected]>
Cc: Josef Šimánek <[email protected]>
Cc: Matthias van de Meent <[email protected]>
Cc: Michael Paquier <[email protected]>
Cc: [email protected]
Subject: bug: copy progress reporting of backends which run multiple COPYs
Date: Wed, 18 Jan 2023 23:47:03 -0600
Message-ID: <[email protected]> (raw)

pg_stat_progress_copy was added in v14 (8a4f618e7, 9d2d45700).

But if a command JOINs file_fdw tables, the progress report gets bungled
up.  This will warn/assert during file_fdw tests.

diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6743e68cef6..7abcb4f60db 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -10,6 +10,7 @@
  */
 #include "postgres.h"
 
+#include "commands/progress.h"
 #include "port/atomics.h"		/* for memory barriers */
 #include "utils/backend_progress.h"
 #include "utils/backend_status.h"
@@ -105,6 +106,20 @@ pgstat_progress_end_command(void)
 	if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
 		return;
 
+// This currently fails file_fdw tests, since pgstat_progress evidently fails
+// to support simultaneous copy commands, as happens during JOIN.
+	/* bytes progress is not available in all cases */
+	if (beentry->st_progress_command == PROGRESS_COMMAND_COPY &&
+			beentry->st_progress_param[PROGRESS_COPY_BYTES_TOTAL] > 0)
+	{
+		volatile int64 *a = beentry->st_progress_param;
+		if (a[PROGRESS_COPY_BYTES_PROCESSED] > a[PROGRESS_COPY_BYTES_TOTAL])
+			elog(WARNING, "PROGRESS_COPY_BYTES_PROCESSED %ld %ld",
+					a[PROGRESS_COPY_BYTES_PROCESSED],
+					a[PROGRESS_COPY_BYTES_TOTAL]);
+		// Assert(a[PROGRESS_COPY_BYTES_PROCESSED] <= a[PROGRESS_COPY_BYTES_TOTAL]);
+	}
+
 	PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
 	beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
 	beentry->st_progress_command_target = InvalidOid;

view thread (2+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: bug: copy progress reporting of backends which run multiple COPYs
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox