agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feed[PATCH 03/17] Teach postmaster to accept encryption key
416+ messages / 35 participants
[nested] [flat]
* [PATCH 03/17] Teach postmaster to accept encryption key
@ 2019-07-05 14:24 Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2019-07-05 14:24 UTC (permalink / raw)
This patch teaches postmaster to accept encryption key (as a special kind of
startup packet) before recovery starts. In fact the key is received by a
backend that postmaster launches if a client tries to initiate connection
while the postmaster is in PM_INIT state. At this stage, only the key is
received, whereas regular connection attempt ends up with
FATAL: the database system is starting up
If postmaster receives no key for some time, it raises this error and exits:
FATAL: Encryption key not received.
With this patch, user can start postmaster as usual:
postgres -D data
and use pg_keytool utility in another session to send the key:
echo 71916b8b93063c280e4e0ee646395f450bda4f1bac85c57bc3a6afd168f3ab47 | pg_keytool -s
---
src/backend/postmaster/postmaster.c | 211 +++++++++++++++++++++++++++++++++-
src/backend/storage/file/encryption.c | 34 ++++++
src/backend/storage/ipc/ipci.c | 2 +
src/bin/Makefile | 1 +
src/bin/pg_keytool/.gitignore | 1 +
src/bin/pg_keytool/Makefile | 31 +++++
src/bin/pg_keytool/pg_keytool.c | 199 ++++++++++++++++++++++++++++++++
src/fe_utils/encryption.c | 101 ++++++++++++++++
src/include/fe_utils/encryption.h | 2 +
src/include/libpq/pqcomm.h | 24 ++++
src/include/storage/encryption.h | 26 +++++
11 files changed, 628 insertions(+), 4 deletions(-)
create mode 100644 src/bin/pg_keytool/.gitignore
create mode 100644 src/bin/pg_keytool/Makefile
create mode 100644 src/bin/pg_keytool/pg_keytool.c
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e8e3f8a4d3..47f7bc93f7 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -117,6 +117,7 @@
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pg_shmem.h"
@@ -402,12 +403,13 @@ static void PostmasterStateMachine(void);
static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
-static int ServerLoop(void);
+static int ServerLoop(bool receive_encryption_key);
static void ServerLoopCheckTimeouts(void);
static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
+static void processEncryptionKey(void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(void);
@@ -1365,6 +1367,41 @@ PostmasterMain(int argc, char *argv[])
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STARTING);
/*
+ * If encryption key is needed and we don't have it yet, call ServerLoop()
+ * in a restricted mode which allows a client application to send us the
+ * key. Regular client connections are not allowed yet.
+ */
+ if (data_encrypted && !encryption_setup_done)
+ {
+ char sample[ENCRYPTION_SAMPLE_SIZE];
+
+ status = ServerLoop(true);
+
+ /* No other return code should be seen here. */
+ Assert(status == STATUS_OK);
+
+ /* ServerLoop() shouldn't have exited otherwise. */
+ Assert(encryption_key_shmem->initialized);
+
+ /*
+ * Take a local copy of the key so that we don't have to receive it
+ * again if restarting after a crash.
+ */
+ memcpy(encryption_key, encryption_key_shmem, ENCRYPTION_KEY_LENGTH);
+
+ /* Finalize the setup. */
+ setup_encryption();
+
+ /* Verify the key. */
+ sample_encryption(sample);
+ if (memcmp(encryption_verification, sample, ENCRYPTION_SAMPLE_SIZE))
+ ereport(FATAL,
+ (errmsg("invalid encryption key"),
+ errdetail("The passed encryption key does not match"
+ " database encryption key.")));
+ }
+
+ /*
* We're ready to rock and roll...
*/
StartupPID = StartupDataBase();
@@ -1375,7 +1412,7 @@ PostmasterMain(int argc, char *argv[])
/* Some workers may be scheduled to start now */
maybe_start_bgworkers();
- status = ServerLoop();
+ status = ServerLoop(false);
/*
* ServerLoop probably shouldn't ever return, but if it does, close down.
@@ -1615,18 +1652,37 @@ DetermineSleepTime(struct timeval *timeout)
static time_t last_lockfile_recheck_time, last_touch_time;
/*
+ * The maximum amount of time postmaster can wait for encryption key if the
+ * cluster is encrypted. It should be shorter than the time pg_ctl waits for
+ * postgres to start (60 seconds) and should not be too long so that failure
+ * to specify encryption key command is recognized soon.
+ */
+#define MAX_WAIT_FOR_KEY_SECS 20
+
+/* When should waiting for encryption key end. */
+static TimestampTz wait_for_keys_until = 0;
+
+/*
* Main idle loop of postmaster
*
+ * If receive_encryption_key is true, only launch backend(s) to receive
+ * cluster encryption key and stop as soon as the key is in shared memory.
+ *
* NB: Needs to be called with signals blocked
*/
static int
-ServerLoop(void)
+ServerLoop(bool receive_encryption_key)
{
fd_set readmask;
int nSockets;
last_lockfile_recheck_time = last_touch_time = time(NULL);
+ /* Compute wait_for_keys_until if needed and not done yet. */
+ if (receive_encryption_key && wait_for_keys_until == 0)
+ wait_for_keys_until = PgStartTime +
+ MAX_WAIT_FOR_KEY_SECS * USECS_PER_SEC;
+
nSockets = initMasks(&readmask);
for (;;)
@@ -1663,6 +1719,27 @@ ServerLoop(void)
/* Needs to run with blocked signals! */
DetermineSleepTime(&timeout);
+ /*
+ * If waiting for the encryption key, make sure
+ * MAX_WAIT_FOR_KEY_SECS is not exceeded.
+ */
+ if (receive_encryption_key)
+ {
+ TimestampTz timeout_tz, timeout_end;;
+
+ timeout_tz = timeout.tv_sec * USECS_PER_SEC +
+ timeout.tv_usec;
+ timeout_end = GetCurrentTimestamp() + timeout_tz;
+
+ if (timeout_end > wait_for_keys_until)
+ {
+ timeout_tz -= (timeout_end - wait_for_keys_until);
+
+ timeout.tv_sec = timeout_tz / USECS_PER_SEC;
+ timeout.tv_usec = timeout_tz % USECS_PER_SEC;
+ }
+ }
+
PG_SETMASK(&UnBlockSig);
selres = select(nSockets, &rmask, NULL, NULL, &timeout);
@@ -1714,6 +1791,33 @@ ServerLoop(void)
}
}
+ /*
+ * If only waiting for the encryption key, it's too early to launch
+ * the other processes.
+ */
+ if (receive_encryption_key)
+ {
+ /* The regular checks should take place though. */
+ ServerLoopCheckTimeouts();
+
+ /* Done if the key has been delivered. */
+ if (encryption_key_shmem->initialized)
+ return STATUS_OK;
+
+ /*
+ * Do not wait for the key forever. One problem we solve here is
+ * that pg_ctl (or custom script that starts postgres) does not
+ * have to check whether the cluster is encrypted. If DBA forgets
+ * to pass the encryption key command, he'll simply see the
+ * startup to fail and the appropriate message to appear in the
+ * server log.
+ */
+ if (GetCurrentTimestamp() >= wait_for_keys_until)
+ ereport(FATAL, (errmsg("Encryption key not received.")));
+
+ continue;
+ }
+
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
SysLoggerPID = SysLogger_Start();
@@ -1990,6 +2094,13 @@ ProcessStartupPacket(Port *port, bool secure_done)
return STATUS_ERROR;
}
+ if (proto == ENCRYPTION_KEY_MSG_CODE && data_encrypted)
+ {
+ processEncryptionKey(buf);
+ /* Not really an error, but we don't want to proceed further */
+ return STATUS_ERROR;
+ }
+
if (proto == NEGOTIATE_SSL_CODE && !secure_done)
{
char SSLok;
@@ -2394,6 +2505,77 @@ processCancelRequest(Port *port, void *pkt)
}
/*
+ * Process a message from backend that contains the encryption key.
+ *
+ * Currently there's no guarantee that only a single backend tries to deliver
+ * the key, but it's not worth trying to synchronize the access. If multiple
+ * callers try to process the same message, nothing should get broken. If some
+ * caller is delivering a wrong key (e.g. due to misconfiguration of another
+ * cluster), it does not help if it waits until processing of the correct key
+ * is finished.
+ *
+ * If the key gets messed up, the worst case is that the server fails to start
+ * up. (No data corruption is expected.)
+ *
+ * Client should not expect any response to this request: failure on client
+ * side indicates either broken connection, wrong key, bug in
+ * send_key_to_postmaster() or repeated call of the client - nothing to be
+ * handled easily by client code. Most of these failures need attention of the
+ * DBA.
+ */
+void
+processEncryptionKey(void *pkt)
+{
+ EncryptionKeyMsg *msg = (EncryptionKeyMsg *) pkt;
+
+ /* Backend to receive the key should not be launched otherwise. */
+ Assert(data_encrypted);
+
+ /*
+ * The checks below would fire in this case too, but this error message is
+ * clearer for the case like this.
+ */
+ if (pmState != PM_INIT)
+ {
+ ereport(COMMERROR,
+ (errmsg("encryption key can only be setup during startup"),
+ errhint("Check if the cluster is encrypted.")));
+ return;
+ }
+
+ /*
+ * Someone else already initialized the key. This is not fatal but seems
+ * to indicate misconfiguration which DBA should be aware of. Check also
+ * encryption_setup_done because shared memory could have been reset. As
+ * postmaster has a local copy of the key, the new key should not be used,
+ * but it'd be inconsistent if we didn't complain in this special case.
+ */
+ if (encryption_key_shmem->initialized || encryption_setup_done)
+ {
+ ereport(COMMERROR,
+ (errmsg("received encryption key more than once")));
+ return;
+ }
+
+ if (msg->version != 1)
+ {
+ ereport(COMMERROR,
+ (errmsg("unexpected version of encryption key message %d",
+ msg->version)));
+ return;
+ }
+
+ memcpy(encryption_key_shmem->data, msg->data, ENCRYPTION_KEY_LENGTH);
+ /*
+ * XXX Is memory barrier needed here to make sure that the copying is
+ * done?
+ */
+ encryption_key_shmem->initialized = true;
+
+ ereport(DEBUG1, (errmsg_internal("encryption key received")));
+}
+
+/*
* canAcceptConnections --- check to see if database state allows connections.
*/
static CAC_state
@@ -2418,7 +2600,8 @@ canAcceptConnections(void)
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
- (pmState == PM_STARTUP ||
+ ((data_encrypted && pmState == PM_INIT) ||
+ pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
return CAC_STARTUP; /* normal startup */
else if (!FatalError &&
@@ -2761,6 +2944,16 @@ pmdie(SIGNAL_ARGS)
pmState = (pmState == PM_RUN) ?
PM_WAIT_BACKUP : PM_WAIT_READONLY;
}
+ else if (pmState == PM_INIT && data_encrypted &&
+ !encryption_setup_done)
+ {
+ /*
+ * Only backends to receive encryption key may be active now,
+ * and these should not be involved in any transactions.
+ */
+ SignalSomeChildren(SIGTERM, BACKEND_TYPE_NORMAL);
+ pmState = PM_WAIT_BACKENDS;
+ }
/*
* Now wait for online backup mode to end and backends to exit. If
@@ -2828,6 +3021,16 @@ pmdie(SIGNAL_ARGS)
signal_child(WalWriterPID, SIGTERM);
pmState = PM_WAIT_BACKENDS;
}
+ else if (pmState == PM_INIT && data_encrypted &&
+ !encryption_setup_done)
+ {
+ /*
+ * Only backends to receive encryption key may be active now,
+ * and these should not be involved in any transactions.
+ */
+ SignalSomeChildren(SIGTERM, BACKEND_TYPE_NORMAL);
+ pmState = PM_WAIT_BACKENDS;
+ }
/*
* Now wait for backends to exit. If there are none,
diff --git a/src/backend/storage/file/encryption.c b/src/backend/storage/file/encryption.c
index 629138fae7..1eda334762 100644
--- a/src/backend/storage/file/encryption.c
+++ b/src/backend/storage/file/encryption.c
@@ -53,6 +53,10 @@ EVP_CIPHER_CTX *ctx_encrypt,
unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+#ifndef FRONTEND
+ShmemEncryptionKey *encryption_key_shmem = NULL;
+#endif /* FRONTEND */
+
bool data_encrypted = false;
char encryption_verification[ENCRYPTION_SAMPLE_SIZE];
@@ -66,6 +70,36 @@ static void evp_error(void);
#ifndef FRONTEND
/*
+ * Report space needed for our shared memory area
+ */
+Size
+EncryptionShmemSize(void)
+{
+ return sizeof(ShmemEncryptionKey);
+}
+
+/*
+ * Initialize our shared memory area
+ */
+void
+EncryptionShmemInit(void)
+{
+ bool found;
+
+ encryption_key_shmem = ShmemInitStruct("Cluster Encryption Key",
+ EncryptionShmemSize(),
+ &found);
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ encryption_key_shmem->initialized = false;
+ }
+ else
+ Assert(found);
+}
+
+/*
* Read encryption key in hexadecimal form from stdin and store it in
* encryption_key variable.
*/
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index d7d733530f..771bf30c63 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(int port)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, EncryptionShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(int port)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ EncryptionShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 903e58121f..b05a8bbb4e 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -22,6 +22,7 @@ SUBDIRS = \
pg_controldata \
pg_ctl \
pg_dump \
+ pg_keytool \
pg_resetwal \
pg_rewind \
pg_test_fsync \
diff --git a/src/bin/pg_keytool/.gitignore b/src/bin/pg_keytool/.gitignore
new file mode 100644
index 0000000000..249876a6ed
--- /dev/null
+++ b/src/bin/pg_keytool/.gitignore
@@ -0,0 +1 @@
+/pg_keytool
diff --git a/src/bin/pg_keytool/Makefile b/src/bin/pg_keytool/Makefile
new file mode 100644
index 0000000000..6da0631f32
--- /dev/null
+++ b/src/bin/pg_keytool/Makefile
@@ -0,0 +1,31 @@
+# src/bin/pg_keysetup/Makefile
+
+PGFILEDESC = "pg_keytool - handle cluster encryption key"
+PGAPPICON=win32
+
+subdir = src/bin/pg_keytool
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pg_keytool.o $(RMGRDESCOBJS) $(WIN32RES)
+
+override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+all: pg_keytool
+
+pg_keytool: $(OBJS) | submake-libpgport
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_keytool$(X) '$(DESTDIR)$(bindir)/pg_keytool$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_keytool$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_keytool$(X) $(OBJS) encryption.c
diff --git a/src/bin/pg_keytool/pg_keytool.c b/src/bin/pg_keytool/pg_keytool.c
new file mode 100644
index 0000000000..322625da41
--- /dev/null
+++ b/src/bin/pg_keytool/pg_keytool.c
@@ -0,0 +1,199 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_keytool.c - Handle cluster encryption key.
+ *
+ * Copyright (c) 2013-2019, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/bin/pg_keytool/pg_keytool.c
+ *-------------------------------------------------------------------------
+ */
+/*
+ * TODO Adopt the new frontend logging API, after some things are clarified:
+ * https://www.postgresql.org/message-id/1939.1560773970%40localhost
+ */
+#define FRONTEND 1
+#include "postgres.h"
+
+#include <dirent.h>
+#include <unistd.h>
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "fe_utils/encryption.h"
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq/pqcomm.h"
+#include "port/pg_crc32c.h"
+#include "storage/encryption.h"
+#include "getopt_long.h"
+
+#ifdef USE_ENCRYPTION
+static const char *progname;
+
+unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+
+static void
+usage(const char *progname)
+{
+ const char *env;
+
+ printf(_("%s is a tool to handle cluster encryption key.\n\n"),
+ progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION]...\n"), progname);
+ printf(_("\nOptions:\n"));
+ /* Display default host */
+ env = getenv("PGHOST");
+ printf(_(" -h, --host=HOSTNAME database server host or socket directory (default: \"%s\")\n"),
+ env ? env : _("local socket"));
+ /* Display default port */
+ env = getenv("PGPORT");
+ printf(_(" -p, --port=PORT database server port (default: \"%s\")\n"),
+ env ? env : DEF_PGPORT_STR);
+ printf(_(" -s, send output to database server\n"));
+ printf(_(" -?, --help show this help, then exit\n\n"));
+ printf(_("Key is read from stdin and sent either to stdout or to PostgreSQL server being started\n"));
+}
+#endif /* USE_ENCRYPTION */
+
+int
+main(int argc, char **argv)
+{
+/*
+ * If no encryption library is linked, let the utility fail immediately. It'd
+ * be weird if we reported incorrect usage just to say later that no useful
+ * work can be done anyway.
+ */
+#ifdef USE_ENCRYPTION
+ int c;
+ char *host = NULL;
+ char *port_str = NULL;
+ bool to_server = false;
+ int i, n;
+ int optindex;
+ char key_chars[ENCRYPTION_KEY_CHARS];
+
+ static struct option long_options[] =
+ {
+ {"pgdata", required_argument, NULL, 'D'},
+ {"host", required_argument, NULL, 'h'},
+ {"port", required_argument, NULL, 'p'},
+ {NULL, 0, NULL, 0}
+ };
+
+ pg_logging_init(argv[0]);
+ progname = get_progname(argv[0]);
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage(progname);
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_keytool (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ while ((c = getopt_long(argc, argv, "h:p:s",
+ long_options, &optindex)) != -1)
+ {
+ switch (c)
+ {
+ case 'h':
+ host = pg_strdup(optarg);
+ break;
+
+ case 'p':
+ port_str = pg_strdup(optarg);
+ break;
+
+ case 's':
+ to_server = true;
+ break;
+
+ case '?':
+ /* Actual help option given */
+ if (strcmp(argv[optind - 1], "-?") == 0)
+ {
+ usage(progname);
+ exit(EXIT_SUCCESS);
+ }
+
+ default:
+ pg_log_error("Try \"%s --help\" for more information.", progname);
+ exit(1);
+ }
+ }
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ fprintf(stderr, _("%s: too many command-line arguments (first is \"%s\")\n"),
+ progname, argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if ((host || port_str) && !to_server)
+ {
+ pg_log_error("host and port can only be passed along with the -s option");
+ exit(1);
+ }
+
+ /* Read the key. */
+ n = 0;
+ /* Key length in characters (two characters per hexadecimal digit) */
+ while ((c = getchar()) != EOF && c != '\n')
+ {
+ if (n >= ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_error("The key is too long");
+ exit(EXIT_FAILURE);
+ }
+
+ key_chars[n++] = c;
+ }
+
+ if (n < ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_error("The key is too short");
+ exit(EXIT_FAILURE);
+ }
+
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ {
+ if (sscanf(key_chars + 2 * i, "%2hhx", encryption_key + i) == 0)
+ {
+ pg_log_error("Invalid character in encryption key at position %d",
+ 2 * i);
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ /*
+ * Send the encryption key either to stdout or to server.
+ */
+ if (!to_server)
+ {
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ printf("%.2x", encryption_key[i]);
+ printf("\n");
+ }
+ else
+ {
+ if (!send_key_to_postmaster(host, port_str, encryption_key))
+ pg_log_error("could not send encryption key to server");
+ }
+
+#else
+ pg_log_fatal(ENCRYPTION_NOT_SUPPORTED_MSG);
+ exit(EXIT_FAILURE);
+#endif /* USE_ENCRYPTION */
+ return 0;
+}
diff --git a/src/fe_utils/encryption.c b/src/fe_utils/encryption.c
index dab1435014..ca37c9f373 100644
--- a/src/fe_utils/encryption.c
+++ b/src/fe_utils/encryption.c
@@ -88,3 +88,104 @@ run_encryption_key_command(unsigned char *encryption_key)
pfree(buf);
pclose(fp);
}
+
+/*
+ * Send the contents of encryption_key in the form of special startup packet
+ * to a server that is being started.
+ *
+ * Returns true if we could send the message and false if not, however even
+ * success does not guarantee that server started up - caller should
+ * eventually test server connection himself.
+ *
+ * TODO Windows stuff?
+ */
+bool
+send_key_to_postmaster(const char *host, const char *port,
+ const unsigned char *encryption_Key)
+{
+ const char **keywords = pg_malloc0(3 * sizeof(*keywords));
+ const char **values = pg_malloc0(3 * sizeof(*values));
+ int i;
+ PGconn *conn = NULL;
+ int sock = -1;
+ EncryptionKeyMsg message;
+ int msg_size, packet_size;
+ char *packet = NULL;
+ bool res = true;
+
+/* How many seconds we can wait for the postmaster to receive the key. */
+#define SEND_ENCRYPT_KEY_TIMEOUT 60
+
+ if (host)
+ {
+ keywords[0] = "host";
+ values[0] = host;
+ }
+ keywords[1] = "port";
+ values[1] = port;
+
+ /* Compose the message. */
+ message.encryptionKeyCode = pg_hton32(ENCRYPTION_KEY_MSG_CODE);
+ message.version = 1;
+ memcpy(message.data, encryption_key, ENCRYPTION_KEY_LENGTH);
+ msg_size = offsetof(EncryptionKeyMsg, data) + ENCRYPTION_KEY_LENGTH;
+
+ packet_size = msg_size + 4;
+ packet = (char *) palloc(packet_size);
+ *((int32 *) packet) = pg_hton32(packet_size);
+ memcpy(packet + 4, &message, msg_size);
+
+ /*
+ * Although we don't expect the server to accept regular libpq messages,
+ * we try to get at least a valid socket.
+ */
+ for (i = 0; i < SEND_ENCRYPT_KEY_TIMEOUT + 1; i++)
+ {
+ if (i > 0)
+ /* Sleep for 1 second. */
+ pg_usleep(1000000L);
+
+ if (conn)
+ {
+ PQfinish(conn);
+ conn = NULL;
+ }
+
+ conn = PQconnectStartParams(keywords, values, false);
+ if (conn == NULL)
+ continue;
+
+ sock = PQsocket(conn);
+ /* Cannot send the key if there's no valid socket. */
+ if (sock == -1)
+ continue;
+
+ /* Non-blocking write would only make this simple case tricky. */
+ if (!pg_set_block(sock))
+ continue;
+
+ retry:
+ /*
+ * Send the packet. Here we need to use low level API because the server
+ * does is not fully up so libpq cannot be used properly.
+ */
+ if (send(sock, (char *) packet, packet_size, 0) != packet_size)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry;
+ continue;
+ }
+ else
+ /* Success */
+ break;
+ }
+
+ pg_free(keywords);
+ pg_free(values);
+ if (conn)
+ PQfinish(conn);
+ pfree(packet);
+
+ return res;
+}
diff --git a/src/include/fe_utils/encryption.h b/src/include/fe_utils/encryption.h
index 69e3b2f756..7307557ed6 100644
--- a/src/include/fe_utils/encryption.h
+++ b/src/include/fe_utils/encryption.h
@@ -16,3 +16,5 @@
extern char *encryption_key_command;
extern void run_encryption_key_command(unsigned char *encryption_key);
+extern bool send_key_to_postmaster(const char *host, const char *port,
+ const unsigned char *encryption_Key);
diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h
index baf6a4b6c0..09822801c7 100644
--- a/src/include/libpq/pqcomm.h
+++ b/src/include/libpq/pqcomm.h
@@ -23,6 +23,8 @@
#endif
#include <netinet/in.h>
+#include "storage/encryption.h"
+
#ifdef HAVE_STRUCT_SOCKADDR_STORAGE
#ifndef HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY
@@ -205,4 +207,26 @@ typedef struct CancelRequestPacket
#define NEGOTIATE_SSL_CODE PG_PROTOCOL(1234,5679)
#define NEGOTIATE_GSS_CODE PG_PROTOCOL(1234,5680)
+/* Client can also send cluster encryption key to the postmaster. */
+#define ENCRYPTION_KEY_MSG_CODE PG_PROTOCOL(1234,5681)
+
+/*
+ * Message containing the encryption key, received by postmaster during
+ * startup.
+ *
+ * TODO Consider adding extension identifier and key_length field so that
+ * postmaster can also receove extension-specific keys. Extension that needs a
+ * key would call a function from _PG_init() that tells postmaster where to
+ * store the key and how long the key is. (Should also postgres in standalone
+ * mode accept keys for extensions?)
+ */
+typedef struct EncryptionKeyMsg
+{
+ /* Note this integer field is stored in network byte order! */
+ MsgType encryptionKeyCode; /* code to identify a key message */
+
+ unsigned char version; /* message format version. */
+ char data[ENCRYPTION_KEY_LENGTH]; /* the key itself. */
+} EncryptionKeyMsg;
+
#endif /* PQCOMM_H */
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
index 8479b76a34..4f7b96d4f3 100644
--- a/src/include/storage/encryption.h
+++ b/src/include/storage/encryption.h
@@ -60,6 +60,29 @@ typedef enum CipherKind
/* Key to encrypt / decrypt data. */
extern unsigned char encryption_key[];
+#ifndef FRONTEND
+/*
+ * Space for the encryption key in shared memory. Backend that receives the
+ * key during startup stores it here so postmaster can eventually take a local
+ * copy.
+ *
+ * We cannot protect the "initialized" field with a spinlock because the data
+ * will be accessed by postmaster, but it should not be necessary for bool
+ * type. Furthermore, synchronization is not really critical here, see
+ * processEncryptionKey().
+ */
+typedef struct ShmemEncryptionKey
+{
+ char data[ENCRYPTION_KEY_LENGTH]; /* the key */
+ bool initialized; /* received the key? */
+} ShmemEncryptionKey;
+
+/*
+ * Encryption key in the shared memory.
+ */
+extern ShmemEncryptionKey *encryption_key_shmem;
+#endif /* FRONTEND */
+
/*
* The encrypted data is a series of blocks of size
* ENCRYPTION_BLOCK. Currently we use the EVP_aes_256_xts implementation. Make
@@ -86,6 +109,9 @@ extern char encryption_verification[];
extern bool encryption_setup_done;
#ifndef FRONTEND
+extern Size EncryptionShmemSize(void);
+extern void EncryptionShmemInit(void);
+
typedef int (*read_encryption_key_cb) (void);
extern void read_encryption_key(read_encryption_key_cb read_char);
#endif /* FRONTEND */
--
2.13.7
--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
filename=v04-0004-Minor-refactoring-of-pg_ctl.c.patch
^ permalink raw reply [nested|flat] 416+ messages in thread
* [PATCH v32 3/3] Avoid some calls to RelationGetRelationName
@ 2020-02-27 01:22 Justin Pryzby <pryzbyj@telsasoft.com>
0 siblings, 0 replies; 416+ messages in thread
From: Justin Pryzby @ 2020-02-27 01:22 UTC (permalink / raw)
---
src/backend/access/heap/vacuumlazy.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3b4c7fe66c..652446dbf2 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -634,8 +634,8 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
}
appendStringInfo(&buf, msgfmt,
get_database_name(MyDatabaseId),
- get_namespace_name(RelationGetNamespace(onerel)),
- RelationGetRelationName(onerel),
+ vacrelstats->relnamespace,
+ vacrelstats->relname,
vacrelstats->num_index_scans);
appendStringInfo(&buf, _("pages: %u removed, %u remain, %u skipped due to pins, %u skipped frozen\n"),
vacrelstats->pages_removed,
@@ -807,7 +807,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (params->nworkers > 0)
ereport(WARNING,
(errmsg("disabling parallel option of vacuum on \"%s\" --- cannot vacuum temporary tables in parallel",
- RelationGetRelationName(onerel))));
+ vacrelstats->relname)));
}
else
lps = begin_parallel_vacuum(RelationGetRelid(onerel), Irel,
@@ -1711,7 +1711,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (vacuumed_pages)
ereport(elevel,
(errmsg("\"%s\": removed %.0f row versions in %u pages",
- RelationGetRelationName(onerel),
+ vacrelstats->relname,
tups_vacuumed, vacuumed_pages)));
/*
@@ -1740,7 +1740,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
ereport(elevel,
(errmsg("\"%s\": found %.0f removable, %.0f nonremovable row versions in %u out of %u pages",
- RelationGetRelationName(onerel),
+ vacrelstats->relname,
tups_vacuumed, num_tuples,
vacrelstats->scanned_pages, nblocks),
errdetail_internal("%s", buf.data)));
@@ -1872,7 +1872,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
ereport(elevel,
(errmsg("\"%s\": removed %d row versions in %d pages",
- RelationGetRelationName(onerel),
+ vacrelstats->relname,
tupindex, npages),
errdetail_internal("%s", pg_rusage_show(&ru0))));
@@ -2420,7 +2420,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ereport(elevel,
(errmsg(msg,
- RelationGetRelationName(indrel),
+ vacrelstats->relname,
dead_tuples->num_tuples),
errdetail_internal("%s", pg_rusage_show(&ru0))));
@@ -2589,7 +2589,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
vacrelstats->lock_waiter_detected = true;
ereport(elevel,
(errmsg("\"%s\": stopping truncate due to conflicting lock request",
- RelationGetRelationName(onerel))));
+ vacrelstats->relname)));
return;
}
@@ -2668,7 +2668,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
ereport(elevel,
(errmsg("\"%s\": truncated %u to %u pages",
- RelationGetRelationName(onerel),
+ vacrelstats->relname,
old_rel_pages, new_rel_pages),
errdetail_internal("%s",
pg_rusage_show(&ru0))));
@@ -2733,7 +2733,7 @@ count_nondeletable_pages(Relation onerel, LVRelStats *vacrelstats)
{
ereport(elevel,
(errmsg("\"%s\": suspending truncate due to conflicting lock request",
- RelationGetRelationName(onerel))));
+ vacrelstats->relname)));
vacrelstats->lock_waiter_detected = true;
return blkno;
--
2.17.0
--MW5yreqqjyrRcusr--
^ permalink raw reply [nested|flat] 416+ messages in thread
* Adding REPACK [concurrently]
@ 2025-07-26 21:56 Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-27 06:00 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-05 08:58 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-10 14:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 7 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-07-26 21:56 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; +Cc: Antonin Houska <ah@cybertec.at>
Hello,
Here's a patch to add REPACK and eventually the CONCURRENTLY flag to it.
This is coming from [1]. The ultimate goal is to have an in-core tool
to allow concurrent table rewrite to get rid of bloat; right now, VACUUM
FULL does that, but it's not concurrent. Users have resorted to using
the pg_repack third-party tool, which is ancient and uses a weird
internal implementation, as well as pg_squeeze, which uses logical
decoding to capture changes that occur during the table rewrite. The
patch submitted here, largely by Antonin Houska with some changes by me,
is based on the the pg_squeeze code which he authored, and first
introduces a new command called REPACK to absorb both VACUUM FULL and
CLUSTER, followed by addition of a CONCURRENTLY flag to allow some forms
of REPACK to operate online using logical decoding.
Essentially, this first patch just reshuffles the CLUSTER code to create
the REPACK command.
I made a few changes from Antonin's original at [2]. First, I modified
the grammar to support "REPACK [tab] USING INDEX" without specifying the
index name. With this change, all possibilities of the old commands are
covered, which gives us the chance to flag them as obsolete. (This is
good, because having VACUUM FULL do something completely different from
regular VACUUM confuses users all the time; and on the other hand,
having a command called CLUSTER which is at odds with what most people
think of as a "database cluster" is also confusing.)
Here's a list of existing commands, and how to write them in the current
patch's proposal for REPACK:
-- re-clusters all tables that have a clustered index set
CLUSTER -> REPACK USING INDEX
-- clusters the given table using the given index
CLUSTER tab USING idx -> REPACK tab USING INDEX idx
-- clusters this table using a clustered index; error if no index clustered
CLUSTER tab -> REPACK tab USING INDEX
-- vacuum-full all tables
VACUUM FULL -> REPACK
-- vacuum-full the specified table
VACUUM FULL tab -> REPACK tab
My other change to Antonin's patch is that I made REPACK USING INDEX set
the 'indisclustered' flag to the index being used, so REPACK behaves
identically to CLUSTER. We can discuss whether we really want this.
For instance we could add an option so that by default REPACK omits
persisting the clustered index, and instead it only does that when you
give it some special option, say something like
"REPACK (persist_clustered_index=true) tab USING INDEX idx"
Overall I'm not sure this is terribly interesting, since clustered
indexes are not very useful for most users anyway.
I made a few other minor changes not worthy of individual mention, and
there are a few others pending, such as updates to the
pg_stat_progress_repack view infrastructure, as well as phasing out
pg_stat_progress_cluster (maybe the latter would offer a subset of the
former; not yet sure about this.) Also, I'd like to work on adding a
`repackdb` command for completeness.
On repackdb: I think is going to be very similar to vacuumdb, mostly in
that it is going to need to be able to run tasks in parallel; but there
are things it doesn't have to deal with, such as analyze-in-stages,
which I think is a large burden. I estimate about 1k LOC there,
extremely similar to vacuumdb. Maybe it makes sense to share the source
code and make the new executable a symlink instead, with some additional
code to support the two different modes. Again, I'm not sure about
this -- I like the idea, but I'd have to see the implementation.
I'll be rebasing the rest of Antonin's patch series afterwards,
including the logical decoding changes necessary for CONCURRENTLY. In
the meantime, if people want to review those, which would be very
valuable, they can go back to branch master from around the time he
submitted it and apply the old patches there.
[1] https://postgr.es/m/76278.1724760050@antos
[2] https://postgr.es/m/152010.1751307725@localhost
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-07-27 01:59 ` Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
6 siblings, 1 reply; 416+ messages in thread
From: Robert Treat @ 2025-07-27 01:59 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
On Sat, Jul 26, 2025 at 5:56 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Hello,
>
> Here's a patch to add REPACK and eventually the CONCURRENTLY flag to it.
> This is coming from [1]. The ultimate goal is to have an in-core tool
> to allow concurrent table rewrite to get rid of bloat; right now, VACUUM
> FULL does that, but it's not concurrent. Users have resorted to using
> the pg_repack third-party tool, which is ancient and uses a weird
> internal implementation, as well as pg_squeeze, which uses logical
> decoding to capture changes that occur during the table rewrite. The
> patch submitted here, largely by Antonin Houska with some changes by me,
> is based on the the pg_squeeze code which he authored, and first
> introduces a new command called REPACK to absorb both VACUUM FULL and
> CLUSTER, followed by addition of a CONCURRENTLY flag to allow some forms
> of REPACK to operate online using logical decoding.
>
> Essentially, this first patch just reshuffles the CLUSTER code to create
> the REPACK command.
>
Thanks for keeping this ball rolling.
>
> My other change to Antonin's patch is that I made REPACK USING INDEX set
> the 'indisclustered' flag to the index being used, so REPACK behaves
> identically to CLUSTER. We can discuss whether we really want this.
> For instance we could add an option so that by default REPACK omits
> persisting the clustered index, and instead it only does that when you
> give it some special option, say something like
> "REPACK (persist_clustered_index=true) tab USING INDEX idx"
> Overall I'm not sure this is terribly interesting, since clustered
> indexes are not very useful for most users anyway.
>
I think I would lean towards having it work like CLUSTER (preserve the
index), since that helps people making the transition, and it doesn't
feel terribly useful to invent new syntax for a feature that I would
agree isn't very useful for most people.
> I made a few other minor changes not worthy of individual mention, and
> there are a few others pending, such as updates to the
> pg_stat_progress_repack view infrastructure, as well as phasing out
> pg_stat_progress_cluster (maybe the latter would offer a subset of the
> former; not yet sure about this.) Also, I'd like to work on adding a
> `repackdb` command for completeness.
>
> On repackdb: I think is going to be very similar to vacuumdb, mostly in
> that it is going to need to be able to run tasks in parallel; but there
> are things it doesn't have to deal with, such as analyze-in-stages,
> which I think is a large burden. I estimate about 1k LOC there,
> extremely similar to vacuumdb. Maybe it makes sense to share the source
> code and make the new executable a symlink instead, with some additional
> code to support the two different modes. Again, I'm not sure about
> this -- I like the idea, but I'd have to see the implementation.
>
> I'll be rebasing the rest of Antonin's patch series afterwards,
> including the logical decoding changes necessary for CONCURRENTLY. In
> the meantime, if people want to review those, which would be very
> valuable, they can go back to branch master from around the time he
> submitted it and apply the old patches there.
>
For clarity, are you intending to commit this patch before having the
other parts ready? (If that sounds like an objection, it isn't) After
a first pass, I think there's some confusing bits in the new docs that
could use straightening out, but there likely going to overlap changes
once concurrently is brought in, so it might make sense to hold off on
those. Either way I definitely want to dive into this a bit deeper
with some fresh eyes, there's a lot to digest... speaking of, for this
bit in src/backend/commands/cluster.c
+ switch (cmd)
+ {
+ case REPACK_COMMAND_REPACK:
+ return "REPACK";
+ case REPACK_COMMAND_VACUUMFULL:
+ return "VACUUM";
+ case REPACK_COMMAND_CLUSTER:
+ return "VACUUM";
+ }
+ return "???";
The last one should return "CLUSTER" no?
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-07-31 16:50 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2025-07-31 16:50 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
On 2025-Jul-26, Robert Treat wrote:
> For clarity, are you intending to commit this patch before having the
> other parts ready? (If that sounds like an objection, it isn't) After
> a first pass, I think there's some confusing bits in the new docs that
> could use straightening out, but there likely going to overlap changes
> once concurrently is brought in, so it might make sense to hold off on
> those.
I'm aiming at getting 0001 committed during the September commitfest,
and the CONCURRENTLY flag addition later in the pg19 cycle. But I'd
rather have good-enough docs at every step of the way. They don't have
to be *perfect* if we want to get everything in pg19, but I'd rather not
leave anything openly confusing even transiently.
That said, I did not review the docs this time around, so here's them
the same as they were in the previous post. But if you want to suggest
changes for the docs in 0001, please do. Just don't get too carried
away.
> speaking of, for this bit in src/backend/commands/cluster.c
>
> + switch (cmd)
> + {
> + case REPACK_COMMAND_REPACK:
> + return "REPACK";
> + case REPACK_COMMAND_VACUUMFULL:
> + return "VACUUM";
> + case REPACK_COMMAND_CLUSTER:
> + return "VACUUM";
> + }
> + return "???";
>
> The last one should return "CLUSTER" no?
Absolutely -- my blunder.
On 2025-Jul-27, Fujii Masao wrote:
> > The patch submitted here, largely by Antonin Houska with some
> > changes by me, is based on the the pg_squeeze code which he
> > authored, and first introduces a new command called REPACK to absorb
> > both VACUUM FULL and CLUSTER, followed by addition of a CONCURRENTLY
> > flag to allow some forms of REPACK to operate online using logical
> > decoding.
>
> Does this mean REPACK CONCURRENTLY requires wal_level = logical, while
> plain REPACK (without CONCURRENTLY) works with any wal_level setting?
> If we eventually deprecate VACUUM FULL and CLUSTER, I think plain
> REPACK should still be allowed with wal_level = minimal or replica, so
> users with those settings can perform equivalent processing.
Absolutely.
One of the later patches in the series, which I have not included yet,
intends to implement the idea of transiently enabling wal_level=logical
for the table being repacked concurrently, so that you can still use
the concurrent mode if you have a non-logical-wal_level instance.
> + if (!cluster_is_permitted_for_relation(tableOid, userid,
> + CLUSTER_COMMAND_CLUSTER))
>
> As for the patch you attached, it seems to be an early WIP and
> might not be ready for review yet?? BTW, I got the following
> compilation failure and probably CLUSTER_COMMAND_CLUSTER
> the above should be GetUserId().
This was a silly merge mistake, caused by my squashing Antonin's 0004
(trivial code restructuring) into 0001 at the last minute and failing to
"git add" the compile fixes before doing git-format-patch.
Here's v17. (I decided that calling my previous one "v1" after Antonin
had gone all the way to v15 was stupid on my part.) The important part
here is that I rebased Antonin 0004's, that is, the addition of the
CONCURRENTLY flag, plus 0005 regression tests.
The only interesting change here is that I decided to not mess with the
grammar by allowing an unparenthesized CONCURRENTLY keyword; if you want
concurrent, you have to say "REPACK (CONCURRENTLY)". This is at odds
with the way we use the keyword in other commands, but ISTM we don't
_need_ to support that legacy syntax. Anyway, this is easy to put back
afterwards, if enough people find it not useless.
I've not reviewed 0003 in depth yet, just rebased it. But it works to
the point that CI is happy with it.
I've not yet included Antonin's 0006 and 0007.
TODO list for 0001:
- addition of src/bin/scripts/repackdb
- clean up the progress report infrastructure
- doc review
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Thou shalt check the array bounds of all strings (indeed, all arrays), for
surely where thou typest "foo" someone someday shall type
"supercalifragilisticexpialidocious" (5th Commandment for C programmers)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-01 11:07 ` Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Fujii Masao @ 2025-08-01 11:07 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
On Fri, Aug 1, 2025 at 1:50 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> One of the later patches in the series, which I have not included yet,
> intends to implement the idea of transiently enabling wal_level=logical
> for the table being repacked concurrently, so that you can still use
> the concurrent mode if you have a non-logical-wal_level instance.
Sounds good to me!
> Here's v17.
I just tried REPACK command and observed a few things:
When I repeatedly ran REPACK on the regression database
while make installcheck was running, I got the following error:
ERROR: StartTransactionCommand: unexpected state STARTED
"REPACK (VERBOSE);" failed with the following error.
ERROR: syntax error at or near ";"
REPACK (CONCURRENTLY) USING INDEX failed with the following error,
while the same command without CONCURRENTLY completed successfully:
=# REPACK (CONCURRENTLY) parallel_vacuum_table using index
regular_sized_index ;
ERROR: cannot process relation "parallel_vacuum_table"
HINT: Relation "parallel_vacuum_table" has no identity index.
When I ran REPACK (CONCURRENTLY) on a table that's also a logical
replication target, I saw the following log messages. Is this expected?
=# REPACK (CONCURRENTLY) t;
LOG: logical decoding found consistent point at 1/00021F20
DETAIL: There are no running transactions.
STATEMENT: REPACK (CONCURRENTLY) t;
Regards,
--
Fujii Masao
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
@ 2025-08-04 23:21 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-04 23:21 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
Hello Álvaro,
Should we skip non-ready indexes in build_new_indexes?
Also, in the current implementation, concurrent mode is marked as
non-MVCC safe. From my point of view, this is a significant limitation
for practical use.
Should we consider an option to exchange non-MVCC issues to short
exclusive lock of ProcArrayLock + cancellation of some transactions
with older xmin?
Best regards,
Mikhail
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-09 12:55 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-11 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-09 12:55 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
Hello!
One more thing - I think build_new_indexes and
index_concurrently_create_copy are very close in semantics, so it
might be a good idea to refactor them a bit.
I’m still concerned about MVCC-related issues. For multiple
applications, this is a dealbreaker, because in some cases correctness
is a higher priority than availability.
Possible options:
1) Terminate connections with old snapshots.
Add a flag to terminate all connections with snapshots during the
ExclusiveLock period for the swap. From the application’s perspective,
this is not a big deal - it's similar to a primary switch. We would
also need to prevent new snapshots from being taken during the swap
transaction, so a short exclusive lock on ProcArrayLock would also be
required.
2) MVCC-safe two-phase approach (inspired by CREATE INDEX).
- copy the data from T1 to the new table T2.
- apply the log.
- take a table-exclusive lock on T1
- apply the log again.
- instead of swapping, mark the T2 as a kind of shadow table - any
transaction applying changes to T1 must also apply them to T2, while
reads still use T1 as the source of truth.
- commit (and record the transaction ID as XID1).
- at this point, all changes are applied to both tables with the same
XIDs because of the "shadow table" mechanism.
- wait until older snapshots no longer treat XID1 as uncommitted.
- now the tables are identical from the MVCC perspective.
- take an exclusive lock on both T1 and T2.
- perform the swap and drop T1.
- commit.
This is more complex and would require implementing some sort of
"shadow table" mechanism, so it might not be worth the effort. Option
1 feels more appealing to me.
If others think this is a good idea, I might try implementing a proof
of concept.
Best regards,
Mikhail
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-09 13:33 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-09 13:33 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
On 2025-Aug-09, Mihail Nikalayeu wrote:
> Hello!
>
> One more thing - I think build_new_indexes and
> index_concurrently_create_copy are very close in semantics, so it
> might be a good idea to refactor them a bit.
>
> I’m still concerned about MVCC-related issues. For multiple
> applications, this is a dealbreaker, because in some cases correctness
> is a higher priority than availability.
Please note that Antonin already implemented this. See his patches
here:
https://www.postgresql.org/message-id/77690.1725610115%40antos
I proposed to leave this part out initially, which is why it hasn't been
reposted. We can review and discuss after the initial patches are in.
Because having an MVCC-safe mode has drawbacks, IMO we should make it
optional.
But you're welcome to review that part specifically if you're so
inclined, and offer feedback on it. (I suggest to rewind back your
checked-out tree to branch master at the time that patch was posted, for
easy application. We can deal with a rebase later.)
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Officer Krupke, what are we to do?
Gee, officer Krupke, Krup you! (West Side Story, "Gee, Officer Krupke")
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-20 23:44 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-28 21:39 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-20 23:44 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
Hello everyone!
Alvaro Herrera <alvherre@alvh.no-ip.org>:
> Please note that Antonin already implemented this. See his patches
> here:
> https://www.postgresql.org/message-id/77690.1725610115%40antos
> I proposed to leave this part out initially, which is why it hasn't been
> reposted. We can review and discuss after the initial patches are in.
I think it is worth pushing it at least in the same release cycle.
> But you're welcome to review that part specifically if you're so
> inclined, and offer feedback on it. (I suggest to rewind back your
> checked-out tree to branch master at the time that patch was posted, for
> easy application. We can deal with a rebase later.)
I have rebased that on top of v18 (attached).
Also, I think I found an issue (or lost something during rebase): we
must preserve xmin,cmin during initial copy
to make sure that data is going to be visible by snapshots of
concurrent changes later:
static void
reform_and_rewrite_tuple(......)
.....
/*It is also crucial to stamp the new record with the exact same
xid and cid,
* because the tuple must be visible to the snapshot of the
applied concurrent
* change later.
*/
CommandId cid = HeapTupleHeaderGetRawCommandId(tuple->t_data);
TransactionId xid = HeapTupleHeaderGetXmin(tuple->t_data);
heap_insert(NewHeap, copiedTuple, xid, cid, HEAP_INSERT_NO_LOGICAL, NULL);
I'll try to polish that part a little bit.
> Because having an MVCC-safe mode has drawbacks, IMO we should make it
> optional.
Do you mean some option for the command? Like REPACK (CONCURRENTLY, SAFE)?
Best regards,
Mikhail.
Attachments:
[application/octet-stream] v18-0006-Preserve-visibility-information-of-the-concurren.patch (56.7K, ../../CADzfLwW=b=U3e6aasi=XorN8hZSiCKZErKs9qhyK7m=w=wokAg@mail.gmail.com/2-v18-0006-Preserve-visibility-information-of-the-concurren.patch)
download | inline diff:
From b2f4d126e04d0396f8ad69f5113ed405a4efd723 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Thu, 21 Aug 2025 00:19:24 +0200
Subject: [PATCH v18 6/6] Preserve visibility information of the concurrent
data changes.
As explained in the commit message of the preceding patch of the series, the
data changes done by applications while REPACK CONCURRENTLY is copying the
table contents to a new file are decoded from WAL and eventually also applied
to the new file. To reduce the complexity a little bit, the preceding patch
uses the current transaction (i.e. transaction opened by the REPACK command)
to execute those INSERT, UPDATE and DELETE commands.
However, REPACK is not expected to change visibility of tuples. Therefore,
this patch fixes the handling of the "concurrent data changes". It ensures
that tuples written into the new table have the same XID and command ID (CID)
as they had in the old table.
To "replay" an UPDATE or DELETE command on the new table, we need the
appropriate snapshot to find the previous tuple version in the new table. The
(historic) snapshot we used to decode the UPDATE / DELETE should (by
definition) see the state of the catalog prior to that UPDATE / DELETE. Thus
we can use the same snapshot to find the "old tuple" for UPDATE / DELETE in
the new table if:
1) REPACK CONCURRENTLY preserves visibility information of all tuples - that's
the purpose of this part of the patch series.
2) The table being REPACKed is treated as a system catalog by all transactions
that modify its data. This ensures that reorderbuffer.c generates a new
snapshot for each data change in the table.
We ensure 2) by maintaining a shared hashtable of tables being REPACKed
CONCURRENTLY and by adjusting the RelationIsAccessibleInLogicalDecoding()
macro so it checks this hashtable. (The corresponding flag is also added to
the relation cache, so that the shared hashtable does not have to be accessed
too often.) It's essential that after adding an entry to the hashtable we wait
for completion of all the transactions that might have started to modify our
table before our entry has was added. We achieve that by upgrading our lock on
the table to ShareLock temporarily: as soon as we acquire it, no DML command
should be running on the table. (This lock upgrade shouldn't cause any
deadlock because we care to not hold a lock on other objects at the same
time.)
As long as we preserve the tuple visibility information (which includes XID),
it's important to avoid logical decoding of the WAL generated by DMLs on the
new table: the logical decoding subsystem probably does not expect that the
incoming WAL records contain XIDs of an already decoded transactions. (And of
course, repeated decoding would be wasted effort.)
Author: Antonin Houska <ah@cybertec.at> with small changes from Mikhail Nikalayeu <mihailnikalayeu@gmail.com
>
---
src/backend/access/common/toast_internals.c | 3 +-
src/backend/access/heap/heapam.c | 54 ++-
src/backend/access/heap/heapam_handler.c | 23 +-
src/backend/access/transam/xact.c | 52 +++
src/backend/commands/cluster.c | 400 ++++++++++++++++--
src/backend/replication/logical/decode.c | 28 +-
src/backend/replication/logical/snapbuild.c | 22 +-
.../pgoutput_repack/pgoutput_repack.c | 68 ++-
src/backend/storage/ipc/ipci.c | 2 +
.../utils/activity/wait_event_names.txt | 1 +
src/backend/utils/cache/inval.c | 21 +
src/backend/utils/cache/relcache.c | 4 +
src/include/access/heapam.h | 12 +-
src/include/access/xact.h | 2 +
src/include/commands/cluster.h | 22 +
src/include/storage/lwlocklist.h | 1 +
src/include/utils/inval.h | 2 +
src/include/utils/rel.h | 7 +-
src/include/utils/snapshot.h | 3 +
.../injection_points/specs/repack.spec | 4 -
src/tools/pgindent/typedefs.list | 1 +
21 files changed, 636 insertions(+), 96 deletions(-)
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index a1d0eed8953..586eb42a137 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -320,7 +320,8 @@ toast_save_datum(Relation rel, Datum value,
memcpy(VARDATA(&chunk_data), data_p, chunk_size);
toasttup = heap_form_tuple(toasttupDesc, t_values, t_isnull);
- heap_insert(toastrel, toasttup, mycid, options, NULL);
+ heap_insert(toastrel, toasttup, GetCurrentTransactionId(), mycid,
+ options, NULL);
/*
* Create the index entry. We cheat a little here by not using
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4fdb3e880e4..e7b9f7b6374 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2059,7 +2059,7 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
/*
* heap_insert - insert tuple into a heap
*
- * The new tuple is stamped with current transaction ID and the specified
+ * The new tuple is stamped with specified transaction ID and the specified
* command ID.
*
* See table_tuple_insert for comments about most of the input flags, except
@@ -2075,15 +2075,16 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
* reflected into *tup.
*/
void
-heap_insert(Relation relation, HeapTuple tup, CommandId cid,
- int options, BulkInsertState bistate)
+heap_insert(Relation relation, HeapTuple tup, TransactionId xid,
+ CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
bool all_visible_cleared = false;
+ Assert(TransactionIdIsValid(xid));
+
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(tup->t_data) <=
RelationGetNumberOfAttributes(relation));
@@ -2165,8 +2166,15 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
/*
* If this is a catalog, we need to transmit combo CIDs to properly
* decode, so log that as well.
+ *
+ * HEAP_INSERT_NO_LOGICAL should be set when applying data changes
+ * done by other transactions during REPACK CONCURRENTLY. In such a
+ * case, the insertion should not be decoded at all - see
+ * heap_decode(). (It's also set by raw_heap_insert() for TOAST, but
+ * TOAST does not pass this test anyway.)
*/
- if (RelationIsAccessibleInLogicalDecoding(relation))
+ if ((options & HEAP_INSERT_NO_LOGICAL) == 0 &&
+ RelationIsAccessibleInLogicalDecoding(relation))
log_heap_new_cid(relation, heaptup);
/*
@@ -2712,7 +2720,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
void
simple_heap_insert(Relation relation, HeapTuple tup)
{
- heap_insert(relation, tup, GetCurrentCommandId(true), 0, NULL);
+ heap_insert(relation, tup, GetCurrentTransactionId(),
+ GetCurrentCommandId(true), 0, NULL);
}
/*
@@ -2769,11 +2778,11 @@ xmax_infomask_changed(uint16 new_infomask, uint16 old_infomask)
*/
TM_Result
heap_delete(Relation relation, ItemPointer tid,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, bool changingPart, bool wal_logical)
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, bool changingPart,
+ bool wal_logical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
ItemId lp;
HeapTupleData tp;
Page page;
@@ -2790,6 +2799,7 @@ heap_delete(Relation relation, ItemPointer tid,
bool old_key_copied = false;
Assert(ItemPointerIsValid(tid));
+ Assert(TransactionIdIsValid(xid));
AssertHasSnapshotForToast(relation);
@@ -3086,8 +3096,12 @@ l1:
/*
* For logical decode we need combo CIDs to properly decode the
* catalog
+ *
+ * Like in heap_insert(), visibility is unchanged when called from
+ * VACUUM FULL / CLUSTER.
*/
- if (RelationIsAccessibleInLogicalDecoding(relation))
+ if (wal_logical &&
+ RelationIsAccessibleInLogicalDecoding(relation))
log_heap_new_cid(relation, &tp);
xlrec.flags = 0;
@@ -3206,11 +3220,11 @@ simple_heap_delete(Relation relation, ItemPointer tid)
TM_Result result;
TM_FailureData tmfd;
- result = heap_delete(relation, tid,
+ result = heap_delete(relation, tid, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, false, /* changingPart */
- true /* wal_logical */);
+ &tmfd, false, /* changingPart */
+ true /* wal_logical */ );
switch (result)
{
case TM_SelfModified:
@@ -3249,12 +3263,11 @@ simple_heap_delete(Relation relation, ItemPointer tid)
*/
TM_Result
heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
+ TransactionId xid, CommandId cid, Snapshot crosscheck,
+ bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool wal_logical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
Bitmapset *key_attrs;
@@ -3294,6 +3307,7 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
infomask2_new_tuple;
Assert(ItemPointerIsValid(otid));
+ Assert(TransactionIdIsValid(xid));
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
@@ -4133,8 +4147,12 @@ l2:
/*
* For logical decoding we need combo CIDs to properly decode the
* catalog.
+ *
+ * Like in heap_insert(), visibility is unchanged when called from
+ * VACUUM FULL / CLUSTER.
*/
- if (RelationIsAccessibleInLogicalDecoding(relation))
+ if (wal_logical &&
+ RelationIsAccessibleInLogicalDecoding(relation))
{
log_heap_new_cid(relation, &oldtup);
log_heap_new_cid(relation, heaptup);
@@ -4500,7 +4518,7 @@ simple_heap_update(Relation relation, ItemPointer otid, HeapTuple tup,
TM_FailureData tmfd;
LockTupleMode lockmode;
- result = heap_update(relation, otid, tup,
+ result = heap_update(relation, otid, tup, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode, update_indexes,
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c829c06f769..c42a1bd55ee 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -253,7 +253,8 @@ heapam_tuple_insert(Relation relation, TupleTableSlot *slot, CommandId cid,
tuple->t_tableOid = slot->tts_tableOid;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -276,7 +277,8 @@ heapam_tuple_insert_speculative(Relation relation, TupleTableSlot *slot,
options |= HEAP_INSERT_SPECULATIVE;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -310,8 +312,8 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
* the storage itself is cleaning the dead tuples by itself, it is the
* time to call the index tuple deletion also.
*/
- return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart,
- true);
+ return heap_delete(relation, tid, GetCurrentTransactionId(), cid,
+ crosscheck, wait, tmfd, changingPart, true);
}
@@ -329,7 +331,8 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
+ result = heap_update(relation, otid, tuple, GetCurrentTransactionId(),
+ cid, crosscheck, wait,
tmfd, lockmode, update_indexes, true);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
@@ -2477,9 +2480,15 @@ reform_and_rewrite_tuple(HeapTuple tuple,
* flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps
* the relation files, it drops this relation, so no logical
* replication subscription should need the data.
+ *
+ * It is also crucial to stamp the new record with the exact same xid and cid,
+ * because the tuple must be visible to the snapshot of the applied concurrent
+ * change later.
*/
- heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true),
- HEAP_INSERT_NO_LOGICAL, NULL);
+ CommandId cid = HeapTupleHeaderGetRawCommandId(tuple->t_data);
+ TransactionId xid = HeapTupleHeaderGetXmin(tuple->t_data);
+
+ heap_insert(NewHeap, copiedTuple, xid, cid, HEAP_INSERT_NO_LOGICAL, NULL);
}
heap_freetuple(copiedTuple);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 5670f2bfbde..e913594fc07 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -126,6 +126,18 @@ static FullTransactionId XactTopFullTransactionId = {InvalidTransactionId};
static int nParallelCurrentXids = 0;
static TransactionId *ParallelCurrentXids;
+/*
+ * Another case that requires TransactionIdIsCurrentTransactionId() to behave
+ * specially is when REPACK CONCURRENTLY is processing data changes made in
+ * the old storage of a table by other transactions. When applying the changes
+ * to the new storage, the backend executing the CLUSTER command needs to act
+ * on behalf on those other transactions. The transactions responsible for the
+ * changes in the old storage are stored in this array, sorted by
+ * xidComparator.
+ */
+static int nRepackCurrentXids = 0;
+static TransactionId *RepackCurrentXids = NULL;
+
/*
* Miscellaneous flag bits to record events which occur on the top level
* transaction. These flags are only persisted in MyXactFlags and are intended
@@ -973,6 +985,8 @@ TransactionIdIsCurrentTransactionId(TransactionId xid)
int low,
high;
+ Assert(nRepackCurrentXids == 0);
+
low = 0;
high = nParallelCurrentXids - 1;
while (low <= high)
@@ -992,6 +1006,21 @@ TransactionIdIsCurrentTransactionId(TransactionId xid)
return false;
}
+ /*
+ * When executing CLUSTER CONCURRENTLY, the array of current transactions
+ * is given.
+ */
+ if (nRepackCurrentXids > 0)
+ {
+ Assert(nParallelCurrentXids == 0);
+
+ return bsearch(&xid,
+ RepackCurrentXids,
+ nRepackCurrentXids,
+ sizeof(TransactionId),
+ xidComparator) != NULL;
+ }
+
/*
* We will return true for the Xid of the current subtransaction, any of
* its subcommitted children, any of its parents, or any of their
@@ -5661,6 +5690,29 @@ EndParallelWorkerTransaction(void)
CurrentTransactionState->blockState = TBLOCK_DEFAULT;
}
+/*
+ * SetRepackCurrentXids
+ * Set the XID array that TransactionIdIsCurrentTransactionId() should
+ * use.
+ */
+void
+SetRepackCurrentXids(TransactionId *xip, int xcnt)
+{
+ RepackCurrentXids = xip;
+ nRepackCurrentXids = xcnt;
+}
+
+/*
+ * ResetRepackCurrentXids
+ * Undo the effect of SetRepackCurrentXids().
+ */
+void
+ResetRepackCurrentXids(void)
+{
+ RepackCurrentXids = NULL;
+ nRepackCurrentXids = 0;
+}
+
/*
* ShowTransactionState
* Debug support
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index aa3ae85bcee..a4d4d37d211 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -82,6 +82,11 @@ typedef struct
* The following definitions are used for concurrent processing.
*/
+/*
+ * OID of the table being repacked by this backend.
+ */
+static Oid repacked_rel = InvalidOid;
+
/*
* The locators are used to avoid logical decoding of data that we do not need
* for our table.
@@ -125,8 +130,10 @@ static List *get_tables_to_repack_partitioned(RepackCommand cmd,
static bool cluster_is_permitted_for_relation(RepackCommand cmd,
Oid relid, Oid userid);
-static void begin_concurrent_repack(Relation rel);
-static void end_concurrent_repack(void);
+static void begin_concurrent_repack(Relation rel, Relation *index_p,
+ bool *entered_p);
+static void end_concurrent_repack(bool error);
+static void cluster_before_shmem_exit_callback(int code, Datum arg);
static LogicalDecodingContext *setup_logical_decoding(Oid relid,
const char *slotname,
TupleDesc tupdesc);
@@ -146,6 +153,7 @@ static void apply_concurrent_delete(Relation rel, HeapTuple tup_target,
ConcurrentChange *change);
static HeapTuple find_target_tuple(Relation rel, ScanKey key, int nkeys,
HeapTuple tup_key,
+ Snapshot snapshot,
IndexInsertState *iistate,
TupleTableSlot *ident_slot,
IndexScanDesc *scan_p);
@@ -437,6 +445,8 @@ cluster_rel(RepackCommand cmd, bool usingindex,
bool verbose = ((params->options & CLUOPT_VERBOSE) != 0);
bool recheck = ((params->options & CLUOPT_RECHECK) != 0);
bool concurrent = ((params->options & CLUOPT_CONCURRENT) != 0);
+ bool entered,
+ success;
/*
* Check that the correct lock is held. The lock mode is
@@ -607,23 +617,30 @@ cluster_rel(RepackCommand cmd, bool usingindex,
TransferPredicateLocksToHeapRelation(OldHeap);
/* rebuild_relation does all the dirty work */
+ entered = false;
+ success = false;
PG_TRY();
{
/*
- * For concurrent processing, make sure that our logical decoding
- * ignores data changes of other tables than the one we are
- * processing.
+ * For concurrent processing, make sure that
+ *
+ * 1) our logical decoding ignores data changes of other tables than
+ * the one we are processing.
+ *
+ * 2) other transactions treat this table as if it was a system / user
+ * catalog, and WAL the relevant additional information.
*/
if (concurrent)
- begin_concurrent_repack(OldHeap);
+ begin_concurrent_repack(OldHeap, &index, &entered);
rebuild_relation(cmd, usingindex, OldHeap, index, save_userid,
verbose, concurrent);
+ success = true;
}
PG_FINALLY();
{
- if (concurrent)
- end_concurrent_repack();
+ if (concurrent && entered)
+ end_concurrent_repack(!success);
}
PG_END_TRY();
@@ -2383,6 +2400,47 @@ determine_clustered_index(Relation rel, bool usingindex, const char *indexname)
}
+/*
+ * Each relation being processed by REPACK CONCURRENTLY must be in the
+ * repackedRels hashtable.
+ */
+typedef struct RepackedRel
+{
+ Oid relid;
+ Oid dbid;
+} RepackedRel;
+
+static HTAB *RepackedRelsHash = NULL;
+
+/*
+ * Maximum number of entries in the hashtable.
+ *
+ * A replication slot is needed for the processing, so use this GUC to
+ * allocate memory for the hashtable.
+ */
+#define MAX_REPACKED_RELS (max_replication_slots)
+
+Size
+RepackShmemSize(void)
+{
+ return hash_estimate_size(MAX_REPACKED_RELS, sizeof(RepackedRel));
+}
+
+void
+RepackShmemInit(void)
+{
+ HASHCTL info;
+
+ info.keysize = sizeof(RepackedRel);
+ info.entrysize = info.keysize;
+
+ RepackedRelsHash = ShmemInitHash("Repacked Relations",
+ MAX_REPACKED_RELS,
+ MAX_REPACKED_RELS,
+ &info,
+ HASH_ELEM | HASH_BLOBS);
+}
+
/*
* Call this function before REPACK CONCURRENTLY starts to setup logical
* decoding. It makes sure that other users of the table put enough
@@ -2397,11 +2455,119 @@ determine_clustered_index(Relation rel, bool usingindex, const char *indexname)
*
* Note that TOAST table needs no attention here as it's not scanned using
* historic snapshot.
+ *
+ * 'index_p' is in/out argument because the function unlocks the index
+ * temporarily.
+ *
+ * 'enter_p' receives a bool value telling whether relation OID was entered
+ * into RepackedRelsHash or not.
*/
static void
-begin_concurrent_repack(Relation rel)
+begin_concurrent_repack(Relation rel, Relation *index_p, bool *entered_p)
{
- Oid toastrelid;
+ Oid relid,
+ toastrelid;
+ Relation index = NULL;
+ Oid indexid = InvalidOid;
+ RepackedRel key,
+ *entry;
+ bool found;
+ static bool before_shmem_exit_callback_setup = false;
+
+ relid = RelationGetRelid(rel);
+ index = index_p ? *index_p : NULL;
+
+ /*
+ * Make sure that we do not leave an entry in RepackedRelsHash if exiting
+ * due to FATAL.
+ */
+ if (!before_shmem_exit_callback_setup)
+ {
+ before_shmem_exit(cluster_before_shmem_exit_callback, 0);
+ before_shmem_exit_callback_setup = true;
+ }
+
+ memset(&key, 0, sizeof(key));
+ key.relid = relid;
+ key.dbid = MyDatabaseId;
+
+ *entered_p = false;
+ LWLockAcquire(RepackedRelsLock, LW_EXCLUSIVE);
+ entry = (RepackedRel *)
+ hash_search(RepackedRelsHash, &key, HASH_ENTER_NULL, &found);
+ if (found)
+ {
+ /*
+ * Since REPACK CONCURRENTLY takes ShareRowExclusiveLock, a conflict
+ * should occur much earlier. However that lock may be released
+ * temporarily, see below. Anyway, we should complain whatever the
+ * reason of the conflict might be.
+ */
+ ereport(ERROR,
+ (errmsg("relation \"%s\" is already being processed by REPACK CONCURRENTLY",
+ RelationGetRelationName(rel))));
+ }
+ if (entry == NULL)
+ ereport(ERROR,
+ (errmsg("too many requests for REPACK CONCURRENTLY at a time")),
+ (errhint("Please consider increasing the \"max_replication_slots\" configuration parameter.")));
+
+ /*
+ * Even if anything fails below, the caller has to do cleanup in the
+ * shared memory.
+ */
+ *entered_p = true;
+
+ /*
+ * Enable the callback to remove the entry in case of exit. We should not
+ * do this earlier, otherwise an attempt to insert already existing entry
+ * could make us remove that entry (inserted by another backend) during
+ * ERROR handling.
+ */
+ Assert(!OidIsValid(repacked_rel));
+ repacked_rel = relid;
+
+ LWLockRelease(RepackedRelsLock);
+
+ /*
+ * Make sure that other backends are aware of the new hash entry as soon
+ * as they open our table.
+ */
+ CacheInvalidateRelcacheImmediate(relid);
+
+ /*
+ * Also make sure that the existing users of the table update their
+ * relcache entry as soon as they try to run DML commands on it.
+ *
+ * ShareLock is the weakest lock that conflicts with DMLs. If any backend
+ * has a lower lock, we assume it'll accept our invalidation message when
+ * it changes the lock mode.
+ *
+ * Before upgrading the lock on the relation, close the index temporarily
+ * to avoid a deadlock if another backend running DML already has its lock
+ * (ShareLock) on the table and waits for the lock on the index.
+ */
+ if (index)
+ {
+ indexid = RelationGetRelid(index);
+ index_close(index, ShareUpdateExclusiveLock);
+ }
+ LockRelationOid(relid, ShareLock);
+ UnlockRelationOid(relid, ShareLock);
+ if (OidIsValid(indexid))
+ {
+ /*
+ * Re-open the index and check that it hasn't changed while unlocked.
+ */
+ check_index_is_clusterable(rel, indexid, ShareUpdateExclusiveLock);
+
+ /*
+ * Return the new relcache entry to the caller. (It's been locked by
+ * the call above.)
+ */
+ index = index_open(indexid, NoLock);
+ *index_p = index;
+ }
/* Avoid logical decoding of other relations by this backend. */
repacked_rel_locator = rel->rd_locator;
@@ -2419,15 +2585,122 @@ begin_concurrent_repack(Relation rel)
/*
* Call this when done with REPACK CONCURRENTLY.
+ *
+ * 'error' tells whether the function is being called in order to handle
+ * error.
*/
static void
-end_concurrent_repack(void)
+end_concurrent_repack(bool error)
{
+ RepackedRel key;
+ RepackedRel *entry = NULL;
+ Oid relid = repacked_rel;
+
+ /* Remove the relation from the hash if we managed to insert one. */
+ if (OidIsValid(repacked_rel))
+ {
+ memset(&key, 0, sizeof(key));
+ key.relid = repacked_rel;
+ key.dbid = MyDatabaseId;
+ LWLockAcquire(RepackedRelsLock, LW_EXCLUSIVE);
+ entry = hash_search(RepackedRelsHash, &key, HASH_REMOVE, NULL);
+ LWLockRelease(RepackedRelsLock);
+
+ /*
+ * Make others refresh their information whether they should still
+ * treat the table as catalog from the perspective of writing WAL.
+ *
+ * XXX Unlike entering the entry into the hashtable, we do not bother
+ * with locking and unlocking the table here:
+ *
+ * 1) On normal completion (and sometimes even on ERROR), the caller
+ * is already holding AccessExclusiveLock on the table, so there
+ * should be no relcache reference unaware of this change.
+ *
+ * 2) In the other cases, the worst scenario is that the other
+ * backends will write unnecessary information to WAL until they close
+ * the relation.
+ *
+ * Should we use ShareLock mode to fix 2) at least for the non-FATAL
+ * errors? (Our before_shmem_exit callback is in charge of FATAL, and
+ * that probably should not try to acquire any lock.)
+ */
+ CacheInvalidateRelcacheImmediate(repacked_rel);
+
+ /*
+ * By clearing this variable we also disable
+ * cluster_before_shmem_exit_callback().
+ */
+ repacked_rel = InvalidOid;
+ }
+
/*
* Restore normal function of (future) logical decoding for this backend.
*/
repacked_rel_locator.relNumber = InvalidOid;
repacked_rel_toast_locator.relNumber = InvalidOid;
+
+ /*
+ * On normal completion (!error), we should not really fail to remove the
+ * entry. But if it wasn't there for any reason, raise ERROR to make sure
+ * the transaction is aborted: if other transactions, while changing the
+ * contents of the relation, didn't know that REPACK CONCURRENTLY was in
+ * progress, they could have missed to WAL enough information, and thus we
+ * could have produced an inconsistent table contents.
+ *
+ * On the other hand, if we are already handling an error, there's no
+ * reason to worry about inconsistent contents of the new storage because
+ * the transaction is going to be rolled back anyway. Furthermore, by
+ * raising ERROR here we'd shadow the original error.
+ */
+ if (!error)
+ {
+ char *relname;
+
+ if (OidIsValid(relid) && entry == NULL)
+ {
+ relname = get_rel_name(relid);
+ if (!relname)
+ ereport(ERROR,
+ (errmsg("cache lookup failed for relation %u",
+ relid)));
+
+ ereport(ERROR,
+ (errmsg("relation \"%s\" not found among repacked relations",
+ relname)));
+ }
+ }
+}
+
+/*
+ * A wrapper to call end_concurrent_repack() as a before_shmem_exit callback.
+ */
+static void
+cluster_before_shmem_exit_callback(int code, Datum arg)
+{
+ if (OidIsValid(repacked_rel))
+ end_concurrent_repack(true);
+}
+
+/*
+ * Check if relation is currently being processed by REPACK CONCURRENTLY.
+ */
+bool
+is_concurrent_repack_in_progress(Oid relid)
+{
+ RepackedRel key,
+ *entry;
+
+ memset(&key, 0, sizeof(key));
+ key.relid = relid;
+ key.dbid = MyDatabaseId;
+
+ LWLockAcquire(RepackedRelsLock, LW_SHARED);
+ entry = (RepackedRel *)
+ hash_search(RepackedRelsHash, &key, HASH_FIND, NULL);
+ LWLockRelease(RepackedRelsLock);
+
+ return entry != NULL;
}
/*
@@ -2489,6 +2762,9 @@ setup_logical_decoding(Oid relid, const char *slotname, TupleDesc tupdesc)
dstate->relid = relid;
dstate->tstore = tuplestore_begin_heap(false, false,
maintenance_work_mem);
+#ifdef USE_ASSERT_CHECKING
+ dstate->last_change_xid = InvalidTransactionId;
+#endif
dstate->tupdesc = tupdesc;
@@ -2636,6 +2912,7 @@ apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
char *change_raw,
*src;
ConcurrentChange change;
+ Snapshot snapshot;
bool isnull[1];
Datum values[1];
@@ -2704,8 +2981,30 @@ apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
/*
* Find the tuple to be updated or deleted.
+ *
+ * As the table being REPACKed concurrently is treated like a
+ * catalog, new CID is WAL-logged and decoded. And since we use
+ * the same XID that the original DMLs did, the snapshot used for
+ * the logical decoding (by now converted to a non-historic MVCC
+ * snapshot) should see the tuples inserted previously into the
+ * new heap and/or updated there.
*/
- tup_exist = find_target_tuple(rel, key, nkeys, tup_key,
+ snapshot = change.snapshot;
+
+ /*
+ * Set what should be considered current transaction (and
+ * subtransactions) during visibility check.
+ *
+ * Note that this snapshot was created from a historic snapshot
+ * using SnapBuildMVCCFromHistoric(), which does not touch
+ * 'subxip'. Thus, unlike in a regular MVCC snapshot, the array
+ * only contains the transactions whose data changes we are
+ * applying, and its subtransactions. That's exactly what we need
+ * to check if particular xact is a "current transaction:".
+ */
+ SetRepackCurrentXids(snapshot->subxip, snapshot->subxcnt);
+
+ tup_exist = find_target_tuple(rel, key, nkeys, tup_key, snapshot,
iistate, ident_slot, &ind_scan);
if (tup_exist == NULL)
elog(ERROR, "Failed to find target tuple");
@@ -2716,6 +3015,8 @@ apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
else
apply_concurrent_delete(rel, tup_exist, &change);
+ ResetRepackCurrentXids();
+
if (tup_old != NULL)
{
pfree(tup_old);
@@ -2728,14 +3029,14 @@ apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
else
elog(ERROR, "Unrecognized kind of change: %d", change.kind);
- /*
- * If a change was applied now, increment CID for next writes and
- * update the snapshot so it sees the changes we've applied so far.
- */
- if (change.kind != CHANGE_UPDATE_OLD)
+ /* Free the snapshot if this is the last change that needed it. */
+ Assert(change.snapshot->active_count > 0);
+ change.snapshot->active_count--;
+ if (change.snapshot->active_count == 0)
{
- CommandCounterIncrement();
- UpdateActiveSnapshotCommandId();
+ if (change.snapshot == dstate->snapshot)
+ dstate->snapshot = NULL;
+ FreeSnapshot(change.snapshot);
}
/* TTSOpsMinimalTuple has .get_heap_tuple==NULL. */
@@ -2755,16 +3056,35 @@ static void
apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup,
IndexInsertState *iistate, TupleTableSlot *index_slot)
{
+ Snapshot snapshot = change->snapshot;
List *recheck;
+ /*
+ * For INSERT, the visibility information is not important, but we use the
+ * snapshot to get CID. Index functions might need the whole snapshot
+ * anyway.
+ */
+ SetRepackCurrentXids(snapshot->subxip, snapshot->subxcnt);
+
+ /*
+ * Write the tuple into the new heap.
+ *
+ * The snapshot is the one we used to decode the insert (though converted
+ * to "non-historic" MVCC snapshot), i.e. the snapshot's curcid is the
+ * tuple CID incremented by one (due to the "new CID" WAL record that got
+ * written along with the INSERT record). Thus if we want to use the
+ * original CID, we need to subtract 1 from curcid.
+ */
+ Assert(snapshot->curcid != InvalidCommandId &&
+ snapshot->curcid > FirstCommandId);
/*
* Like simple_heap_insert(), but make sure that the INSERT is not
* logically decoded - see reform_and_rewrite_tuple() for more
* information.
*/
- heap_insert(rel, tup, GetCurrentCommandId(true), HEAP_INSERT_NO_LOGICAL,
- NULL);
+ heap_insert(rel, tup, change->xid, snapshot->curcid - 1,
+ HEAP_INSERT_NO_LOGICAL, NULL);
/*
* Update indexes.
@@ -2772,6 +3092,7 @@ apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup,
* In case functions in the index need the active snapshot and caller
* hasn't set one.
*/
+ PushActiveSnapshot(snapshot);
ExecStoreHeapTuple(tup, index_slot, false);
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
@@ -2782,6 +3103,8 @@ apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup,
NIL, /* arbiterIndexes */
false /* onlySummarizing */
);
+ PopActiveSnapshot();
+ ResetRepackCurrentXids();
/*
* If recheck is required, it must have been preformed on the source
@@ -2803,6 +3126,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
TU_UpdateIndexes update_indexes;
TM_Result res;
List *recheck;
+ Snapshot snapshot = change->snapshot;
/*
* Write the new tuple into the new heap. ('tup' gets the TID assigned
@@ -2810,13 +3134,19 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
*
* Do it like in simple_heap_update(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Regarding CID, see the comment in apply_concurrent_insert().
*/
+ Assert(snapshot->curcid != InvalidCommandId &&
+ snapshot->curcid > FirstCommandId);
+
res = heap_update(rel, &tup_target->t_self, tup,
- GetCurrentCommandId(true),
+ change->xid, snapshot->curcid - 1,
InvalidSnapshot,
false, /* no wait - only we are doing changes */
&tmfd, &lockmode, &update_indexes,
- false /* wal_logical */ );
+ /* wal_logical */
+ false);
if (res != TM_Ok)
ereport(ERROR, (errmsg("failed to apply concurrent UPDATE")));
@@ -2824,6 +3154,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
if (update_indexes != TU_None)
{
+ PushActiveSnapshot(snapshot);
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
iistate->estate,
@@ -2833,6 +3164,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
NIL, /* arbiterIndexes */
/* onlySummarizing */
update_indexes == TU_Summarizing);
+ PopActiveSnapshot();
list_free(recheck);
}
@@ -2845,6 +3177,12 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target,
{
TM_Result res;
TM_FailureData tmfd;
+ Snapshot snapshot = change->snapshot;
+
+
+ /* Regarding CID, see the comment in apply_concurrent_insert(). */
+ Assert(snapshot->curcid != InvalidCommandId &&
+ snapshot->curcid > FirstCommandId);
/*
* Delete tuple from the new heap.
@@ -2852,11 +3190,11 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target,
* Do it like in simple_heap_delete(), except for 'wal_logical' (and
* except for 'wait').
*/
- res = heap_delete(rel, &tup_target->t_self, GetCurrentCommandId(true),
- InvalidSnapshot, false,
- &tmfd,
- false, /* no wait - only we are doing changes */
- false /* wal_logical */ );
+ res = heap_delete(rel, &tup_target->t_self, change->xid,
+ snapshot->curcid - 1, InvalidSnapshot, false,
+ &tmfd, false,
+ /* wal_logical */
+ false);
if (res != TM_Ok)
ereport(ERROR, (errmsg("failed to apply concurrent DELETE")));
@@ -2877,7 +3215,7 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target,
*/
static HeapTuple
find_target_tuple(Relation rel, ScanKey key, int nkeys, HeapTuple tup_key,
- IndexInsertState *iistate,
+ Snapshot snapshot, IndexInsertState *iistate,
TupleTableSlot *ident_slot, IndexScanDesc *scan_p)
{
IndexScanDesc scan;
@@ -2886,7 +3224,7 @@ find_target_tuple(Relation rel, ScanKey key, int nkeys, HeapTuple tup_key,
HeapTuple result = NULL;
/* XXX no instrumentation for now */
- scan = index_beginscan(rel, iistate->ident_index, GetActiveSnapshot(),
+ scan = index_beginscan(rel, iistate->ident_index, snapshot,
NULL, nkeys, 0);
*scan_p = scan;
index_rescan(scan, key, nkeys, NULL, 0);
@@ -2958,6 +3296,8 @@ process_concurrent_changes(LogicalDecodingContext *ctx, XLogRecPtr end_of_wal,
}
PG_FINALLY();
{
+ ResetRepackCurrentXids();
+
if (rel_src)
rel_dst->rd_toastoid = InvalidOid;
}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 5dc4ae58ffe..9fefcffd8b3 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -475,9 +475,14 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
/*
* If the change is not intended for logical decoding, do not even
- * establish transaction for it - REPACK CONCURRENTLY is the typical use
- * case.
- *
+ * establish transaction for it. This is particularly important if the
+ * record was generated by REPACK CONCURRENTLY because this command uses
+ * the original XID when doing changes in the new storage. The decoding
+ * system probably does not expect to see the same transaction multiple
+ * times.
+ */
+
+ /*
* First, check if REPACK CONCURRENTLY is being performed by this backend.
* If so, only decode data changes of the table that it is processing, and
* the changes of its TOAST relation.
@@ -504,11 +509,11 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
* Second, skip records which do not contain sufficient information for
* the decoding.
*
- * The problem we solve here is that REPACK CONCURRENTLY generates WAL
- * when doing changes in the new table. Those changes should not be useful
- * for any other user (such as logical replication subscription) because
- * the new table will eventually be dropped (after REPACK CONCURRENTLY has
- * assigned its file to the "old table").
+ * One particular problem we solve here is that REPACK CONCURRENTLY
+ * generates WAL when doing changes in the new table. Those changes should
+ * not be decoded because reorderbuffer.c considers their XID already
+ * committed. (REPACK CONCURRENTLY deliberately generates WAL records in
+ * such a way that they are skipped here.)
*/
switch (info)
{
@@ -995,13 +1000,6 @@ DecodeInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
xlrec = (xl_heap_insert *) XLogRecGetData(r);
- /*
- * Ignore insert records without new tuples (this does happen when
- * raw_heap_insert marks the TOAST record as HEAP_INSERT_NO_LOGICAL).
- */
- if (!(xlrec->flags & XLH_INSERT_CONTAINS_NEW_TUPLE))
- return;
-
/* only interested in our database */
XLogRecGetBlockTag(r, 0, &target_locator, NULL, NULL);
if (target_locator.dbOid != ctx->slot->data.database)
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 8e5116a9cab..72a38074a7b 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -155,7 +155,7 @@ static bool ExportInProgress = false;
static void SnapBuildPurgeOlderTxn(SnapBuild *builder);
/* snapshot building/manipulation/distribution functions */
-static Snapshot SnapBuildBuildSnapshot(SnapBuild *builder);
+static Snapshot SnapBuildBuildSnapshot(SnapBuild *builder, XLogRecPtr lsn);
static void SnapBuildFreeSnapshot(Snapshot snap);
@@ -352,12 +352,17 @@ SnapBuildSnapDecRefcount(Snapshot snap)
* Build a new snapshot, based on currently committed catalog-modifying
* transactions.
*
+ * 'lsn' is the location of the commit record (of a catalog-changing
+ * transaction) that triggered creation of the snapshot. Pass
+ * InvalidXLogRecPtr for the transaction base snapshot or if it the user of
+ * the snapshot should not need the LSN.
+ *
* In-progress transactions with catalog access are *not* allowed to modify
* these snapshots; they have to copy them and fill in appropriate ->curcid
* and ->subxip/subxcnt values.
*/
static Snapshot
-SnapBuildBuildSnapshot(SnapBuild *builder)
+SnapBuildBuildSnapshot(SnapBuild *builder, XLogRecPtr lsn)
{
Snapshot snapshot;
Size ssize;
@@ -425,6 +430,7 @@ SnapBuildBuildSnapshot(SnapBuild *builder)
snapshot->active_count = 0;
snapshot->regd_count = 0;
snapshot->snapXactCompletionCount = 0;
+ snapshot->lsn = lsn;
return snapshot;
}
@@ -461,7 +467,7 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
if (TransactionIdIsValid(MyProc->xmin))
elog(ERROR, "cannot build an initial slot snapshot when MyProc->xmin already is valid");
- snap = SnapBuildBuildSnapshot(builder);
+ snap = SnapBuildBuildSnapshot(builder, InvalidXLogRecPtr);
/*
* We know that snap->xmin is alive, enforced by the logical xmin
@@ -502,7 +508,7 @@ SnapBuildInitialSnapshotForRepack(SnapBuild *builder)
Assert(builder->state == SNAPBUILD_CONSISTENT);
- snap = SnapBuildBuildSnapshot(builder);
+ snap = SnapBuildBuildSnapshot(builder, InvalidXLogRecPtr);
return SnapBuildMVCCFromHistoric(snap, false);
}
@@ -636,7 +642,7 @@ SnapBuildGetOrBuildSnapshot(SnapBuild *builder)
/* only build a new snapshot if we don't have a prebuilt one */
if (builder->snapshot == NULL)
{
- builder->snapshot = SnapBuildBuildSnapshot(builder);
+ builder->snapshot = SnapBuildBuildSnapshot(builder, InvalidXLogRecPtr);
/* increase refcount for the snapshot builder */
SnapBuildSnapIncRefcount(builder->snapshot);
}
@@ -716,7 +722,7 @@ SnapBuildProcessChange(SnapBuild *builder, TransactionId xid, XLogRecPtr lsn)
/* only build a new snapshot if we don't have a prebuilt one */
if (builder->snapshot == NULL)
{
- builder->snapshot = SnapBuildBuildSnapshot(builder);
+ builder->snapshot = SnapBuildBuildSnapshot(builder, lsn);
/* increase refcount for the snapshot builder */
SnapBuildSnapIncRefcount(builder->snapshot);
}
@@ -1130,7 +1136,7 @@ SnapBuildCommitTxn(SnapBuild *builder, XLogRecPtr lsn, TransactionId xid,
if (builder->snapshot)
SnapBuildSnapDecRefcount(builder->snapshot);
- builder->snapshot = SnapBuildBuildSnapshot(builder);
+ builder->snapshot = SnapBuildBuildSnapshot(builder, lsn);
/* we might need to execute invalidations, add snapshot */
if (!ReorderBufferXidHasBaseSnapshot(builder->reorder, xid))
@@ -1958,7 +1964,7 @@ SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn)
{
SnapBuildSnapDecRefcount(builder->snapshot);
}
- builder->snapshot = SnapBuildBuildSnapshot(builder);
+ builder->snapshot = SnapBuildBuildSnapshot(builder, InvalidXLogRecPtr);
SnapBuildSnapIncRefcount(builder->snapshot);
ReorderBufferSetRestartPoint(builder->reorder, lsn);
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
index 687fbbc59bb..28bd16f9cc7 100644
--- a/src/backend/replication/pgoutput_repack/pgoutput_repack.c
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -32,7 +32,8 @@ static void plugin_truncate(struct LogicalDecodingContext *ctx,
Relation relations[],
ReorderBufferChange *change);
static void store_change(LogicalDecodingContext *ctx,
- ConcurrentChangeKind kind, HeapTuple tuple);
+ ConcurrentChangeKind kind, HeapTuple tuple,
+ TransactionId xid);
void
_PG_output_plugin_init(OutputPluginCallbacks *cb)
@@ -100,6 +101,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Relation relation, ReorderBufferChange *change)
{
RepackDecodingState *dstate;
+ Snapshot snapshot;
dstate = (RepackDecodingState *) ctx->output_writer_private;
@@ -107,6 +109,48 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relation->rd_id != dstate->relid)
return;
+ /*
+ * Catalog snapshot is fine because the table we are processing is
+ * temporarily considered a user catalog table.
+ */
+ snapshot = GetCatalogSnapshot(InvalidOid);
+ Assert(snapshot->snapshot_type == SNAPSHOT_HISTORIC_MVCC);
+ Assert(!snapshot->suboverflowed);
+
+ /*
+ * This should not happen, but if we don't have enough information to
+ * apply a new snapshot, the consequences would be bad. Thus prefer ERROR
+ * to Assert().
+ */
+ if (XLogRecPtrIsInvalid(snapshot->lsn))
+ ereport(ERROR, (errmsg("snapshot has invalid LSN")));
+
+ /*
+ * reorderbuffer.c changes the catalog snapshot as soon as it sees a new
+ * CID or a commit record of a catalog-changing transaction.
+ */
+ if (dstate->snapshot == NULL || snapshot->lsn != dstate->snapshot_lsn ||
+ snapshot->curcid != dstate->snapshot->curcid)
+ {
+ /* CID should not go backwards. */
+ Assert(dstate->snapshot == NULL ||
+ snapshot->curcid >= dstate->snapshot->curcid ||
+ change->txn->xid != dstate->last_change_xid);
+
+ /*
+ * XXX Is it a problem that the copy is created in
+ * TopTransactionContext?
+ *
+ * XXX Wouldn't it be o.k. for SnapBuildMVCCFromHistoric() to set xcnt
+ * to 0 instead of converting xip in this case? The point is that
+ * transactions which are still in progress from the perspective of
+ * reorderbuffer.c could not be replayed yet, so we do not need to
+ * examine their XIDs.
+ */
+ dstate->snapshot = SnapBuildMVCCFromHistoric(snapshot, false);
+ dstate->snapshot_lsn = snapshot->lsn;
+ }
+
/* Decode entry depending on its type */
switch (change->action)
{
@@ -124,7 +168,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (newtuple == NULL)
elog(ERROR, "Incomplete insert info.");
- store_change(ctx, CHANGE_INSERT, newtuple);
+ store_change(ctx, CHANGE_INSERT, newtuple, change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -141,9 +185,11 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
elog(ERROR, "Incomplete update info.");
if (oldtuple != NULL)
- store_change(ctx, CHANGE_UPDATE_OLD, oldtuple);
+ store_change(ctx, CHANGE_UPDATE_OLD, oldtuple,
+ change->txn->xid);
- store_change(ctx, CHANGE_UPDATE_NEW, newtuple);
+ store_change(ctx, CHANGE_UPDATE_NEW, newtuple,
+ change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_DELETE:
@@ -156,7 +202,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (oldtuple == NULL)
elog(ERROR, "Incomplete delete info.");
- store_change(ctx, CHANGE_DELETE, oldtuple);
+ store_change(ctx, CHANGE_DELETE, oldtuple, change->txn->xid);
}
break;
default:
@@ -190,13 +236,13 @@ plugin_truncate(struct LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (i == nrelations)
return;
- store_change(ctx, CHANGE_TRUNCATE, NULL);
+ store_change(ctx, CHANGE_TRUNCATE, NULL, InvalidTransactionId);
}
/* Store concurrent data change. */
static void
store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
- HeapTuple tuple)
+ HeapTuple tuple, TransactionId xid)
{
RepackDecodingState *dstate;
char *change_raw;
@@ -266,6 +312,11 @@ store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
dst = dst_start + SizeOfConcurrentChange;
memcpy(dst, tuple->t_data, tuple->t_len);
+ /* Initialize the other fields. */
+ change.xid = xid;
+ change.snapshot = dstate->snapshot;
+ dstate->snapshot->active_count++;
+
/* The data has been copied. */
if (flattened)
pfree(tuple);
@@ -279,6 +330,9 @@ store:
isnull[0] = false;
tuplestore_putvalues(dstate->tstore, dstate->tupdesc_change,
values, isnull);
+#ifdef USE_ASSERT_CHECKING
+ dstate->last_change_xid = xid;
+#endif
/* Accounting. */
dstate->nchanges++;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index e9ddf39500c..e24e1795aa9 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -151,6 +151,7 @@ CalculateShmemSize(int *num_semaphores)
size = add_size(size, InjectionPointShmemSize());
size = add_size(size, SlotSyncShmemSize());
size = add_size(size, AioShmemSize());
+ size = add_size(size, RepackShmemSize());
/* include additional requested shmem from preload libraries */
size = add_size(size, total_addin_request);
@@ -344,6 +345,7 @@ CreateOrAttachShmemStructs(void)
WaitEventCustomShmemInit();
InjectionPointShmemInit();
AioShmemInit();
+ RepackShmemInit();
}
/*
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index 0be307d2ca0..f546c2e12ca 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -352,6 +352,7 @@ DSMRegistry "Waiting to read or update the dynamic shared memory registry."
InjectionPoint "Waiting to read or update information related to injection points."
SerialControl "Waiting to read or update shared <filename>pg_serial</filename> state."
AioWorkerSubmissionQueue "Waiting to access AIO worker submission queue."
+RepackedRels "Waiting to access to hash table with list of repacked relations."
#
# END OF PREDEFINED LWLOCKS (DO NOT CHANGE THIS LINE)
diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c
index 02505c88b8e..ecaa2283c2a 100644
--- a/src/backend/utils/cache/inval.c
+++ b/src/backend/utils/cache/inval.c
@@ -1643,6 +1643,27 @@ CacheInvalidateRelcache(Relation relation)
databaseId, relationId);
}
+/*
+ * CacheInvalidateRelcacheImmediate
+ * Send invalidation message for the specified relation's relcache entry.
+ *
+ * Currently this is used in REPACK CONCURRENTLY, to make sure that other
+ * backends are aware that the command is being executed for the relation.
+ */
+void
+CacheInvalidateRelcacheImmediate(Oid relid)
+{
+ SharedInvalidationMessage msg;
+
+ msg.rc.id = SHAREDINVALRELCACHE_ID;
+ msg.rc.dbId = MyDatabaseId;
+ msg.rc.relId = relid;
+ /* check AddCatcacheInvalidationMessage() for an explanation */
+ VALGRIND_MAKE_MEM_DEFINED(&msg, sizeof(msg));
+
+ SendSharedInvalidMessages(&msg, 1);
+}
+
/*
* CacheInvalidateRelcacheAll
* Register invalidation of the whole relcache at the end of command.
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d27a4c30548..ea565b5b053 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -1279,6 +1279,10 @@ retry:
/* make sure relation is marked as having no open file yet */
relation->rd_smgr = NULL;
+ /* Is REPACK CONCURRENTLY in progress? */
+ relation->rd_repack_concurrent =
+ is_concurrent_repack_in_progress(targetRelId);
+
/*
* now we can free the memory allocated for pg_class_tuple
*/
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index b82dd17a966..981425f23b6 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -316,22 +316,24 @@ extern BulkInsertState GetBulkInsertState(void);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
-extern void heap_insert(Relation relation, HeapTuple tup, CommandId cid,
- int options, BulkInsertState bistate);
+extern void heap_insert(Relation relation, HeapTuple tup, TransactionId xid,
+ CommandId cid, int options, BulkInsertState bistate);
extern void heap_multi_insert(Relation relation, struct TupleTableSlot **slots,
int ntuples, CommandId cid, int options,
BulkInsertState bistate);
extern TM_Result heap_delete(Relation relation, ItemPointer tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid,
+ Snapshot crosscheck, bool wait,
struct TM_FailureData *tmfd, bool changingPart,
bool wal_logical);
extern void heap_finish_speculative(Relation relation, ItemPointer tid);
extern void heap_abort_speculative(Relation relation, ItemPointer tid);
extern TM_Result heap_update(Relation relation, ItemPointer otid,
- HeapTuple newtup,
+ HeapTuple newtup, TransactionId xid,
CommandId cid, Snapshot crosscheck, bool wait,
struct TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes, bool wal_logical);
+ TU_UpdateIndexes *update_indexes,
+ bool wal_logical);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index b2bc10ee041..fbb66d559b6 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -482,6 +482,8 @@ extern Size EstimateTransactionStateSpace(void);
extern void SerializeTransactionState(Size maxsize, char *start_address);
extern void StartParallelWorkerTransaction(char *tstatespace);
extern void EndParallelWorkerTransaction(void);
+extern void SetRepackCurrentXids(TransactionId *xip, int xcnt);
+extern void ResetRepackCurrentXids(void);
extern bool IsTransactionBlock(void);
extern bool IsTransactionOrTransactionBlock(void);
extern char TransactionBlockStatusCode(void);
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
index 532ffa7208d..00ae6fc18e8 100644
--- a/src/include/commands/cluster.h
+++ b/src/include/commands/cluster.h
@@ -59,6 +59,14 @@ typedef struct ConcurrentChange
/* See the enum above. */
ConcurrentChangeKind kind;
+ /* Transaction that changes the data. */
+ TransactionId xid;
+
+ /*
+ * Historic catalog snapshot that was used to decode this change.
+ */
+ Snapshot snapshot;
+
/*
* The actual tuple.
*
@@ -90,6 +98,8 @@ typedef struct RepackDecodingState
* tuplestore does this transparently.
*/
Tuplestorestate *tstore;
+ /* XID of the last change added to tstore. */
+ TransactionId last_change_xid PG_USED_FOR_ASSERTS_ONLY;
/* The current number of changes in tstore. */
double nchanges;
@@ -110,6 +120,14 @@ typedef struct RepackDecodingState
/* Slot to retrieve data from tstore. */
TupleTableSlot *tsslot;
+ /*
+ * Historic catalog snapshot that was used to decode the most recent
+ * change.
+ */
+ Snapshot snapshot;
+ /* LSN of the record */
+ XLogRecPtr snapshot_lsn;
+
ResourceOwner resowner;
} RepackDecodingState;
@@ -139,4 +157,8 @@ extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
MultiXactId cutoffMulti,
char newrelpersistence);
+extern Size RepackShmemSize(void);
+extern void RepackShmemInit(void);
+extern bool is_concurrent_repack_in_progress(Oid relid);
+
#endif /* CLUSTER_H */
diff --git a/src/include/storage/lwlocklist.h b/src/include/storage/lwlocklist.h
index 208d2e3a8ed..869d9da7337 100644
--- a/src/include/storage/lwlocklist.h
+++ b/src/include/storage/lwlocklist.h
@@ -85,6 +85,7 @@ PG_LWLOCK(50, DSMRegistry)
PG_LWLOCK(51, InjectionPoint)
PG_LWLOCK(52, SerialControl)
PG_LWLOCK(53, AioWorkerSubmissionQueue)
+PG_LWLOCK(54, RepackedRels)
/*
* There also exist several built-in LWLock tranches. As with the predefined
diff --git a/src/include/utils/inval.h b/src/include/utils/inval.h
index 9b871caef62..ae9dee394dc 100644
--- a/src/include/utils/inval.h
+++ b/src/include/utils/inval.h
@@ -50,6 +50,8 @@ extern void CacheInvalidateCatalog(Oid catalogId);
extern void CacheInvalidateRelcache(Relation relation);
+extern void CacheInvalidateRelcacheImmediate(Oid relid);
+
extern void CacheInvalidateRelcacheAll(void);
extern void CacheInvalidateRelcacheByTuple(HeapTuple classTuple);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index b552359915f..66de3bc0c29 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -253,6 +253,9 @@ typedef struct RelationData
bool pgstat_enabled; /* should relation stats be counted */
/* use "struct" here to avoid needing to include pgstat.h: */
struct PgStat_TableStatus *pgstat_info; /* statistics collection area */
+
+ /* Is REPACK CONCURRENTLY being performed on this relation? */
+ bool rd_repack_concurrent;
} RelationData;
@@ -695,7 +698,9 @@ RelationCloseSmgr(Relation relation)
#define RelationIsAccessibleInLogicalDecoding(relation) \
(XLogLogicalInfoActive() && \
RelationNeedsWAL(relation) && \
- (IsCatalogRelation(relation) || RelationIsUsedAsCatalogTable(relation)))
+ (IsCatalogRelation(relation) || \
+ RelationIsUsedAsCatalogTable(relation) || \
+ (relation)->rd_repack_concurrent))
/*
* RelationIsLogicallyLogged
diff --git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h
index 0e546ec1497..014f27db7d7 100644
--- a/src/include/utils/snapshot.h
+++ b/src/include/utils/snapshot.h
@@ -13,6 +13,7 @@
#ifndef SNAPSHOT_H
#define SNAPSHOT_H
+#include "access/xlogdefs.h"
#include "lib/pairingheap.h"
@@ -201,6 +202,8 @@ typedef struct SnapshotData
uint32 regd_count; /* refcount on RegisteredSnapshots */
pairingheap_node ph_node; /* link in the RegisteredSnapshots heap */
+ XLogRecPtr lsn; /* position in the WAL stream when taken */
+
/*
* The transaction completion count at the time GetSnapshotData() built
* this snapshot. Allows to avoid re-computing static snapshots when no
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index 75850334986..3711a7c92b9 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -86,9 +86,6 @@ step change_new
# When applying concurrent data changes, we should see the effects of an
# in-progress subtransaction.
#
-# XXX Not sure this test is useful now - it was designed for the patch that
-# preserves tuple visibility and which therefore modifies
-# TransactionIdIsCurrentTransactionId().
step change_subxact1
{
BEGIN;
@@ -103,7 +100,6 @@ step change_subxact1
# When applying concurrent data changes, we should not see the effects of a
# rolled back subtransaction.
#
-# XXX Is this test useful? See above.
step change_subxact2
{
BEGIN;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 480ce894978..e42bb5209c0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2540,6 +2540,7 @@ ReorderBufferTupleCidKey
ReorderBufferUpdateProgressTxnCB
ReorderTuple
RepOriginId
+RepackedRel
RepackCommand
RepackDecodingState
RepackStmt
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-21 18:07 ` Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-21 18:07 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Also, I think I found an issue (or lost something during rebase): we
> must preserve xmin,cmin during initial copy
> to make sure that data is going to be visible by snapshots of
> concurrent changes later:
>
> static void
> reform_and_rewrite_tuple(......)
> .....
> /*It is also crucial to stamp the new record with the exact same
> xid and cid,
> * because the tuple must be visible to the snapshot of the
> applied concurrent
> * change later.
> */
> CommandId cid = HeapTupleHeaderGetRawCommandId(tuple->t_data);
> TransactionId xid = HeapTupleHeaderGetXmin(tuple->t_data);
>
> heap_insert(NewHeap, copiedTuple, xid, cid, HEAP_INSERT_NO_LOGICAL, NULL);
When posting version 12 of the patch [1] I raised a concern that the the MVCC
safety is too expensive when it comes to logical decoding. Therefore, I
abandoned the concept for now, and v13 [2] uses plain heap_insert(). Once we
implement the MVCC safety, we simply rewrite the tuple like v12 did - that's
the simplest way to preserve fields like xmin, cmin, ...
[1] https://www.postgresql.org/message-id/178741.1743514291%40localhost
[2] https://www.postgresql.org/message-id/97795.1744363522%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-24 16:52 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-24 16:52 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Hello, Antonin!
> When posting version 12 of the patch [1] I raised a concern that the the MVCC
> safety is too expensive when it comes to logical decoding. Therefore, I
> abandoned the concept for now, and v13 [2] uses plain heap_insert(). Once we
> implement the MVCC safety, we simply rewrite the tuple like v12 did - that's
> the simplest way to preserve fields like xmin, cmin, ...
Thanks for the explanation.
I was looking into catalog-related logical decoding features, and it
seems like they are clearly overkill for the repack case.
We don't need CID tracking or even a snapshot for each commit if we’re
okay with passing xmin/xmax as arguments.
What do you think about the following approach for replaying:
* use the extracted XID as the value for xmin/xmax.
* use SnapshotSelf to find the tuple for update/delete operations.
SnapshotSelf seems like a good fit here:
* it sees the last "existing" version.
* any XID set as xmin/xmax in the repacked version is already
committed - so each update/insert is effectively "committed" once
written.
* it works with multiple updates of the same tuple within a single
transaction - SnapshotSelf sees the last version.
* all updates are ordered and replayed sequentially - so the last
version is always the one we want.
If I'm not missing anything, this looks like something worth including
in the patch set.
If so, I can try implementing a test version.
Best regards,
Mikhail
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-25 13:09 ` Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-25 13:09 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> I was looking into catalog-related logical decoding features, and it
> seems like they are clearly overkill for the repack case.
> We don't need CID tracking or even a snapshot for each commit if we’re
> okay with passing xmin/xmax as arguments.
I assume you are concerned with the patch part 0005 of the v12 patch
("Preserve visibility information of the concurrent data changes."), aren't
you?
> What do you think about the following approach for replaying:
> * use the extracted XID as the value for xmin/xmax.
> * use SnapshotSelf to find the tuple for update/delete operations.
>
> SnapshotSelf seems like a good fit here:
> * it sees the last "existing" version.
> * any XID set as xmin/xmax in the repacked version is already
> committed - so each update/insert is effectively "committed" once
> written.
> * it works with multiple updates of the same tuple within a single
> transaction - SnapshotSelf sees the last version.
> * all updates are ordered and replayed sequentially - so the last
> version is always the one we want.
>
> If I'm not missing anything, this looks like something worth including
> in the patch set.
> If so, I can try implementing a test version.
Not sure I understand in all details, but I don't think SnapshotSelf is the
correct snapshot. Note that HeapTupleSatisfiesSelf() does not use its
'snapshot' argument at all. Instead, it considers the set of running
transactions as it is at the time the function is called.
One particular problem I imagine is replaying an UPDATE to a row that some
later transaction will eventually delete, but the transaction that ran the
UPDATE obviously had to see it. When looking for the old version during the
replay, HeapTupleSatisfiesMVCC() will find the old version as long as we pass
the correct snapshot to it.
However, at the time we're replaying the UPDATE in the new table, the tuple
may have been already deleted from the old table, and the deleting transaction
may already have committed. In such a case, HeapTupleSatisfiesSelf() will
conclude the old version invisible and the we'll fail to replay the UPDATE.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-25 14:15 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:36 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-25 14:15 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Hi, Antonin!
> I assume you are concerned with the patch part 0005 of the v12 patch
> ("Preserve visibility information of the concurrent data changes."), aren't
> you?
Yes, of course. I got an idea while trying to find a way to optimize it.
> Not sure I understand in all details, but I don't think SnapshotSelf is the
> correct snapshot. Note that HeapTupleSatisfiesSelf() does not use its
> 'snapshot' argument at all. Instead, it considers the set of running
> transactions as it is at the time the function is called.
Yes, and it is almost the same behavior when a typical MVCC snapshot
encounters a tuple created by its own transaction.
So, how it works in the non MVCC-safe case (current patch behaviour):
1) we have a whole initial table snapshot with all the xmin = repack XID
2) appling transaction sees ALL the self-alive (no xmax) tuples in it
because all tuples created\deleted by transaction itself
3) each update/delete during the replay selects the last existing
tuple version, updates it xmax and inserts a new one
4) so, there is no any real MVCC involved - just find the latest
version and create a new version
5) and it works correctly because all ordering issues were resolved by
locking mechanisms on the original table or by reordering buffer
How it maps to MVCC-safe case (SnapshotSelf):
1) we have a whole initial table snapshot with all xmin copied from
the original table. All such xmin are committed.
2) appling transaction sees ALL the self-alive (no xmax) tuple in it
because its xmin\xmax is committed and SnapshotSelf is happy with it
3) each update/delete during the replay selects the last existing
tuple version, updates it xmax=original xid and inserts a new one
keeping with xmin=orignal xid
4) --//--
5) --//--
> However, at the time we're replaying the UPDATE in the new table, the tuple
> may have been already deleted from the old table, and the deleting transaction
> may already have committed. In such a case, HeapTupleSatisfiesSelf() will
> conclude the old version invisible and the we'll fail to replay the UPDATE.
No, it will see it - because its xmax will be empty in the repacked
version of the table.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-25 15:42 ` Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-25 15:42 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > Not sure I understand in all details, but I don't think SnapshotSelf is the
> > correct snapshot. Note that HeapTupleSatisfiesSelf() does not use its
> > 'snapshot' argument at all. Instead, it considers the set of running
> > transactions as it is at the time the function is called.
>
> Yes, and it is almost the same behavior when a typical MVCC snapshot
> encounters a tuple created by its own transaction.
>
> So, how it works in the non MVCC-safe case (current patch behaviour):
>
> 1) we have a whole initial table snapshot with all the xmin = repack XID
> 2) appling transaction sees ALL the self-alive (no xmax) tuples in it
> because all tuples created\deleted by transaction itself
> 3) each update/delete during the replay selects the last existing
> tuple version, updates it xmax and inserts a new one
> 4) so, there is no any real MVCC involved - just find the latest
> version and create a new version
> 5) and it works correctly because all ordering issues were resolved by
> locking mechanisms on the original table or by reordering buffer
ok
> How it maps to MVCC-safe case (SnapshotSelf):
>
> 1) we have a whole initial table snapshot with all xmin copied from
> the original table. All such xmin are committed.
> 2) appling transaction sees ALL the self-alive (no xmax) tuple in it
> because its xmin\xmax is committed and SnapshotSelf is happy with it
How does HeapTupleSatisfiesSelf() recognize the status of any XID w/o using a
snapshot? Do you mean by checking the commit log (TransactionIdDidCommit) ?
> 3) each update/delete during the replay selects the last existing
> tuple version, updates it xmax=original xid and inserts a new one
> keeping with xmin=orignal xid
> 4) --//--
> 5) --//--
> > However, at the time we're replaying the UPDATE in the new table, the tuple
> > may have been already deleted from the old table, and the deleting transaction
> > may already have committed. In such a case, HeapTupleSatisfiesSelf() will
> > conclude the old version invisible and the we'll fail to replay the UPDATE.
>
> No, it will see it - because its xmax will be empty in the repacked
> version of the table.
You're right, it'll be empty in the new table.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-25 16:23 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-25 16:23 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Hi, Antonin
> How does HeapTupleSatisfiesSelf() recognize the status of any XID w/o using a
> snapshot? Do you mean by checking the commit log (TransactionIdDidCommit) ?
Yes, TransactionIdDidCommit. Another option is just invent a new
snapshot type - SnapshotBelieveEverythingCommitted - for that
particular case it should work - because all xmin/xmax written into
the new table are committed by design.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-25 17:22 ` Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-25 17:22 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hi, Antonin
>
> > How does HeapTupleSatisfiesSelf() recognize the status of any XID w/o using a
> > snapshot? Do you mean by checking the commit log (TransactionIdDidCommit) ?
>
> Yes, TransactionIdDidCommit.
I think the problem is that HeapTupleSatisfiesSelf() uses
TransactionIdIsInProgress() instead of checking the snapshot:
...
else if (TransactionIdIsInProgress(HeapTupleHeaderGetRawXmin(tuple)))
return false;
else if (TransactionIdDidCommit(HeapTupleHeaderGetRawXmin(tuple)))
...
When decoding (and replaying) data changes, you deal with the database state
as it was (far) in the past. However TransactionIdIsInProgress() is not
suitable for this purpose.
And since CommitTransaction() updates the commit log before removing the
transaction from ProcArray, I can even imagine race conditions: if a
transaction is committed and decoded fast enough, TransactionIdIsInProgress()
might still return true. In such a case, HeapTupleSatisfiesSelf() returns
false instead of calling TransactionIdDidCommit().
> Another option is just invent a new
> snapshot type - SnapshotBelieveEverythingCommitted - for that
> particular case it should work - because all xmin/xmax written into
> the new table are committed by design.
I'd prefer optimization of the logical decoding for REPACK CONCURRENTLY, and
using the MVCC snapshots.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-25 18:18 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-25 18:18 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Antonin Houska <ah@cybertec.at>:
> I think the problem is that HeapTupleSatisfiesSelf() uses
> TransactionIdIsInProgress() instead of checking the snapshot:
Yes, some issues might be possible for SnapshotSelf.
Possible solution is to override TransactionIdIsCurrentTransactionId
to true (like you did with nParallelCurrentXids but just return true).
IIUC, in that case all checks are going to behave the same way as in v5 version.
> I'd prefer optimization of the logical decoding for REPACK CONCURRENTLY, and
> using the MVCC snapshots.
It is also possible, but it is much more complex and feels like overkill to me.
We need just a way to find the latest version of row in the world of
all-committed transactions without any concurrent writers - I am
pretty sure it is possible to achieve in a more simple and effective
way.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-26 08:46 ` Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-26 08:46 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Antonin Houska <ah@cybertec.at>:
> > I think the problem is that HeapTupleSatisfiesSelf() uses
> > TransactionIdIsInProgress() instead of checking the snapshot:
>
> Yes, some issues might be possible for SnapshotSelf.
> Possible solution is to override TransactionIdIsCurrentTransactionId
> to true (like you did with nParallelCurrentXids but just return true).
> IIUC, in that case all checks are going to behave the same way as in v5 version.
I assume you mean v12-0005. Yes, that modifies
TransactionIdIsCurrentTransactionId(), so that the the transaction being
replayed recognizes if it (or its subtransaction) performed particular change
itself.
Although it could work, I think it'd be confusing to consider the transactions
being replayed as "current" from the point of view of the backend that
executes REPACK CONCURRENTLY.
But the primary issue is that in v12-0005,
TransactionIdIsCurrentTransactionId() gets the information on "current
transactions" from snapshots - see the calls of SetRepackCurrentXids() before
each scan. It's probably not what you want.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-26 09:02 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-26 09:02 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Antonin Houska <ah@cybertec.at>:
> Although it could work, I think it'd be confusing to consider the transactions
> being replayed as "current" from the point of view of the backend that
> executes REPACK CONCURRENTLY.
Just realized SnapshotDirty is the thing that fits into the role - it
respects not-yet committed transactions, giving enough information to
wait for them.
It is already used in a similar pattern in
check_exclusion_or_unique_constraint and RelationFindReplTupleByIndex.
So, it is easy to detect the case of the race you described previously
and retry + there is no sense to hack around
TransactionIdIsCurrentTransactionId.
BWT, btree + SnapshotDirty has issue [0], but it is a different story
and happens only with concurrent updates which are not present in the
current scope.
[0]: https://www.postgresql.org/message-id/flat/CADzfLwXGhH_qD6RGqPyEeKdmHgr-HpA-tASYdi5onP%2BRyP5TCw%40m...
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-26 13:31 ` Antonin Houska <ah@cybertec.at>
2025-08-27 00:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-26 13:31 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Antonin Houska <ah@cybertec.at>:
>
> > Although it could work, I think it'd be confusing to consider the transactions
> > being replayed as "current" from the point of view of the backend that
> > executes REPACK CONCURRENTLY.
>
> Just realized SnapshotDirty is the thing that fits into the role - it
> respects not-yet committed transactions, giving enough information to
> wait for them.
> It is already used in a similar pattern in
> check_exclusion_or_unique_constraint and RelationFindReplTupleByIndex.
>
> So, it is easy to detect the case of the race you described previously
> and retry + there is no sense to hack around
> TransactionIdIsCurrentTransactionId.
Where exactly should HeapTupleSatisfiesDirty() conclude that the tuple is
visible? TransactionIdIsCurrentTransactionId() will not do w/o the
modifications that you proposed earlier [1] and TransactionIdIsInProgress() is
not suitable as I explained in [2].
I understand your idea that there are no transaction aborts in the new table,
which makes things simpler. I cannot judge if it's worth inventing a new kind
of snapshot. Anyway, I think you'd then also need to hack
HeapTupleSatisfiesUpdate(). Isn't that too invasive?
[1] https://www.postgresql.org/message-id/CADzfLwUqyOmpkLmciecBy4aBN1sohQVZ2Hgc6m-tjSUqDRHwyQ%40mail.gma...
[2] https://www.postgresql.org/message-id/24483.1756142534%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-27 00:38 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 06:16 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-27 00:38 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Hello, Antonin!
Antonin Houska <ah@cybertec.at>:
>
> Where exactly should HeapTupleSatisfiesDirty() conclude that the tuple is
> visible? TransactionIdIsCurrentTransactionId() will not do w/o the
> modifications that you proposed earlier [1] and TransactionIdIsInProgress() is
> not suitable as I explained in [2].
HeapTupleSatisfiesDirty is designed to respect even not-yet-committed
transactions and provides additional related information.
else if (TransactionIdIsInProgress(HeapTupleHeaderGetRawXmin(tuple)))
{
/*
* Return the speculative token to caller. Caller can worry about
* xmax, since it requires a conclusively locked row version, and
* a concurrent update to this tuple is a conflict of its
* purposes.
*/
if (HeapTupleHeaderIsSpeculative(tuple))
{
snapshot->speculativeToken =
HeapTupleHeaderGetSpeculativeToken(tuple);
Assert(snapshot->speculativeToken != 0);
}
snapshot->xmin = HeapTupleHeaderGetRawXmin(tuple);
/* XXX shouldn't we fall through to look at xmax? */
return true; /* in insertion by other */
}
So, it returns true when TransactionIdIsInProgress is true.
However, that alone is not sufficient to trust the result in the common case.
You may check check_exclusion_or_unique_constraint or
RelationFindReplTupleByIndex for that pattern:
if xmin is set in the snapshot (a special hack in SnapshotDirty to
provide additional information from the check), we wait for the
ongoing transaction (or one that is actually committed but not yet
properly reflected in the proc array), and then retry the entire tuple
search.
So, the race condition you explained in [2] will be resolved by a
retry, and the changes to TransactionIdIsInProgress described in [1]
are not necessary.
> I understand your idea that there are no transaction aborts in the new table,
> which makes things simpler. I cannot judge if it's worth inventing a new kind
> of snapshot. Anyway, I think you'd then also need to hack
> HeapTupleSatisfiesUpdate(). Isn't that too invasive?
It seems that HeapTupleSatisfiesUpdate is also fine as it currently
exists (we'll see the committed version after retry)..
The solution appears to be non-invasive:
* uses the existing snapshot type
* follows the existing usage pattern
* leaves TransactionIdIsInProgress and HeapTupleSatisfiesUpdate unchanged
The main change is that xmin/xmax values are forced from the arguments
- but that seems unavoidable in any case.
I'll try to make some kind of prototype this weekend + cover race
condition you mentioned in specs.
Maybe some corner cases will appear.
By the way, there's one more optimization we could apply in both
MVCC-safe and non-MVCC-safe cases: setting the HEAP_XMIN_COMMITTED /
HEAP_XMAX_COMMITTED bit in the new table:
* in the MVCC-safe approach, the transaction is already committed.
* in the non-MVCC-safe case, it isn’t committed yet - but no one will
examine that bit before it commits (though this approach does feel
more fragile).
This could help avoid potential storms of full-page writes caused by
SetHintBit after the table switch.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 00:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-27 06:16 ` Antonin Houska <ah@cybertec.at>
2025-08-27 08:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-27 06:16 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hello, Antonin!
>
> Antonin Houska <ah@cybertec.at>:
> >
> > Where exactly should HeapTupleSatisfiesDirty() conclude that the tuple is
> > visible? TransactionIdIsCurrentTransactionId() will not do w/o the
> > modifications that you proposed earlier [1] and TransactionIdIsInProgress() is
> > not suitable as I explained in [2].
>
> HeapTupleSatisfiesDirty is designed to respect even not-yet-committed
> transactions and provides additional related information.
>
> else if (TransactionIdIsInProgress(HeapTupleHeaderGetRawXmin(tuple)))
> {
> /*
> * Return the speculative token to caller. Caller can worry about
> * xmax, since it requires a conclusively locked row version, and
> * a concurrent update to this tuple is a conflict of its
> * purposes.
> */
> if (HeapTupleHeaderIsSpeculative(tuple))
> {
> snapshot->speculativeToken =
> HeapTupleHeaderGetSpeculativeToken(tuple);
>
> Assert(snapshot->speculativeToken != 0);
> }
>
> snapshot->xmin = HeapTupleHeaderGetRawXmin(tuple);
> /* XXX shouldn't we fall through to look at xmax? */
> return true; /* in insertion by other */
> }
>
> So, it returns true when TransactionIdIsInProgress is true.
> However, that alone is not sufficient to trust the result in the common case.
>
> You may check check_exclusion_or_unique_constraint or
> RelationFindReplTupleByIndex for that pattern:
> if xmin is set in the snapshot (a special hack in SnapshotDirty to
> provide additional information from the check), we wait for the
> ongoing transaction (or one that is actually committed but not yet
> properly reflected in the proc array), and then retry the entire tuple
> search.
>
> So, the race condition you explained in [2] will be resolved by a
> retry, and the changes to TransactionIdIsInProgress described in [1]
> are not necessary.
I insist that this is a misuse of TransactionIdIsInProgress(). When dealing
with logical decoding, only WAL should tell whether particular transaction is
still running. AFAICS this is how reorderbuffer.c works.
A new kind of snapshot seems like (much) cleaner solution at the moment.
> I'll try to make some kind of prototype this weekend + cover race
> condition you mentioned in specs.
> Maybe some corner cases will appear.
No rush. First, the MVCC safety is not likely to be included in v19
[1]. Second, I think it's good to let others propose their ideas before
writing code.
> By the way, there's one more optimization we could apply in both
> MVCC-safe and non-MVCC-safe cases: setting the HEAP_XMIN_COMMITTED /
> HEAP_XMAX_COMMITTED bit in the new table:
> * in the MVCC-safe approach, the transaction is already committed.
> * in the non-MVCC-safe case, it isn’t committed yet - but no one will
> examine that bit before it commits (though this approach does feel
> more fragile).
>
> This could help avoid potential storms of full-page writes caused by
> SetHintBit after the table switch.
Good idea, thanks.
[1] https://www.postgresql.org/message-id/202504040733.ysuy5gad55md%40alvherre.pgsql
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 00:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 06:16 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-27 08:22 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 10:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-01 00:16 ` Re: Adding REPACK [concurrently] Michael Paquier <michael@paquier.xyz>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-27 08:22 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Antonin Houska <ah@cybertec.at>:
> I insist that this is a misuse of TransactionIdIsInProgress(). When dealing
> with logical decoding, only WAL should tell whether particular transaction is
> still running. AFAICS this is how reorderbuffer.c works.
Hm... Maybe, but at the same time we already have SnapshotDirty used
in that way and it even deals with the same race....
But I agree - a special kind of snapshot is a more accurate solution.
> A new kind of snapshot seems like (much) cleaner solution at the moment.
Do you mean some kind of snapshot which only uses
TransactionIdDidCommit/Abort ignoring
TransactionIdIsCurrentTransactionId/TransactionIdIsInProgress?
Actually it behaves like SnapshotBelieveEverythingCommitted in that
particular case, but TransactionIdDidCommit/Abort may be used as some
kind of assert/error source to be sure everything is going as
designed.
And, yes, for the new snapshot we need to have
HeapTupleSatisfiesUpdate to be modified.
Also, to deal with that particular race we may just use
XactLockTableWait(xid, NULL, NULL, XLTW_None) before starting
transaction replay.
> No rush. First, the MVCC safety is not likely to be included in v19 [1].
That worries me - it is not the behaviour someone expects from a
database by default. At least the warning should be much more visible
and obvious.
I think most of user will expect the same guarantees as [CREATE|RE]
INDEX CONCURRENTLY provides.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 00:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 06:16 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 08:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-27 10:11 ` Antonin Houska <ah@cybertec.at>
2025-08-27 10:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-27 10:11 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > A new kind of snapshot seems like (much) cleaner solution at the moment.
>
> Do you mean some kind of snapshot which only uses
> TransactionIdDidCommit/Abort ignoring
> TransactionIdIsCurrentTransactionId/TransactionIdIsInProgress?
> Actually it behaves like SnapshotBelieveEverythingCommitted in that
> particular case, but TransactionIdDidCommit/Abort may be used as some
> kind of assert/error source to be sure everything is going as
> designed.
Given that there should be no (sub)transaction aborts in the new table, I
think you only need to check that the tuple has valid xmin and invalid xmax.
I think the XID should be in the commit log at the time the transaction is
being replayed, so it should be legal to use TransactionIdDidCommit/Abort in
Assert() statements. (And as long as REPACK CONCURRENTLY will use
ShareUpdateExclusiveLock, which conflicts with VACUUM, pg_class(relfrozenxid)
for given table should not advance during the processing, and therefore the
replayed XIDs should not be truncated from the commit log while REPACK
CONCURRENTLY is running.)
> And, yes, for the new snapshot we need to have
> HeapTupleSatisfiesUpdate to be modified.
>
> Also, to deal with that particular race we may just use
> XactLockTableWait(xid, NULL, NULL, XLTW_None) before starting
> transaction replay.
Do you mean the race related to TransactionIdIsInProgress()? Not sure I
understand, as you suggested above that you no longer need the function.
> > No rush. First, the MVCC safety is not likely to be included in v19 [1].
>
> That worries me - it is not the behaviour someone expects from a
> database by default. At least the warning should be much more visible
> and obvious.
> I think most of user will expect the same guarantees as [CREATE|RE]
> INDEX CONCURRENTLY provides.
It does not really worry me. The pg_squeeze extension is not MVCC-safe and I
remember there were only 1 or 2 related complaints throughout its
existence. (pg_repack isn't MVCC-safe as well, but I don't keep track of its
issues.)
Of course, user documentation should warn about the problem, in a way it does
for other commands (typically ALTER TABLE).
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 00:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 06:16 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 08:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 10:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-27 10:55 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-27 10:55 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Antonin Houska <ah@cybertec.at>:
> Do you mean the race related to TransactionIdIsInProgress()? Not sure I
> understand, as you suggested above that you no longer need the function.
The "lightweight" approaches I see so far:
* XactLockTableWait before replay + SnapshotSelf(GetLatestSnapshot?)
* SnapshotDirty + retry logic
* SnapshotBelieveEverythingCommitted + modification of
HeapTupleSatisfiesUpdate (because it called by heap_update and looks
into TransactionIdIsInProgress)
> It does not really worry me. The pg_squeeze extension is not MVCC-safe and I
> remember there were only 1 or 2 related complaints throughout its
> existence. (pg_repack isn't MVCC-safe as well, but I don't keep track of its
> issues.)
But pg_squeeze and pg_repack are extensions. If we are moving that
mechanics into core I'd expect some improvements over pg_squeeze.
MVCC-safety of REINDEX CONCURRENTLY makes it possible to run it on a
regular basis as some kind of background job. It would be nice to have
something like this for the heap.
I agree the initial approach is too invasive, complex and
performance-heavy to push it forward now.
But, any of "lightweight" feels like a good candidate to be shipped
with the feature itself - relatively easy and non-invasive.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 00:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 06:16 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 08:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-09-01 00:16 ` Michael Paquier <michael@paquier.xyz>
1 sibling, 0 replies; 416+ messages in thread
From: Michael Paquier @ 2025-09-01 00:16 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Aug 27, 2025 at 10:22:24AM +0200, Mihail Nikalayeu wrote:
> That worries me - it is not the behaviour someone expects from a
> database by default. At least the warning should be much more visible
> and obvious.
> I think most of user will expect the same guarantees as [CREATE|RE]
> INDEX CONCURRENTLY provides.
Having a unified path for the handling of the waits and the locking
sounds to me like a pretty good argument in favor of a basic
implementation.
In my experience, users do not really care about the time it takes to
complete a operation involving CONCURRENTLY if we allow concurrent
reads and writes in parallel of it. I have not looked at the proposal
in details, but before trying a more folkloric MVCC approach, relying
on basics that we know have been working for some time seems like a
good and sufficient initial step in terms of handling the waits and
the locks with table AMs (aka heap or something else).
Just my 2c.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../aLTl6mYK7exHnqUJ@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-25 16:36 ` Robert Treat <rob@xzilla.net>
2025-08-25 16:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Robert Treat @ 2025-08-25 16:36 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Aug 25, 2025 at 10:15 AM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
> 1) we have a whole initial table snapshot with all xmin copied from
> the original table. All such xmin are committed.
> 2) appling transaction sees ALL the self-alive (no xmax) tuple in it
> because its xmin\xmax is committed and SnapshotSelf is happy with it
> 3) each update/delete during the replay selects the last existing
> tuple version, updates it xmax=original xid and inserts a new one
> keeping with xmin=orignal xid
> 4) --//--
> 5) --//--
>
Advancing the tables min xid to at least repack XID is a pretty big
feature, but the above scenario sounds like it would result in any
non-modified pre-existing tuples ending up with their original xmin
rather than repack XID, which seems like it could lead to weird
side-effects. Maybe I am mis-thinking it though?
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 16:36 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-08-25 16:54 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-08-25 16:54 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Robert Treat <rob@xzilla.net> wrote:
> On Mon, Aug 25, 2025 at 10:15 AM Mihail Nikalayeu
> <mihailnikalayeu@gmail.com> wrote:
> > 1) we have a whole initial table snapshot with all xmin copied from
> > the original table. All such xmin are committed.
> > 2) appling transaction sees ALL the self-alive (no xmax) tuple in it
> > because its xmin\xmax is committed and SnapshotSelf is happy with it
> > 3) each update/delete during the replay selects the last existing
> > tuple version, updates it xmax=original xid and inserts a new one
> > keeping with xmin=orignal xid
> > 4) --//--
> > 5) --//--
> >
>
> Advancing the tables min xid to at least repack XID is a pretty big
> feature, but the above scenario sounds like it would result in any
> non-modified pre-existing tuples ending up with their original xmin
> rather than repack XID, which seems like it could lead to weird
> side-effects. Maybe I am mis-thinking it though?
What we discuss here is how to keep visibility information of tuples (xmin,
xmax, ...) unchanged. Both CLUSTER and VACUUM FULL already do that. However
it's not trivial to ensure that REPACK with the CONCURRENTLY option does as
well.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-28 21:39 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-29 00:32 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-28 21:39 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
On 2025-Aug-21, Mihail Nikalayeu wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org>:
> > I proposed to leave this part out initially, which is why it hasn't been
> > reposted. We can review and discuss after the initial patches are in.
>
> I think it is worth pushing it at least in the same release cycle.
If others are motivated enough to certify it, maybe we can consider it.
But I don't think I'm going to have time to get this part reviewed and
committed in time for 19, so you might need another committer.
> > Because having an MVCC-safe mode has drawbacks, IMO we should make it
> > optional.
>
> Do you mean some option for the command? Like REPACK (CONCURRENTLY, SAFE)?
Yes, exactly that.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"La grandeza es una experiencia transitoria. Nunca es consistente.
Depende en gran parte de la imaginación humana creadora de mitos"
(Irulan)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-28 21:39 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-29 00:32 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-29 07:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-29 00:32 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
Hello, Álvaro!
> If others are motivated enough to certify it, maybe we can consider it.
> But I don't think I'm going to have time to get this part reviewed and
> committed in time for 19, so you might need another committer.
I don't think it is realistic to involve another committer - it is
just a well-known curse of all non-committers :)
> > > Because having an MVCC-safe mode has drawbacks, IMO we should make it
> > > optional.
As far as I can see, the proposed "lightweight" solutions don't
introduce any drawbacks - unless something has been overlooked.
> > Do you mean some option for the command? Like REPACK (CONCURRENTLY, SAFE)?
> Yes, exactly that.
To be honest that approach feels a little bit strange for me. I work
in the database-consumer (not database-developer) industry and 90% of
DevOps engineers (or similar roles who deal with database maintenance
now) have no clue what MVCC is - and it is industry standard nowadays.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-28 21:39 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-29 00:32 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-29 07:41 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-08-29 07:41 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> In case of some incident related to that (in a large well-known
> company) the typical takeaway for readers of tech blogs will simply be
> "some command in Postgres is broken".
For _responsible_ users, the message will rather be that "some tech bloggers
do not bother to read user documentation".
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-11 14:22 ` Antonin Houska <ah@cybertec.at>
2025-08-15 12:32 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-11 14:22 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Fujii Masao <masao.fujii@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> One more thing - I think build_new_indexes and
> index_concurrently_create_copy are very close in semantics, so it
> might be a good idea to refactor them a bit.
You're right. I think I even used the latter for reference when writing the
first.
0002 in the attached series tries to fix that. build_new_indexes() (in 0004)
is simpler now.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-11 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-15 12:32 ` Antonin Houska <ah@cybertec.at>
2025-08-15 12:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-15 12:32 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Antonin Houska <ah@cybertec.at> wrote:
> Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
>
> > One more thing - I think build_new_indexes and
> > index_concurrently_create_copy are very close in semantics, so it
> > might be a good idea to refactor them a bit.
>
> You're right. I think I even used the latter for reference when writing the
> first.
>
> 0002 in the attached series tries to fix that. build_new_indexes() (in 0004)
> is simpler now.
This is v18 again. Parts 0001 through 0004 are unchanged, however 0005 is
added. It implements a new client application pg_repackdb. (If I posted 0005
alone its regression tests would not work. I wonder if the cfbot handles the
repeated occurence of the 'v18-' prefix correctly.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-11 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-15 12:32 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-15 12:48 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-15 12:48 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Fujii Masao <masao.fujii@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On 2025-Aug-15, Antonin Houska wrote:
> This is v18 again.
Thanks for this!
> Parts 0001 through 0004 are unchanged, however 0005 is added. It
> implements a new client application pg_repackdb. (If I posted 0005
> alone its regression tests would not work. I wonder if the cfbot
> handles the repeated occurence of the 'v18-' prefix correctly.)
Yeah, the cfbot is just going to take the attachments from the latest
email in the thread that has any, and assume they are the whole that
make up the patch. It wouldn't work to post just v18-0005 and assume
that the bot is going grab patches 0001 through 0004 from a previous
email, if that's what you're thinking. In short, what you did is
correct and necessary.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-07-27 06:00 ` Fujii Masao <masao.fujii@gmail.com>
6 siblings, 0 replies; 416+ messages in thread
From: Fujii Masao @ 2025-07-27 06:00 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>
On Sun, Jul 27, 2025 at 6:56 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Hello,
>
> Here's a patch to add REPACK and eventually the CONCURRENTLY flag to it.
> This is coming from [1]. The ultimate goal is to have an in-core tool
> to allow concurrent table rewrite to get rid of bloat;
+1
> right now, VACUUM
> FULL does that, but it's not concurrent. Users have resorted to using
> the pg_repack third-party tool, which is ancient and uses a weird
> internal implementation, as well as pg_squeeze, which uses logical
> decoding to capture changes that occur during the table rewrite. The
> patch submitted here, largely by Antonin Houska with some changes by me,
> is based on the the pg_squeeze code which he authored, and first
> introduces a new command called REPACK to absorb both VACUUM FULL and
> CLUSTER, followed by addition of a CONCURRENTLY flag to allow some forms
> of REPACK to operate online using logical decoding.
Does this mean REPACK CONCURRENTLY requires wal_level = logical,
while plain REPACK (without CONCURRENTLY) works with any wal_level
setting? If we eventually deprecate VACUUM FULL and CLUSTER,
I think plain REPACK should still be allowed with wal_level = minimal
or replica, so users with those settings can perform equivalent
processing.
+ if (!cluster_is_permitted_for_relation(tableOid, userid,
+ CLUSTER_COMMAND_CLUSTER))
As for the patch you attached, it seems to be an early WIP and
might not be ready for review yet?? BTW, I got the following
compilation failure and probably CLUSTER_COMMAND_CLUSTER
the above should be GetUserId().
-----------------
cluster.c:455:14: error: use of undeclared identifier 'CLUSTER_COMMAND_CLUSTER'
455 |
CLUSTER_COMMAND_CLUSTER))
|
^
1 error generated.
-----------------
Regards,
--
Fujii Masao
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-05 08:58 ` Antonin Houska <ah@cybertec.at>
2025-08-16 13:41 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
6 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-05 08:58 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> I made a few changes from Antonin's original at [2]. First, I modified
> the grammar to support "REPACK [tab] USING INDEX" without specifying the
> index name. With this change, all possibilities of the old commands are
> covered,
...
> Here's a list of existing commands, and how to write them in the current
> patch's proposal for REPACK:
>
> -- re-clusters all tables that have a clustered index set
> CLUSTER -> REPACK USING INDEX
>
> -- clusters the given table using the given index
> CLUSTER tab USING idx -> REPACK tab USING INDEX idx
>
> -- clusters this table using a clustered index; error if no index clustered
> CLUSTER tab -> REPACK tab USING INDEX
>
> -- vacuum-full all tables
> VACUUM FULL -> REPACK
>
> -- vacuum-full the specified table
> VACUUM FULL tab -> REPACK tab
>
Now that we want to cover the CLUSTER/VACUUM FULL completely, I've checked the
options of VACUUM FULL. I found two items not supported by REPACK (but also
not supported by by CLUSTER): ANALYZE and SKIP_DATABASE_STATS. Maybe just
let's mention that in the user documentation of REPACK?
(Besides that, VACUUM FULL accepts TRUNCATE and INDEX_CLEANUP options, but I
think these have no effect.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-05 08:58 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-16 13:41 ` Robert Treat <rob@xzilla.net>
2025-08-19 12:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 12:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Robert Treat @ 2025-08-16 13:41 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Aug 5, 2025 at 4:59 AM Antonin Houska <ah@cybertec.at> wrote:
>
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > I made a few changes from Antonin's original at [2]. First, I modified
> > the grammar to support "REPACK [tab] USING INDEX" without specifying the
> > index name. With this change, all possibilities of the old commands are
> > covered,
>
> ...
>
> > Here's a list of existing commands, and how to write them in the current
> > patch's proposal for REPACK:
> >
> > -- re-clusters all tables that have a clustered index set
> > CLUSTER -> REPACK USING INDEX
> >
> > -- clusters the given table using the given index
> > CLUSTER tab USING idx -> REPACK tab USING INDEX idx
> >
> > -- clusters this table using a clustered index; error if no index clustered
> > CLUSTER tab -> REPACK tab USING INDEX
> >
In the v18 patch, the docs say that repack doesn't remember the index,
but it seems we are still calling mark_index_clustered, so I think the
above is true but we need to update the docs(?).
> > -- vacuum-full all tables
> > VACUUM FULL -> REPACK
> >
> > -- vacuum-full the specified table
> > VACUUM FULL tab -> REPACK tab
> >
>
> Now that we want to cover the CLUSTER/VACUUM FULL completely, I've checked the
> options of VACUUM FULL. I found two items not supported by REPACK (but also
> not supported by by CLUSTER): ANALYZE and SKIP_DATABASE_STATS. Maybe just
> let's mention that in the user documentation of REPACK?
>
I would note that both pg_repack and pg_squeeze analyze by default,
and running "vacuum full analyze" is the recommended behavior, so not
having analyze included is a step backwards.
> (Besides that, VACUUM FULL accepts TRUNCATE and INDEX_CLEANUP options, but I
> think these have no effect.)
>
Yeah, these seem safe to ignore.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-05 08:58 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-16 13:41 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-08-19 12:22 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 08:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-19 12:22 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Antonin Houska <ah@cybertec.at>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On 2025-Aug-16, Robert Treat wrote:
> On Tue, Aug 5, 2025 at 4:59 AM Antonin Houska <ah@cybertec.at> wrote:
> > Now that we want to cover the CLUSTER/VACUUM FULL completely, I've checked the
> > options of VACUUM FULL. I found two items not supported by REPACK (but also
> > not supported by by CLUSTER): ANALYZE and SKIP_DATABASE_STATS. Maybe just
> > let's mention that in the user documentation of REPACK?
>
> I would note that both pg_repack and pg_squeeze analyze by default,
> and running "vacuum full analyze" is the recommended behavior, so not
> having analyze included is a step backwards.
Make sense to add ANALYZE as an option to repack, yeah.
So if I repack a single table with
REPACK (ANALYZE) table USING INDEX;
then do you expect that this would first cluster the table under
AccessExclusiveLock, then release the lock to do the analyze step, or
would the analyze be done under the same lock? This is significant for
a query that starts while repack is running, because if we release the
AEL then the query is planned when there are no stats for the table,
which might be bad.
I think the time to run the analyze step should be considerable shorter
than the time to run the repacking step, so running both together under
the same lock should be okay.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Computing is too important to be left to men." (Karen Spärck Jones)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-05 08:58 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-16 13:41 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-19 12:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-20 08:33 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-08-20 08:33 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2025-Aug-16, Robert Treat wrote:
>
> > On Tue, Aug 5, 2025 at 4:59 AM Antonin Houska <ah@cybertec.at> wrote:
>
> > > Now that we want to cover the CLUSTER/VACUUM FULL completely, I've checked the
> > > options of VACUUM FULL. I found two items not supported by REPACK (but also
> > > not supported by by CLUSTER): ANALYZE and SKIP_DATABASE_STATS. Maybe just
> > > let's mention that in the user documentation of REPACK?
> >
> > I would note that both pg_repack and pg_squeeze analyze by default,
> > and running "vacuum full analyze" is the recommended behavior, so not
> > having analyze included is a step backwards.
>
> Make sense to add ANALYZE as an option to repack, yeah.
>
> So if I repack a single table with
> REPACK (ANALYZE) table USING INDEX;
>
> then do you expect that this would first cluster the table under
> AccessExclusiveLock, then release the lock to do the analyze step, or
> would the analyze be done under the same lock? This is significant for
> a query that starts while repack is running, because if we release the
> AEL then the query is planned when there are no stats for the table,
> which might be bad.
>
> I think the time to run the analyze step should be considerable shorter
> than the time to run the repacking step, so running both together under
> the same lock should be okay.
AFAICS, VACUUM FULL first releases the AEL, then it analyzes the table. If
users did not complain so far, I'd assume that vacuum_rel() (effectively
cluster_rel() in the FULL case) does not change the stats that much.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-05 08:58 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-16 13:41 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-08-19 12:23 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-19 12:23 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Antonin Houska <ah@cybertec.at>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On 2025-Aug-16, Robert Treat wrote:
> In the v18 patch, the docs say that repack doesn't remember the index,
> but it seems we are still calling mark_index_clustered, so I think the
> above is true but we need to update the docs(?).
Yes, the docs are obsolete on this point, I'm in the process of updating
them. Thanks for pointing this out.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"La victoria es para quien se atreve a estar solo"
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-19 18:53 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 08:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-21 22:06 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
6 siblings, 2 replies; 416+ messages in thread
From: Álvaro Herrera @ 2025-08-19 18:53 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; +Cc: Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Hello,
Here's a second cut of the initial REPACK work. Antonin added an
implementation of pg_repackdb, and there's also a couple of bug fixes
that were reported in the thread. I also added support for the ANALYZE
option as noted by Robert Treat, though it only works if you specify a
single non-partitioned table. Adding for the multi-table case is likely
easy, but I didn't try.
I purposefully do not include the CONCURRENTLY work yet -- I want to get
this part commitable-clean first, then we can continue work on the
logical decoding work on top of that.
Note choice of shell command name: though all the other programs in
src/bin/scripts do not use the "pg_" prefix, this one does; we thought
it made no sense to follow the old programs as precedent because there
seems to be a lament for the lack of pg_ prefix in those, and we only
keep what they are because of their long history. This one has no
history.
Still on pg_repackdb, the implementation here is to install a symlink
called pg_repackdb which points to vacuumdb, and make the program behave
differently when called in this way. The amount of additional code for
this is relatively small, so I think this is a worthy technique --
assuming it works. If it doesn't, Antonin proposed a separate binary
that just calls some functions from vacuumdb. Or maybe we could have a
common source file that both utilities call.
I edited the docs a bit, limiting the exposure of CLUSTER and VACUUM
FULL, and instead redirecting the user to the REPACK docs. In the
REPACK docs I modified things for additional clarity.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Selbst das größte Genie würde nicht weit kommen, wenn es
alles seinem eigenen Innern verdanken wollte." (Johann Wolfgang von Goethe)
Ni aún el genio más grande llegaría muy lejos si
quisiera sacarlo todo de su propio interior.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-08-20 08:53 ` Antonin Houska <ah@cybertec.at>
2025-08-20 12:07 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-20 08:53 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org; +Cc: alvherre@kurilemu.de; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Álvaro Herrera <alvherre@kurilemu.de> wrote:
> Still on pg_repackdb, the implementation here is to install a symlink
> called pg_repackdb which points to vacuumdb, and make the program behave
> differently when called in this way. The amount of additional code for
> this is relatively small, so I think this is a worthy technique --
> assuming it works. If it doesn't, Antonin proposed a separate binary
> that just calls some functions from vacuumdb. Or maybe we could have a
> common source file that both utilities call.
There's an issue with the symlink, maybe some meson expert can help. In
particular, the CI on Windows ends up with the following error:
ERROR: Tried to install symlink to missing file C:/cirrus/build/tmp_install/usr/local/pgsql/bin/vacuumdb
(The reason it does not happen on other platforms might be that the build is
slower on Windows, and thus it's more prone to some specific race conditions.)
It appears that the 'point_to' argument of the 'install_symlink()' function
[1] is only a string rather than a "real target" [2]. That's likely the reason
the function does not wait for the creation of the 'vacuumdb' executable.
I could not find another symlink of this kind in the tree. (AFAICS, the
postmaster->postgres symlink had been removed before Meson has been
introduced.)
Does anyone happen to have a clue? Thanks.
[1] https://mesonbuild.com/Reference-manual_functions.html#install_symlink
[2] https://mesonbuild.com/Reference-manual_returned_tgt.html
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 08:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-20 12:07 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Álvaro Herrera @ 2025-08-20 12:07 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On 2025-Aug-20, Antonin Houska wrote:
> There's an issue with the symlink, maybe some meson expert can help. In
> particular, the CI on Windows ends up with the following error:
>
> ERROR: Tried to install symlink to missing file C:/cirrus/build/tmp_install/usr/local/pgsql/bin/vacuumdb
Hmm, that's not the problem I see in the CI run from the commitfest app:
https://cirrus-ci.com/task/5608274336153600
[19:11:00.642] FAILED: [code=2] src/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj
[19:11:00.642] "cl" "-Isrc\bin\scripts\vacuumdb.exe.p" "-Isrc\include" "-I..\src\include" "-Ic:\openssl\1.1\include" "-I..\src\include\port\win32" "-I..\src\include\port\win32_msvc" "-Isrc/interfaces/libpq" "-I..\src\interfaces\libpq" "/MDd" "/nologo" "/showIncludes" "/utf-8" "/W2" "/Od" "/Zi" "/Zc:preprocessor" "/DWIN32" "/DWINDOWS" "/D__WINDOWS__" "/D__WIN32__" "/D_CRT_SECURE_NO_DEPRECATE" "/D_CRT_NONSTDC_NO_DEPRECATE" "/wd4018" "/wd4244" "/wd4273" "/wd4101" "/wd4102" "/wd4090" "/wd4267" "/Fdsrc\bin\scripts\vacuumdb.exe.p\vacuumdb.c.pdb" /Fosrc/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj "/c" ../src/bin/scripts/vacuumdb.c
[19:11:00.642] ../src/bin/scripts/vacuumdb.c(186): error C2059: syntax error: '}'
[19:11:00.642] ../src/bin/scripts/vacuumdb.c(197): warning C4034: sizeof returns 0
The real problem here seems to be the empty long_options_repack array.
I removed it and started a new run to see what happens. Running now:
https://cirrus-ci.com/build/4961902171783168
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 08:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 12:07 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-08-20 14:22 ` Antonin Houska <ah@cybertec.at>
2025-08-20 16:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-20 14:22 UTC (permalink / raw)
To: alvherre@kurilemu.de; +Cc: pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Álvaro Herrera <alvherre@kurilemu.de> wrote:
> On 2025-Aug-20, Antonin Houska wrote:
>
> > There's an issue with the symlink, maybe some meson expert can help. In
> > particular, the CI on Windows ends up with the following error:
> >
> > ERROR: Tried to install symlink to missing file C:/cirrus/build/tmp_install/usr/local/pgsql/bin/vacuumdb
>
> Hmm, that's not the problem I see in the CI run from the commitfest app:
>
> https://cirrus-ci.com/task/5608274336153600
I was referring to the other build that you shared off-list (probably
independent from cfbot):
https://cirrus-ci.com/build/4726227505774592
> [19:11:00.642] FAILED: [code=2] src/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj
> [19:11:00.642] "cl" "-Isrc\bin\scripts\vacuumdb.exe.p" "-Isrc\include" "-I..\src\include" "-Ic:\openssl\1.1\include" "-I..\src\include\port\win32" "-I..\src\include\port\win32_msvc" "-Isrc/interfaces/libpq" "-I..\src\interfaces\libpq" "/MDd" "/nologo" "/showIncludes" "/utf-8" "/W2" "/Od" "/Zi" "/Zc:preprocessor" "/DWIN32" "/DWINDOWS" "/D__WINDOWS__" "/D__WIN32__" "/D_CRT_SECURE_NO_DEPRECATE" "/D_CRT_NONSTDC_NO_DEPRECATE" "/wd4018" "/wd4244" "/wd4273" "/wd4101" "/wd4102" "/wd4090" "/wd4267" "/Fdsrc\bin\scripts\vacuumdb.exe.p\vacuumdb.c.pdb" /Fosrc/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj "/c" ../src/bin/scripts/vacuumdb.c
> [19:11:00.642] ../src/bin/scripts/vacuumdb.c(186): error C2059: syntax error: '}'
> [19:11:00.642] ../src/bin/scripts/vacuumdb.c(197): warning C4034: sizeof returns 0
>
> The real problem here seems to be the empty long_options_repack array.
> I removed it and started a new run to see what happens. Running now:
> https://cirrus-ci.com/build/4961902171783168
The symlink issue occurred at "Windows - Server 2019, MinGW64 - Meson", where
the code compiled well. The compilation failure mentioned above comes from
"Windows - Server 2019, VS 2019 - Meson & ninja". I think it's still possible
that the symlink issue will occur there once the compilation is fixed.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 08:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 12:07 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-20 16:11 ` Andres Freund <andres@anarazel.de>
2025-08-21 18:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Andres Freund @ 2025-08-20 16:11 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: alvherre@kurilemu.de; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Hi,
On 2025-08-20 16:22:41 +0200, Antonin Houska wrote:
> Álvaro Herrera <alvherre@kurilemu.de> wrote:
>
> > On 2025-Aug-20, Antonin Houska wrote:
> >
> > > There's an issue with the symlink, maybe some meson expert can help. In
> > > particular, the CI on Windows ends up with the following error:
> > >
> > > ERROR: Tried to install symlink to missing file C:/cirrus/build/tmp_install/usr/local/pgsql/bin/vacuumdb
> >
> > Hmm, that's not the problem I see in the CI run from the commitfest app:
> >
> > https://cirrus-ci.com/task/5608274336153600
>
> I was referring to the other build that you shared off-list (probably
> independent from cfbot):
>
> https://cirrus-ci.com/build/4726227505774592
>
> > [19:11:00.642] FAILED: [code=2] src/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj
> > [19:11:00.642] "cl" "-Isrc\bin\scripts\vacuumdb.exe.p" "-Isrc\include" "-I..\src\include" "-Ic:\openssl\1.1\include" "-I..\src\include\port\win32" "-I..\src\include\port\win32_msvc" "-Isrc/interfaces/libpq" "-I..\src\interfaces\libpq" "/MDd" "/nologo" "/showIncludes" "/utf-8" "/W2" "/Od" "/Zi" "/Zc:preprocessor" "/DWIN32" "/DWINDOWS" "/D__WINDOWS__" "/D__WIN32__" "/D_CRT_SECURE_NO_DEPRECATE" "/D_CRT_NONSTDC_NO_DEPRECATE" "/wd4018" "/wd4244" "/wd4273" "/wd4101" "/wd4102" "/wd4090" "/wd4267" "/Fdsrc\bin\scripts\vacuumdb.exe.p\vacuumdb.c.pdb" /Fosrc/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj "/c" ../src/bin/scripts/vacuumdb.c
> > [19:11:00.642] ../src/bin/scripts/vacuumdb.c(186): error C2059: syntax error: '}'
> > [19:11:00.642] ../src/bin/scripts/vacuumdb.c(197): warning C4034: sizeof returns 0
> >
> > The real problem here seems to be the empty long_options_repack array.
> > I removed it and started a new run to see what happens. Running now:
> > https://cirrus-ci.com/build/4961902171783168
>
> The symlink issue occurred at "Windows - Server 2019, MinGW64 - Meson", where
> the code compiled well. The compilation failure mentioned above comes from
> "Windows - Server 2019, VS 2019 - Meson & ninja". I think it's still possible
> that the symlink issue will occur there once the compilation is fixed.
FWIW, I don't think it's particularly wise to rely on symlinks on windows -
IIRC they will often not be enabled outside of development environments.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 08:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 12:07 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 16:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2025-08-21 18:14 ` Antonin Houska <ah@cybertec.at>
2025-08-21 18:16 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-08-21 18:14 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: alvherre@kurilemu.de; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Andres Freund <andres@anarazel.de> wrote:
> Hi,
>
> On 2025-08-20 16:22:41 +0200, Antonin Houska wrote:
> > Álvaro Herrera <alvherre@kurilemu.de> wrote:
> >
> > > On 2025-Aug-20, Antonin Houska wrote:
> > >
> > > > There's an issue with the symlink, maybe some meson expert can help. In
> > > > particular, the CI on Windows ends up with the following error:
> > > >
> > > > ERROR: Tried to install symlink to missing file C:/cirrus/build/tmp_install/usr/local/pgsql/bin/vacuumdb
> > >
> > > Hmm, that's not the problem I see in the CI run from the commitfest app:
> > >
> > > https://cirrus-ci.com/task/5608274336153600
> >
> > I was referring to the other build that you shared off-list (probably
> > independent from cfbot):
> >
> > https://cirrus-ci.com/build/4726227505774592
> >
> > > [19:11:00.642] FAILED: [code=2] src/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj
> > > [19:11:00.642] "cl" "-Isrc\bin\scripts\vacuumdb.exe.p" "-Isrc\include" "-I..\src\include" "-Ic:\openssl\1.1\include" "-I..\src\include\port\win32" "-I..\src\include\port\win32_msvc" "-Isrc/interfaces/libpq" "-I..\src\interfaces\libpq" "/MDd" "/nologo" "/showIncludes" "/utf-8" "/W2" "/Od" "/Zi" "/Zc:preprocessor" "/DWIN32" "/DWINDOWS" "/D__WINDOWS__" "/D__WIN32__" "/D_CRT_SECURE_NO_DEPRECATE" "/D_CRT_NONSTDC_NO_DEPRECATE" "/wd4018" "/wd4244" "/wd4273" "/wd4101" "/wd4102" "/wd4090" "/wd4267" "/Fdsrc\bin\scripts\vacuumdb.exe.p\vacuumdb.c.pdb" /Fosrc/bin/scripts/vacuumdb.exe.p/vacuumdb.c.obj "/c" ../src/bin/scripts/vacuumdb.c
> > > [19:11:00.642] ../src/bin/scripts/vacuumdb.c(186): error C2059: syntax error: '}'
> > > [19:11:00.642] ../src/bin/scripts/vacuumdb.c(197): warning C4034: sizeof returns 0
> > >
> > > The real problem here seems to be the empty long_options_repack array.
> > > I removed it and started a new run to see what happens. Running now:
> > > https://cirrus-ci.com/build/4961902171783168
> >
> > The symlink issue occurred at "Windows - Server 2019, MinGW64 - Meson", where
> > the code compiled well. The compilation failure mentioned above comes from
> > "Windows - Server 2019, VS 2019 - Meson & ninja". I think it's still possible
> > that the symlink issue will occur there once the compilation is fixed.
>
> FWIW, I don't think it's particularly wise to rely on symlinks on windows -
> IIRC they will often not be enabled outside of development environments.
ok, installing a copy of the same executable with a different name seems more
reliable. At least that's how the postmaster->postgres link used to be
handled, if I read Makefile correctly. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 08:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 12:07 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 16:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2025-08-21 18:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-08-21 18:16 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 416+ messages in thread
From: Andres Freund @ 2025-08-21 18:16 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: alvherre@kurilemu.de; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Hi,
On 2025-08-21 20:14:14 +0200, Antonin Houska wrote:
> ok, installing a copy of the same executable with a different name seems more
> reliable. At least that's how the postmaster->postgres link used to be
> handled, if I read Makefile correctly. Thanks.
I have not followed this thread, but I don't think the whole thing of having a
single executable with multiple names is worth doing. Just make whatever an
option, instead of having multiple "executables".
Greetings,
Andres
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-08-21 22:06 ` Robert Treat <rob@xzilla.net>
2025-08-22 09:40 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
1 sibling, 1 reply; 416+ messages in thread
From: Robert Treat @ 2025-08-21 22:06 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Tue, Aug 19, 2025 at 2:53 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
> Note choice of shell command name: though all the other programs in
> src/bin/scripts do not use the "pg_" prefix, this one does; we thought
> it made no sense to follow the old programs as precedent because there
> seems to be a lament for the lack of pg_ prefix in those, and we only
> keep what they are because of their long history. This one has no
> history.
>
> Still on pg_repackdb, the implementation here is to install a symlink
> called pg_repackdb which points to vacuumdb, and make the program behave
> differently when called in this way. The amount of additional code for
> this is relatively small, so I think this is a worthy technique --
> assuming it works. If it doesn't, Antonin proposed a separate binary
> that just calls some functions from vacuumdb. Or maybe we could have a
> common source file that both utilities call.
>
What's the plan for clusterdb? It seems like we'd ideally create a
stand alone pg_repackdb which replaces clusterdb and also allows us to
remove the FULL options from vacuumdb.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-21 22:06 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-08-22 09:40 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-08-22 20:32 ` Re: Adding REPACK [concurrently] Euler Taveira <euler@eulerto.com>
0 siblings, 1 reply; 416+ messages in thread
From: Álvaro Herrera @ 2025-08-22 09:40 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On 2025-Aug-21, Robert Treat wrote:
> What's the plan for clusterdb? It seems like we'd ideally create a
> stand alone pg_repackdb which replaces clusterdb and also allows us to
> remove the FULL options from vacuumdb.
I don't think we should remove clusterdb, to avoid breaking any scripts
that work today. As you say, I would create the standalone pg_repackdb
to do what we need it to do (namely: run the REPACK commands) and leave
vacuumdb and clusterdb alone. Removing the obsolete commands and
options can be done in a few years.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-21 22:06 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-22 09:40 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-08-22 20:32 ` Euler Taveira <euler@eulerto.com>
2025-08-23 06:56 ` Re: Adding REPACK [concurrently] Michael Banck <mbanck@gmx.net>
0 siblings, 1 reply; 416+ messages in thread
From: Euler Taveira @ 2025-08-22 20:32 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; Robert Treat <rob@xzilla.net>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Fri, Aug 22, 2025, at 6:40 AM, Álvaro Herrera wrote:
> On 2025-Aug-21, Robert Treat wrote:
>
>> What's the plan for clusterdb? It seems like we'd ideally create a
>> stand alone pg_repackdb which replaces clusterdb and also allows us to
>> remove the FULL options from vacuumdb.
>
> I don't think we should remove clusterdb, to avoid breaking any scripts
> that work today. As you say, I would create the standalone pg_repackdb
> to do what we need it to do (namely: run the REPACK commands) and leave
> vacuumdb and clusterdb alone. Removing the obsolete commands and
> options can be done in a few years.
>
I would say that we need to plan the removal of these binaries (clusterdb and
vacuumdb). We can start with a warning into clusterdb saying they should use
pg_repackdb. In a few years, we can remove clusterdb. There were complaints
about binary names without a pg_ prefix in the past [1].
I don't think we need to keep vacuumdb. Packagers can keep a symlink (vacuumdb)
to pg_repackdb. We can add a similar warning message saying they should use
pg_repackdb if the symlink is used.
[1] https://www.postgresql.org/message-id/CAJgfmqXYYKXR%2BQUhEa3cq6pc8dV0Hu7QvOUccm7R0TkC%3DT-%2B%3DA%40...
--
Euler Taveira
EDB https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-21 22:06 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-22 09:40 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-22 20:32 ` Re: Adding REPACK [concurrently] Euler Taveira <euler@eulerto.com>
@ 2025-08-23 06:56 ` Michael Banck <mbanck@gmx.net>
2025-08-23 14:22 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
0 siblings, 1 reply; 416+ messages in thread
From: Michael Banck @ 2025-08-23 06:56 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: Álvaro Herrera <alvherre@kurilemu.de>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Hi,
On Fri, Aug 22, 2025 at 05:32:34PM -0300, Euler Taveira wrote:
> On Fri, Aug 22, 2025, at 6:40 AM, Álvaro Herrera wrote:
> > On 2025-Aug-21, Robert Treat wrote:
> >> What's the plan for clusterdb? It seems like we'd ideally create a
> >> stand alone pg_repackdb which replaces clusterdb and also allows us to
> >> remove the FULL options from vacuumdb.
> >
> > I don't think we should remove clusterdb, to avoid breaking any scripts
> > that work today. As you say, I would create the standalone pg_repackdb
> > to do what we need it to do (namely: run the REPACK commands) and leave
> > vacuumdb and clusterdb alone. Removing the obsolete commands and
> > options can be done in a few years.
>
> I would say that we need to plan the removal of these binaries (clusterdb and
> vacuumdb). We can start with a warning into clusterdb saying they should use
> pg_repackdb. In a few years, we can remove clusterdb. There were complaints
> about binary names without a pg_ prefix in the past [1].
Yeah.
> I don't think we need to keep vacuumdb. Packagers can keep a symlink (vacuumdb)
> to pg_repackdb. We can add a similar warning message saying they should use
> pg_repackdb if the symlink is used.
Unless pg_repack has the same (or a superset of) CLI and behaviour as
vacuumdb (I haven't checked, but doubt it?), I think replacing vacuumdb
with a symlink to pg_repack will lead to much more breakage in existing
scripts/automation than clusterdb, which I guess is used orders of
magnitude less frequently than vacumdb.
Michael
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-21 22:06 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-22 09:40 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-22 20:32 ` Re: Adding REPACK [concurrently] Euler Taveira <euler@eulerto.com>
2025-08-23 06:56 ` Re: Adding REPACK [concurrently] Michael Banck <mbanck@gmx.net>
@ 2025-08-23 14:22 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-08-25 16:03 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
0 siblings, 1 reply; 416+ messages in thread
From: Álvaro Herrera @ 2025-08-23 14:22 UTC (permalink / raw)
To: Michael Banck <mbanck@gmx.net>; Euler Taveira <euler@eulerto.com>; +Cc: Robert Treat <rob@xzilla.net>; pgsql-hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On 2025-08-23, Michael Banck wrote:
> On Fri, Aug 22, 2025 at 05:32:34PM -0300, Euler Taveira wrote:
>> I don't think we need to keep vacuumdb. Packagers can keep a symlink (vacuumdb)
>> to pg_repackdb. We can add a similar warning message saying they should use
>> pg_repackdb if the symlink is used.
>
> Unless pg_repack has the same (or a superset of) CLI and behaviour as
> vacuumdb (I haven't checked, but doubt it?), I think replacing vacuumdb
> with a symlink to pg_repack will lead to much more breakage in existing
> scripts/automation than clusterdb, which I guess is used orders of
> magnitude less frequently than vacumdb.
Yeah, I completely disagree with the idea of getting rid of vacuumdb. We can, maybe, in a distant future, get rid of the --full option to vacuumdb. But the rest of the vacuumdb behavior must stay, I think, because REPACK is not VACUUM — it is only VACUUM FULL. And we want to make that distinction very clear.
We can also, in a few years, get rid of clusterdb. But I don't think we need to deprecate it just yet.
--
Álvaro Herrera
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-21 22:06 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-22 09:40 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-22 20:32 ` Re: Adding REPACK [concurrently] Euler Taveira <euler@eulerto.com>
2025-08-23 06:56 ` Re: Adding REPACK [concurrently] Michael Banck <mbanck@gmx.net>
2025-08-23 14:22 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-08-25 16:03 ` Robert Treat <rob@xzilla.net>
0 siblings, 0 replies; 416+ messages in thread
From: Robert Treat @ 2025-08-25 16:03 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; +Cc: Michael Banck <mbanck@gmx.net>; Euler Taveira <euler@eulerto.com>; pgsql-hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Sat, Aug 23, 2025 at 10:23 AM Álvaro Herrera <alvherre@kurilemu.de> wrote:
> On 2025-08-23, Michael Banck wrote:
> > On Fri, Aug 22, 2025 at 05:32:34PM -0300, Euler Taveira wrote:
>
> >> I don't think we need to keep vacuumdb. Packagers can keep a symlink (vacuumdb)
> >> to pg_repackdb. We can add a similar warning message saying they should use
> >> pg_repackdb if the symlink is used.
> >
> > Unless pg_repack has the same (or a superset of) CLI and behaviour as
> > vacuumdb (I haven't checked, but doubt it?), I think replacing vacuumdb
> > with a symlink to pg_repack will lead to much more breakage in existing
> > scripts/automation than clusterdb, which I guess is used orders of
> > magnitude less frequently than vacumdb.
>
> Yeah, I completely disagree with the idea of getting rid of vacuumdb. We can, maybe, in a distant future, get rid of the --full option to vacuumdb. But the rest of the vacuumdb behavior must stay, I think, because REPACK is not VACUUM — it is only VACUUM FULL. And we want to make that distinction very clear.
>
Or to put it the other way, VACUUM FULL is not really VACUUM either,
it is really a form of "repack".
> We can also, in a few years, get rid of clusterdb. But I don't think we need to deprecate it just yet.
>
Yeah, ISTM the long term goal should be two binaries, one of which
manages aspects of clustering/repacking type of activities, and one
which manages vacuum type activities. I don't think that's different
that what Alvaro is proposing, FWIW my original question was about
confirming that was the end goal, but also trying to understand the
coordination of when these changes would take place, because the
changes to the code, changes to the SQL commands and their docs, and
changes to the command line tools, seem to be working at different
cadences. Which can be fine if it's on purpose, but maybe needs to be
tightened up if not; for example, the current patchset doesn't make
any changes to clusterdb, which one might expect to emit a warning
about being deprecated in favor of pg_repackdb, if not just a complete
punting to use pg_repackdb instead.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-30 17:50 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
6 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-30 17:50 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; +Cc: Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Hello,
Here's v19 of this patchset. This is mostly Antonin's v18. I added a
preparatory v19-0001 commit, which splits vacuumdb.c to create a new
file, vacuuming.c (and its header file vacuuming.h). If you look at it
under 'git show --color-moved=zebra' you should notice that most of it
is just code movement; there's hardly any code changes.
v19-0002 has absorbed Antonin's v18-0005 (the pg_repackdb binary)
together with the introduction of the REPACK command proper; but instead
of using a symlink, I just created a separate pg_repackdb.c source file
for it and we compile that small new source file with vacuuming.c to
create a regular binary. BTW the meson.build changes look somewhat
duplicative; maybe there's a less dumb way to go about this. (For
instance, maybe just have libscripts.a include vacuuming.o, though it's
not used by any of the other programs in that subdir.)
I'm not wedded to the name "vacuuming.c"; happy to take suggestions.
After 0002, the pg_repackdb utility should be ready to take clusterdb's
place, and also vacuumdb --full, with one gotcha: if you try to use
pg_repackdb with an older server version, it will fail, claiming that
REPACK is not supported. This is not ideal. Instead, we should make it
run VACUUM FULL (or CLUSTER); so if you have a fleet including older
servers you can use the new utils there too.
All the logic for vacuumdb to select tables to operate on has been moved
to vacuuming.c verbatim. This means this logic applies to pg_repackdb
as well. As long as you stick to repacking a single table this is okay
(read: it won't be used at all), but if you want to use parallel mode
(say to process multiple schemas), we might need to change it. For the
same reason, I think we should add an option to it (--index[=indexname])
to select whether to use the USING INDEX clause or not, and optionally
indicate which index to use; right now there's no way to select which
logic (cluster's or vacuum full's) to use.
Then v19-0003 through v19-0005 are Antonin's subsequent patches to add
the CONCURRENTLY option; I have not reviewed these at all, so I'm
including them here just for completion. I also included v18-0006 as
posted by Mihail previously, though I have little faith that we're going
to include it in this release.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Pensar que el espectro que vemos es ilusorio no lo despoja de espanto,
sólo le suma el nuevo terror de la locura" (Perelandra, C.S. Lewis)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-31 12:09 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-23 15:51 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-31 12:09 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; +Cc: Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Apparently I mismerged src/bin/scripts/meson.build. This v20 is
identical to v19, where that mistake has been corrected.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
Al principio era UNIX, y UNIX habló y dijo: "Hello world\n".
No dijo "Hello New Jersey\n", ni "Hello USA\n".
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-08-31 15:29 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-31 17:43 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-01 05:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-08-31 15:29 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Hello!
I started an attempt to make a "lightweight" MVCC-safe prototype and
stuck into the "it is not working" issue.
After some debugging I realized Antonin's variant (catalog-mode based)
seems to be broken also...
And after a few more hours I realized non-MVCC is broken as well :)
This is a patch with a test to reproduce the issue related to repack +
concurrent modifications.
Seems like some updates may be lost.
I hope the patch logic is clear - but feel free to ask if not.
Best regards,
Mikhail.
Attachments:
[application/octet-stream] v22-0002-Add-stress-tests-for-concurrent-index-builds.patch (9.1K, ../../CADzfLwWJqoG6uPt+HywKOFjXhqSbfCr+VXpfio9YQ6yqQaihPA@mail.gmail.com/2-v22-0002-Add-stress-tests-for-concurrent-index-builds.patch)
download | inline diff:
From c7424f44a086433d2eff6153476e0fd0c6b5b576 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sat, 30 Nov 2024 16:24:20 +0100
Subject: [PATCH v22 02/12] Add stress tests for concurrent index builds
Introduce stress tests for concurrent index operations:
- test concurrent inserts/updates during CREATE/REINDEX INDEX CONCURRENTLY
- cover various index types (btree, gin, gist, brin, hash, spgist)
- test unique and non-unique indexes
- test with expressions and predicates
- test both parallel and non-parallel operations
These tests verify the behavior of the following commits.
---
src/bin/pg_amcheck/meson.build | 1 +
src/bin/pg_amcheck/t/006_cic.pl | 223 ++++++++++++++++++++++++++++++++
2 files changed, 224 insertions(+)
create mode 100644 src/bin/pg_amcheck/t/006_cic.pl
diff --git a/src/bin/pg_amcheck/meson.build b/src/bin/pg_amcheck/meson.build
index 316ea0d40b8..7df15435fbb 100644
--- a/src/bin/pg_amcheck/meson.build
+++ b/src/bin/pg_amcheck/meson.build
@@ -28,6 +28,7 @@ tests += {
't/003_check.pl',
't/004_verify_heapam.pl',
't/005_opclass_damage.pl',
+ 't/006_cic.pl',
],
},
}
diff --git a/src/bin/pg_amcheck/t/006_cic.pl b/src/bin/pg_amcheck/t/006_cic.pl
new file mode 100644
index 00000000000..2aad0e8daa8
--- /dev/null
+++ b/src/bin/pg_amcheck/t/006_cic.pl
@@ -0,0 +1,223 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+Test::More->builder->todo_start('filesystem bug')
+ if PostgreSQL::Test::Utils::has_wal_read_bug;
+
+my ($node, $result);
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+$node->safe_psql('postgres', q(CREATE TABLE tbl(i int primary key,
+ c1 money default 0, c2 money default 0,
+ c3 money default 0, updated_at timestamp,
+ ia int4[], p point)));
+$node->safe_psql('postgres', q(CREATE INDEX CONCURRENTLY idx ON tbl(i, updated_at);));
+# create sequence
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE in_row_rebuild START 1 INCREMENT 1;));
+$node->safe_psql('postgres', q(SELECT nextval('in_row_rebuild');));
+
+# Create helper functions for predicate tests
+$node->safe_psql('postgres', q(
+ CREATE FUNCTION predicate_stable() RETURNS bool IMMUTABLE
+ LANGUAGE plpgsql AS $$
+ BEGIN
+ EXECUTE 'SELECT txid_current()';
+ RETURN true;
+ END; $$;
+));
+
+$node->safe_psql('postgres', q(
+ CREATE FUNCTION predicate_const(integer) RETURNS bool IMMUTABLE
+ LANGUAGE plpgsql AS $$
+ BEGIN
+ RETURN MOD($1, 2) = 0;
+ END; $$;
+));
+
+# Run CIC/RIC in different options concurrently with upserts
+$node->pgbench(
+ '--no-vacuum --client=30 --jobs=4 --exit-on-abort --transactions=2500',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY',
+ {
+ 'concurrent_ops' => q(
+ SET debug_parallel_query = off; -- this is because predicate_stable implementation
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \set variant random(0, 5)
+ \set parallels random(0, 4)
+ \if :last_value < 3
+ ALTER TABLE tbl SET (parallel_workers=:parallels);
+ \if :variant = 0
+ CREATE INDEX CONCURRENTLY new_idx ON tbl(i, updated_at);
+ \elif :variant = 1
+ CREATE INDEX CONCURRENTLY new_idx ON tbl(i, updated_at) WHERE predicate_stable();
+ \elif :variant = 2
+ CREATE INDEX CONCURRENTLY new_idx ON tbl(i, updated_at) WHERE MOD(i, 2) = 0;
+ \elif :variant = 3
+ CREATE INDEX CONCURRENTLY new_idx ON tbl(i, updated_at) WHERE predicate_const(i);
+ \elif :variant = 4
+ CREATE INDEX CONCURRENTLY new_idx ON tbl(predicate_const(i));
+ \elif :variant = 5
+ CREATE INDEX CONCURRENTLY new_idx ON tbl(i, predicate_const(i), updated_at) WHERE predicate_const(i);
+ \endif
+ \sleep 10 ms
+ SELECT bt_index_check('new_idx', heapallindexed => true, checkunique => true);
+ REINDEX INDEX CONCURRENTLY new_idx;
+ \sleep 10 ms
+ SELECT bt_index_check('new_idx', heapallindexed => true, checkunique => true);
+ DROP INDEX CONCURRENTLY new_idx;
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1000, 100000)
+ BEGIN;
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ SELECT setval('in_row_rebuild', 1);
+ COMMIT;
+ \endif
+ )
+ });
+
+$node->safe_psql('postgres', q(TRUNCATE TABLE tbl;));
+
+# Run CIC/RIC for unique index concurrently with upserts
+$node->pgbench(
+ '--no-vacuum --client=30 --jobs=4 --exit-on-abort --transactions=2500',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY for unique BTREE',
+ {
+ 'concurrent_ops_unique_idx' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \set parallels random(0, 4)
+ \if :last_value < 3
+ ALTER TABLE tbl SET (parallel_workers=:parallels);
+ CREATE UNIQUE INDEX CONCURRENTLY new_idx ON tbl(i);
+ \sleep 10 ms
+ SELECT bt_index_check('new_idx', heapallindexed => true, checkunique => true);
+ REINDEX INDEX CONCURRENTLY new_idx;
+ \sleep 10 ms
+ SELECT bt_index_check('new_idx', heapallindexed => true, checkunique => true);
+ DROP INDEX CONCURRENTLY new_idx;
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1, power(10, random(1, 5)))
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ SELECT setval('in_row_rebuild', 1);
+ \endif
+ )
+ });
+
+$node->safe_psql('postgres', q(TRUNCATE TABLE tbl;));
+
+# Run CIC/RIC for GIN with upserts
+$node->pgbench(
+ '--no-vacuum --client=30 --jobs=4 --exit-on-abort --transactions=2500',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY for GIN/GIST/BRIN/HASH/SPGIST',
+ {
+ 'concurrent_ops_gin_idx' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \set parallels random(0, 4)
+ \if :last_value < 3
+ ALTER TABLE tbl SET (parallel_workers=:parallels);
+ CREATE INDEX CONCURRENTLY new_idx ON tbl USING GIN (ia);
+ \sleep 10 ms
+ SELECT gin_index_check('new_idx');
+ REINDEX INDEX CONCURRENTLY new_idx;
+ \sleep 10 ms
+ SELECT gin_index_check('new_idx');
+ DROP INDEX CONCURRENTLY new_idx;
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1, power(10, random(1, 5)))
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ SELECT setval('in_row_rebuild', 1);
+ \endif
+ )
+ });
+
+$node->safe_psql('postgres', q(TRUNCATE TABLE tbl;));
+
+# Run CIC/RIC for GIST/BRIN/HASH/SPGIST index concurrently with upserts
+$node->pgbench(
+ '--no-vacuum --client=30 --jobs=4 --exit-on-abort --transactions=2500',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY for GIN/GIST/BRIN/HASH/SPGIST',
+ {
+ 'concurrent_ops_other_idx' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \set parallels random(0, 4)
+ \if :last_value < 3
+ ALTER TABLE tbl SET (parallel_workers=:parallels);
+ \set variant random(0, 3)
+ \if :variant = 0
+ CREATE INDEX CONCURRENTLY new_idx ON tbl USING GIST (p);
+ \elif :variant = 1
+ CREATE INDEX CONCURRENTLY new_idx ON tbl USING BRIN (updated_at);
+ \elif :variant = 2
+ CREATE INDEX CONCURRENTLY new_idx ON tbl USING HASH (updated_at);
+ \elif :variant = 3
+ CREATE INDEX CONCURRENTLY new_idx ON tbl USING SPGIST (p);
+ \endif
+ \sleep 10 ms
+ REINDEX INDEX CONCURRENTLY new_idx;
+ \sleep 10 ms
+ DROP INDEX CONCURRENTLY new_idx;
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1, power(10, random(1, 5)))
+ INSERT INTO tbl VALUES(floor(random()*:num),0,0,0,now())
+ ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ SELECT setval('in_row_rebuild', 1);
+ \endif
+ )
+ });
+
+$node->stop;
+done_testing();
\ No newline at end of file
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-08-31 17:43 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-01 13:00 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2025-08-31 17:43 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
On 2025-Aug-31, Mihail Nikalayeu wrote:
> I started an attempt to make a "lightweight" MVCC-safe prototype and
> stuck into the "it is not working" issue.
> After some debugging I realized Antonin's variant (catalog-mode based)
> seems to be broken also...
>
> And after a few more hours I realized non-MVCC is broken as well :)
Ugh. Well, obviously we need to get this fixed if we want CONCURRENTLY
at all :-)
Please don't post patches that aren't the commitfest item's main patch
as attachment with .patch extension. This confuses the CFbot into
thinking your patch is the patch-of-record (which it isn't) and reports
that the patch fails CI. See here:
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F5117
(For the same reason, it isn't useful to number them as if they were
part of the patch series).
If you want to post secondary patches, please rename them to end in
something like .txt or .nocfbot or whatever. See here:
https://wiki.postgresql.org/wiki/Cfbot#Which_attachments_are_considered_to_be_patches?
Thanks for your interest in this topic,
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-31 17:43 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-09-01 13:00 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-09-01 13:00 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Hello, Álvaro!
Alvaro Herrera <alvherre@alvh.no-ip.org>:
> If you want to post secondary patches, please rename them to end in
> something like .txt or .nocfbot or whatever. See here:
> https://wiki.postgresql.org/wiki/Cfbot#Which_attachments_are_considered_to_be_patches?
Sorry, I missed that.
But now it is possible to send ".patch" without changing the extension [0].
> It also ignores any files that start with "nocfbot".
[0]: https://discord.com/channels/1258108670710124574/1328362897189113867/1412021226528051250
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-09-01 05:12 ` Antonin Houska <ah@cybertec.at>
2025-09-01 09:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-09-01 05:12 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hello!
>
> I started an attempt to make a "lightweight" MVCC-safe prototype and
> stuck into the "it is not working" issue.
> After some debugging I realized Antonin's variant (catalog-mode based)
> seems to be broken also...
>
> And after a few more hours I realized non-MVCC is broken as well :)
>
> This is a patch with a test to reproduce the issue related to repack +
> concurrent modifications.
> Seems like some updates may be lost.
>
> I hope the patch logic is clear - but feel free to ask if not.
Are you sure the test is complete? I see no occurrence of the REPACK command
in it.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 05:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-09-01 09:06 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 15:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-09-01 09:06 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Hello!
Antonin Houska <ah@cybertec.at>:
> Are you sure the test is complete? I see no occurrence of the REPACK command
> in it.
Oops, send invalid file. The correct one in attachment.
Attachments:
[application/octet-stream] Add_test_for_REPACK_CONCURRENTLY_with_concurrent_modifications.patch_ (3.8K, ../../CADzfLwU4kuHSPd9ty8DQpRNWRjX6rJJVVzzWvT4+MEoTyyGtDg@mail.gmail.com/2-Add_test_for_REPACK_CONCURRENTLY_with_concurrent_modifications.patch_)
download
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 05:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-01 09:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-09-01 15:30 ` Antonin Houska <ah@cybertec.at>
2025-09-02 10:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-09-01 15:30 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Antonin Houska <ah@cybertec.at>:
> > Are you sure the test is complete? I see no occurrence of the REPACK command
> > in it.
> Oops, send invalid file. The correct one in attachment.
Thanks!
The problem was that when removing the original "preserve visibility patch"
v12-0005 [1] from the series, I forgot to change the value of
'need_full_snapshot' argument of CreateInitDecodingContext().
v12 and earlier treated the repacked table like system catalog, so it was
o.k. to pass need_full_snapshot=false. However, it must be true now, otherwise
the snapshot created for the initial copy does not see commits of transactions
that do not change regular catalogs.
The fix is as simple as
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index f481a3cec6d..7866ac01278 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -502,6 +502,7 @@ SnapBuildInitialSnapshotForRepack(SnapBuild *builder)
StringInfo buf = makeStringInfo();
Assert(builder->state == SNAPBUILD_CONSISTENT);
+ Assert(builder->building_full_snapshot);
snap = SnapBuildBuildSnapshot(builder);
I'll apply it to the next version of the "Add CONCURRENTLY option to REPACK
command" patch.
[1] https://www.postgresql.org/message-id/flat/CAFj8pRDK89FtY_yyGw7-MW-zTaHOCY4m6qfLRittdoPocz+dMQ@mail....
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 05:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-01 09:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 15:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-09-02 10:44 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-03 09:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-09-02 10:44 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Hello!
Antonin Houska <ah@cybertec.at>:
> I'll apply it to the next version of the "Add CONCURRENTLY option to REPACK
> command" patch.
I have added it to the v21 patchset.
Also, I’ve updated the MVCC-safe patch:
* it uses the "XactLockTableWait before replay + SnapshotSelf" approach from [0]
* it includes a TAP test to ensure MVCC safety - not intended to be
committed in its current form (too heavy)
* documentation has been updated.
It's now much simpler and does not negatively impact performance. It
is less aggressive in tuple freezing, but can be updated to match the
non-MVCC-safe version if needed.
While testing MVCC-safe version with stress-tests
007_repack_concurrently_mvcc.pl I encountered some random crashes with
such logs:
25-09-02 12:24:40.039 CEST client backend[261907]
007_repack_concurrently_mvcc.pl ERROR: relcache reference
0x7715b9f394a8 is not owned by resource owner TopTransaction
2025-09-02 12:24:40.039 CEST client backend[261907]
007_repack_concurrently_mvcc.pl STATEMENT: REPACK (CONCURRENTLY) tbl1
USING INDEX tbl1_pkey;
TRAP: failed Assert("rel->rd_refcnt > 0"), File:
"../src/backend/utils/cache/relcache.c", Line: 6992, PID: 261907
postgres: CIC_test: nkey postgres [local]
REPACK(ExceptionalCondition+0xbe)[0x5b7ac41d79f9]
postgres: CIC_test: nkey postgres [local] REPACK(+0x852d2e)[0x5b7ac41cbd2e]
postgres: CIC_test: nkey postgres [local] REPACK(+0x8aa4a6)[0x5b7ac42234a6]
postgres: CIC_test: nkey postgres [local] REPACK(+0x8aad3b)[0x5b7ac4223d3b]
postgres: CIC_test: nkey postgres [local] REPACK(+0x8aac69)[0x5b7ac4223c69]
postgres: CIC_test: nkey postgres [local]
REPACK(ResourceOwnerRelease+0x32)[0x5b7ac4223c26]
postgres: CIC_test: nkey postgres [local] REPACK(+0x1f43bf)[0x5b7ac3b6d3bf]
postgres: CIC_test: nkey postgres [local] REPACK(+0x1f4dfa)[0x5b7ac3b6ddfa]
postgres: CIC_test: nkey postgres [local]
REPACK(AbortCurrentTransaction+0xe)[0x5b7ac3b6dd6b]
postgres: CIC_test: nkey postgres [local]
REPACK(PostgresMain+0x57d)[0x5b7ac3fd7238]
postgres: CIC_test: nkey postgres [local] REPACK(+0x654102)[0x5b7ac3fcd102]
postgres: CIC_test: nkey postgres [local]
REPACK(postmaster_child_launch+0x191)[0x5b7ac3eceb7a]
postgres: CIC_test: nkey postgres [local] REPACK(+0x55c8c1)[0x5b7ac3ed58c1]
postgres: CIC_test: nkey postgres [local] REPACK(+0x559d1e)[0x5b7ac3ed2d1e]
postgres: CIC_test: nkey postgres [local]
REPACK(PostmasterMain+0x168a)[0x5b7ac3ed25f8]
postgres: CIC_test: nkey postgres [local] REPACK(main+0x3a1)[0x5b7ac3da2bd6]
/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7715b962a1ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7715b962a28b]
This time I was clever and tried to attempt to reproduce the issue on
a non-MVCC safe version at first - and it is reproducible.
Just comment \if :p_t1 != :p_t2 (and its internals, because they
catching non-mvcc behaviour which is expected without 0006 patch); and
set
'--no-vacuum --client=30 --jobs=4 --exit-on-abort --transactions=25000'
It takes about a minute on my PC to get the crash.
[0]: https://www.postgresql.org/message-id/flat/CADzfLwXCTXNdxK-XGTKmObvT%3D_QnaCviwgrcGtG9chsj5sYzrg%40m...
Best regards,
Mikhail.
Attachments:
[application/octet-stream] v21-0006-Preserve-visibility-information-of-the-concurren.patch (30.5K, ../../CADzfLwUgPMLiFkXRnk97ugPqkDfsNJ3TRdw9gjJM=8WB4_nXwQ@mail.gmail.com/2-v21-0006-Preserve-visibility-information-of-the-concurren.patch)
download | inline diff:
From 946862e2a4dbfd91ac6802c2e8da104dce81c43a Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Tue, 2 Sep 2025 11:30:55 +0200
Subject: [PATCH v21 6/6] Preserve visibility information of the concurrent
data changes.
As explained in the commit message of the preceding patch of the series, the
data changes done by applications while REPACK CONCURRENTLY is copying the
table contents to a new file are decoded from WAL and eventually also applied
to the new file. To reduce the complexity a little bit, the preceding patch
uses the current transaction (i.e. transaction opened by the REPACK command)
to execute those INSERT, UPDATE and DELETE commands.
However, REPACK is not expected to change visibility of tuples. Therefore,
this patch fixes the handling of the "concurrent data changes". It ensures
that tuples written into the new table have the same XID and command ID (CID)
as they had in the old table.
To "replay" an UPDATE or DELETE command on the new table, we use SnapshotSelf to find the last alive version of tuple and update with stamp with xid of original transaction. It is safe because:
* all transactions we replaying are committed
* apply worker working without any concurrent modifiers of the table
As long as we preserve the tuple visibility information (which includes XID),
it's important to avoid logical decoding of the WAL generated by DMLs on the
new table: the logical decoding subsystem probably does not expect that the
incoming WAL records contain XIDs of an already decoded transactions. (And of
course, repeated decoding would be wasted effort.)
Author: Antonin Houska <ah@cybertec.at> with changes from Mikhail Nikalayeu <mihailnikalayeu@gmail.com
---
contrib/amcheck/meson.build | 1 +
.../amcheck/t/007_repack_concurrently_mvcc.pl | 113 ++++++++++++++++++
doc/src/sgml/mvcc.sgml | 12 +-
doc/src/sgml/ref/repack.sgml | 9 --
src/backend/access/common/toast_internals.c | 3 +-
src/backend/access/heap/heapam.c | 46 ++++---
src/backend/access/heap/heapam_handler.c | 24 ++--
src/backend/commands/cluster.c | 85 +++++++++----
.../pgoutput_repack/pgoutput_repack.c | 18 +--
src/include/access/heapam.h | 12 +-
src/include/commands/cluster.h | 2 +
.../injection_points/specs/repack.spec | 4 -
12 files changed, 249 insertions(+), 80 deletions(-)
create mode 100644 contrib/amcheck/t/007_repack_concurrently_mvcc.pl
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index 1f0c347ed54..d07d6ed3f0c 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -50,6 +50,7 @@ tests += {
't/004_verify_nbtree_unique.pl',
't/005_pitr.pl',
't/006_verify_gin.pl',
+ 't/007_repack_concurrently_mvcc.pl',
],
},
}
diff --git a/contrib/amcheck/t/007_repack_concurrently_mvcc.pl b/contrib/amcheck/t/007_repack_concurrently_mvcc.pl
new file mode 100644
index 00000000000..a83fd5b8141
--- /dev/null
+++ b/contrib/amcheck/t/007_repack_concurrently_mvcc.pl
@@ -0,0 +1,113 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+));
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl1(i int PRIMARY KEY, j int)));
+$node->safe_psql('postgres', q(CREATE TABLE tbl2(i int PRIMARY KEY, j int)));
+
+
+# Insert 100 rows into tbl1
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl1 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+# Insert 100 rows into tbl2
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl2 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+
+# Insert 100 rows into tbl1
+$node->safe_psql('postgres', q(
+ CREATE OR REPLACE FUNCTION log_raise(i int, j1 int, j2 int) RETURNS VOID AS $$
+ BEGIN
+ RAISE NOTICE 'ERROR i=% j1=% j2=%', i, j1, j2;
+ END;$$ LANGUAGE plpgsql;
+));
+
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE in_row_rebuild START 1 INCREMENT 1;));
+$node->safe_psql('postgres', q(SELECT nextval('in_row_rebuild');));
+
+
+$node->pgbench(
+'--no-vacuum --client=10 --jobs=4 --exit-on-abort --transactions=2500',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY',
+{
+ 'concurrent_ops' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \if :last_value = 2
+ REPACK (CONCURRENTLY) tbl1 USING INDEX tbl1_pkey;
+ \sleep 10 ms
+ REPACK (CONCURRENTLY) tbl2 USING INDEX tbl2_pkey;
+ \sleep 10 ms
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1, 100)
+ BEGIN;
+ UPDATE tbl1 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 4 WHERE i = :num;
+ \sleep 1 ms
+
+ UPDATE tbl2 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 4 WHERE i = :num;
+
+ COMMIT;
+ SELECT setval('in_row_rebuild', 1);
+
+ BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+ SELECT COALESCE(SUM(j), 0) AS t1 FROM tbl1 WHERE i = :num \gset p_
+ \sleep 10 ms
+ SELECT COALESCE(SUM(j), 0) AS t2 FROM tbl2 WHERE i = :num \gset p_
+ \if :p_t1 != :p_t2
+ COMMIT;
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ \sleep 10 ms
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ SELECT (:p_t1 + :p_t2) / 0;
+ \endif
+
+ COMMIT;
+ \endif
+ )
+});
+
+$node->stop;
+done_testing();
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
index 0f5c34af542..049ee75a4ba 100644
--- a/doc/src/sgml/mvcc.sgml
+++ b/doc/src/sgml/mvcc.sgml
@@ -1833,17 +1833,15 @@ SELECT pg_advisory_lock(q.id) FROM
<title>Caveats</title>
<para>
- Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the
- table-rewriting forms of <link linkend="sql-altertable"><command>ALTER
- TABLE</command></link> and <command>REPACK</command> with
- the <literal>CONCURRENTLY</literal> option, are not
+ Some DDL commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link> and the
+ table-rewriting forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link>, are not
MVCC-safe. This means that after the truncation or rewrite commits, the
table will appear empty to concurrent transactions, if they are using a
- snapshot taken before the command committed. This will only be an
+ snapshot taken before the DDL command committed. This will only be an
issue for a transaction that did not access the table in question
- before the command started — any transaction that has done so
+ before the DDL command started — any transaction that has done so
would hold at least an <literal>ACCESS SHARE</literal> table lock,
- which would block the truncating or rewriting command until that transaction completes.
+ which would block the DDL command until that transaction completes.
So these commands will not cause any apparent inconsistency in the
table contents for successive queries on the target table, but they
could cause visible inconsistency between the contents of the target
diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml
index ff5ce48de55..271923a5a60 100644
--- a/doc/src/sgml/ref/repack.sgml
+++ b/doc/src/sgml/ref/repack.sgml
@@ -292,15 +292,6 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
</listitem>
</itemizedlist>
</para>
-
- <warning>
- <para>
- <command>REPACK</command> with the <literal>CONCURRENTLY</literal>
- option is not MVCC-safe, see <xref linkend="mvcc-caveats"/> for
- details.
- </para>
- </warning>
-
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index a1d0eed8953..586eb42a137 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -320,7 +320,8 @@ toast_save_datum(Relation rel, Datum value,
memcpy(VARDATA(&chunk_data), data_p, chunk_size);
toasttup = heap_form_tuple(toasttupDesc, t_values, t_isnull);
- heap_insert(toastrel, toasttup, mycid, options, NULL);
+ heap_insert(toastrel, toasttup, GetCurrentTransactionId(), mycid,
+ options, NULL);
/*
* Create the index entry. We cheat a little here by not using
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index f9a4fe3faed..45da5902de0 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2070,7 +2070,7 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
/*
* heap_insert - insert tuple into a heap
*
- * The new tuple is stamped with current transaction ID and the specified
+ * The new tuple is stamped with specified transaction ID and the specified
* command ID.
*
* See table_tuple_insert for comments about most of the input flags, except
@@ -2086,15 +2086,16 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
* reflected into *tup.
*/
void
-heap_insert(Relation relation, HeapTuple tup, CommandId cid,
- int options, BulkInsertState bistate)
+heap_insert(Relation relation, HeapTuple tup, TransactionId xid,
+ CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
bool all_visible_cleared = false;
+ Assert(TransactionIdIsValid(xid));
+
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(tup->t_data) <=
RelationGetNumberOfAttributes(relation));
@@ -2176,8 +2177,15 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
/*
* If this is a catalog, we need to transmit combo CIDs to properly
* decode, so log that as well.
+ *
+ * HEAP_INSERT_NO_LOGICAL should be set when applying data changes
+ * done by other transactions during REPACK CONCURRENTLY. In such a
+ * case, the insertion should not be decoded at all - see
+ * heap_decode(). (It's also set by raw_heap_insert() for TOAST, but
+ * TOAST does not pass this test anyway.)
*/
- if (RelationIsAccessibleInLogicalDecoding(relation))
+ if ((options & HEAP_INSERT_NO_LOGICAL) == 0 &&
+ RelationIsAccessibleInLogicalDecoding(relation))
log_heap_new_cid(relation, heaptup);
/*
@@ -2723,7 +2731,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
void
simple_heap_insert(Relation relation, HeapTuple tup)
{
- heap_insert(relation, tup, GetCurrentCommandId(true), 0, NULL);
+ heap_insert(relation, tup, GetCurrentTransactionId(),
+ GetCurrentCommandId(true), 0, NULL);
}
/*
@@ -2780,11 +2789,11 @@ xmax_infomask_changed(uint16 new_infomask, uint16 old_infomask)
*/
TM_Result
heap_delete(Relation relation, ItemPointer tid,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, bool changingPart, bool wal_logical)
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
+ TM_FailureData *tmfd, bool changingPart,
+ bool wal_logical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
ItemId lp;
HeapTupleData tp;
Page page;
@@ -2801,6 +2810,7 @@ heap_delete(Relation relation, ItemPointer tid,
bool old_key_copied = false;
Assert(ItemPointerIsValid(tid));
+ Assert(TransactionIdIsValid(xid));
AssertHasSnapshotForToast(relation);
@@ -3217,7 +3227,7 @@ simple_heap_delete(Relation relation, ItemPointer tid)
TM_Result result;
TM_FailureData tmfd;
- result = heap_delete(relation, tid,
+ result = heap_delete(relation, tid, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, false, /* changingPart */
@@ -3260,12 +3270,11 @@ simple_heap_delete(Relation relation, ItemPointer tid)
*/
TM_Result
heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
+ TransactionId xid, CommandId cid, Snapshot crosscheck,
+ bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool wal_logical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
Bitmapset *key_attrs;
@@ -3305,6 +3314,7 @@ heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
infomask2_new_tuple;
Assert(ItemPointerIsValid(otid));
+ Assert(TransactionIdIsValid(xid));
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
@@ -4144,8 +4154,12 @@ l2:
/*
* For logical decoding we need combo CIDs to properly decode the
* catalog.
+ *
+ * Like in heap_insert(), visibility is unchanged when called from
+ * VACUUM FULL / CLUSTER.
*/
- if (RelationIsAccessibleInLogicalDecoding(relation))
+ if (wal_logical &&
+ RelationIsAccessibleInLogicalDecoding(relation))
{
log_heap_new_cid(relation, &oldtup);
log_heap_new_cid(relation, heaptup);
@@ -4511,7 +4525,7 @@ simple_heap_update(Relation relation, ItemPointer otid, HeapTuple tup,
TM_FailureData tmfd;
LockTupleMode lockmode;
- result = heap_update(relation, otid, tup,
+ result = heap_update(relation, otid, tup, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode, update_indexes,
@@ -5351,8 +5365,6 @@ compute_new_xmax_infomask(TransactionId xmax, uint16 old_infomask,
uint16 new_infomask,
new_infomask2;
- Assert(TransactionIdIsCurrentTransactionId(add_to_xmax));
-
l5:
new_infomask = 0;
new_infomask2 = 0;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d03084768e0..6733e5fdda6 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -253,7 +253,8 @@ heapam_tuple_insert(Relation relation, TupleTableSlot *slot, CommandId cid,
tuple->t_tableOid = slot->tts_tableOid;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -276,7 +277,8 @@ heapam_tuple_insert_speculative(Relation relation, TupleTableSlot *slot,
options |= HEAP_INSERT_SPECULATIVE;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -310,8 +312,8 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
* the storage itself is cleaning the dead tuples by itself, it is the
* time to call the index tuple deletion also.
*/
- return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart,
- true);
+ return heap_delete(relation, tid, GetCurrentTransactionId(), cid,
+ crosscheck, wait, tmfd, changingPart, true);
}
@@ -329,7 +331,8 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
+ result = heap_update(relation, otid, tuple, GetCurrentTransactionId(),
+ cid, crosscheck, wait,
tmfd, lockmode, update_indexes, true);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
@@ -2477,9 +2480,16 @@ reform_and_rewrite_tuple(HeapTuple tuple,
* flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps
* the relation files, it drops this relation, so no logical
* replication subscription should need the data.
+ *
+ * It is also crucial to stamp the new record with the exact same xid
+ * and cid, because the tuple must be visible to the snapshots of the
+ * concurrent transactions later.
*/
- heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true),
- HEAP_INSERT_NO_LOGICAL, NULL);
+ // TODO: looks like cid is not required
+ CommandId cid = HeapTupleHeaderGetRawCommandId(tuple->t_data);
+ TransactionId xid = HeapTupleHeaderGetXmin(tuple->t_data);
+
+ heap_insert(NewHeap, copiedTuple, xid, cid, HEAP_INSERT_NO_LOGICAL, NULL);
}
heap_freetuple(copiedTuple);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 61224a3adf2..936cb0ae429 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -55,6 +55,7 @@
#include "storage/ipc.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
+#include "storage/procarray.h"
#include "utils/acl.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -146,6 +147,7 @@ static void apply_concurrent_delete(Relation rel, HeapTuple tup_target,
ConcurrentChange *change);
static HeapTuple find_target_tuple(Relation rel, ScanKey key, int nkeys,
HeapTuple tup_key,
+ Snapshot snapshot,
IndexInsertState *iistate,
TupleTableSlot *ident_slot,
IndexScanDesc *scan_p);
@@ -1008,7 +1010,14 @@ rebuild_relation(RepackCommand cmd, bool usingindex,
/* The historic snapshot won't be needed anymore. */
if (snapshot)
+ {
+ TransactionId xmin = snapshot->xmin;
PopActiveSnapshot();
+ Assert(concurrent);
+ // TODO: seems like it not required: need to check SnapBuildInitialSnapshotForRepack
+ WaitForOlderSnapshots(xmin, false);
+ }
+
if (concurrent)
{
@@ -1299,30 +1308,35 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
* not to be aggressive about this.
*/
memset(¶ms, 0, sizeof(VacuumParams));
- vacuum_get_cutoffs(OldHeap, params, &cutoffs);
-
- /*
- * FreezeXid will become the table's new relfrozenxid, and that mustn't go
- * backwards, so take the max.
- */
+ if (!concurrent)
{
TransactionId relfrozenxid = OldHeap->rd_rel->relfrozenxid;
+ MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
+ vacuum_get_cutoffs(OldHeap, params, &cutoffs);
+ /*
+ * FreezeXid will become the table's new relfrozenxid, and that mustn't go
+ * backwards, so take the max.
+ */
if (TransactionIdIsValid(relfrozenxid) &&
TransactionIdPrecedes(cutoffs.FreezeLimit, relfrozenxid))
cutoffs.FreezeLimit = relfrozenxid;
- }
-
- /*
- * MultiXactCutoff, similarly, shouldn't go backwards either.
- */
- {
- MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
-
+ /*
+ * MultiXactCutoff, similarly, shouldn't go backwards either.
+ */
if (MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(cutoffs.MultiXactCutoff, relminmxid))
cutoffs.MultiXactCutoff = relminmxid;
}
+ else
+ {
+ /*
+ * In concurrent mode we reuse all the xmin/xmax,
+ * so just use current values for simplicity.
+ */
+ cutoffs.FreezeLimit = OldHeap->rd_rel->relfrozenxid;
+ cutoffs.MultiXactCutoff = OldHeap->rd_rel->relminmxid;
+ }
/*
* Decide whether to use an indexscan or seqscan-and-optional-sort to scan
@@ -2675,6 +2689,16 @@ apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
continue;
}
+ if (TransactionIdIsInProgress(change.xid))
+ {
+ /* xid is committed for sure because we got that update from reorderbuffer.
+ * but there is a possibility procarray is not yet updated and current backend still see it as
+ * in-progress. Let's wait for procarray to be updated. */
+ XactLockTableWait(change.xid, NULL, NULL, XLTW_None);
+ Assert(!TransactionIdIsInProgress(change.xid));
+ Assert(TransactionIdDidCommit(change.xid));
+ }
+
/*
* Extract the tuple from the change. The tuple is copied here because
* it might be assigned to 'tup_old', in which case it needs to
@@ -2712,9 +2736,13 @@ apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
}
/*
- * Find the tuple to be updated or deleted.
+ * Find the tuple to be updated or deleted using SnapshotSelf.
+ * That way we receive the last alive version in case of HOT chain.
+ * It is guaranteed there is no any non-yet committed, but updated version
+ * because we here replaying all-committed transactions without any concurrency
+ * involved.
*/
- tup_exist = find_target_tuple(rel, key, nkeys, tup_key,
+ tup_exist = find_target_tuple(rel, key, nkeys, tup_key, SnapshotSelf,
iistate, ident_slot, &ind_scan);
if (tup_exist == NULL)
elog(ERROR, "Failed to find target tuple");
@@ -2743,6 +2771,7 @@ apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
*/
if (change.kind != CHANGE_UPDATE_OLD)
{
+ // TODO: not sure it is required at all: we are replaying committed transactions stamping them with committed XID
CommandCounterIncrement();
UpdateActiveSnapshotCommandId();
}
@@ -2771,9 +2800,11 @@ apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup,
* Like simple_heap_insert(), but make sure that the INSERT is not
* logically decoded - see reform_and_rewrite_tuple() for more
* information.
+ *
+ * Use already committed xid to stamp the tuple.
*/
- heap_insert(rel, tup, GetCurrentCommandId(true), HEAP_INSERT_NO_LOGICAL,
- NULL);
+ heap_insert(rel, tup, change->xid, GetCurrentCommandId(true),
+ HEAP_INSERT_NO_LOGICAL, NULL);
/*
* Update indexes.
@@ -2781,6 +2812,7 @@ apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup,
* In case functions in the index need the active snapshot and caller
* hasn't set one.
*/
+ PushActiveSnapshot(GetLatestSnapshot());
ExecStoreHeapTuple(tup, index_slot, false);
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
@@ -2791,6 +2823,7 @@ apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup,
NIL, /* arbiterIndexes */
false /* onlySummarizing */
);
+ PopActiveSnapshot();
/*
* If recheck is required, it must have been preformed on the source
@@ -2819,9 +2852,11 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
*
* Do it like in simple_heap_update(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
res = heap_update(rel, &tup_target->t_self, tup,
- GetCurrentCommandId(true),
+ change->xid, GetCurrentCommandId(true),
InvalidSnapshot,
false, /* no wait - only we are doing changes */
&tmfd, &lockmode, &update_indexes,
@@ -2833,6 +2868,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
if (update_indexes != TU_None)
{
+ PushActiveSnapshot(GetLatestSnapshot());
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
iistate->estate,
@@ -2842,6 +2878,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
NIL, /* arbiterIndexes */
/* onlySummarizing */
update_indexes == TU_Summarizing);
+ PopActiveSnapshot();
list_free(recheck);
}
@@ -2860,9 +2897,11 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target,
*
* Do it like in simple_heap_delete(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
- res = heap_delete(rel, &tup_target->t_self, GetCurrentCommandId(true),
- InvalidSnapshot, false,
+ res = heap_delete(rel, &tup_target->t_self, change->xid,
+ GetCurrentCommandId(true), InvalidSnapshot, false,
&tmfd,
false, /* no wait - only we are doing changes */
false /* wal_logical */ );
@@ -2886,7 +2925,7 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target,
*/
static HeapTuple
find_target_tuple(Relation rel, ScanKey key, int nkeys, HeapTuple tup_key,
- IndexInsertState *iistate,
+ Snapshot snapshot, IndexInsertState *iistate,
TupleTableSlot *ident_slot, IndexScanDesc *scan_p)
{
IndexScanDesc scan;
@@ -2895,7 +2934,7 @@ find_target_tuple(Relation rel, ScanKey key, int nkeys, HeapTuple tup_key,
HeapTuple result = NULL;
/* XXX no instrumentation for now */
- scan = index_beginscan(rel, iistate->ident_index, GetActiveSnapshot(),
+ scan = index_beginscan(rel, iistate->ident_index, snapshot,
NULL, nkeys, 0);
*scan_p = scan;
index_rescan(scan, key, nkeys, NULL, 0);
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
index 687fbbc59bb..020ff7b7c80 100644
--- a/src/backend/replication/pgoutput_repack/pgoutput_repack.c
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -32,7 +32,8 @@ static void plugin_truncate(struct LogicalDecodingContext *ctx,
Relation relations[],
ReorderBufferChange *change);
static void store_change(LogicalDecodingContext *ctx,
- ConcurrentChangeKind kind, HeapTuple tuple);
+ ConcurrentChangeKind kind, HeapTuple tuple,
+ TransactionId xid);
void
_PG_output_plugin_init(OutputPluginCallbacks *cb)
@@ -124,7 +125,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (newtuple == NULL)
elog(ERROR, "Incomplete insert info.");
- store_change(ctx, CHANGE_INSERT, newtuple);
+ store_change(ctx, CHANGE_INSERT, newtuple, change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -141,9 +142,11 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
elog(ERROR, "Incomplete update info.");
if (oldtuple != NULL)
- store_change(ctx, CHANGE_UPDATE_OLD, oldtuple);
+ store_change(ctx, CHANGE_UPDATE_OLD, oldtuple,
+ change->txn->xid);
- store_change(ctx, CHANGE_UPDATE_NEW, newtuple);
+ store_change(ctx, CHANGE_UPDATE_NEW, newtuple,
+ change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_DELETE:
@@ -156,7 +159,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (oldtuple == NULL)
elog(ERROR, "Incomplete delete info.");
- store_change(ctx, CHANGE_DELETE, oldtuple);
+ store_change(ctx, CHANGE_DELETE, oldtuple, change->txn->xid);
}
break;
default:
@@ -190,13 +193,13 @@ plugin_truncate(struct LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (i == nrelations)
return;
- store_change(ctx, CHANGE_TRUNCATE, NULL);
+ store_change(ctx, CHANGE_TRUNCATE, NULL, InvalidTransactionId);
}
/* Store concurrent data change. */
static void
store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
- HeapTuple tuple)
+ HeapTuple tuple, TransactionId xid)
{
RepackDecodingState *dstate;
char *change_raw;
@@ -266,6 +269,7 @@ store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
dst = dst_start + SizeOfConcurrentChange;
memcpy(dst, tuple->t_data, tuple->t_len);
+ change.xid = xid;
/* The data has been copied. */
if (flattened)
pfree(tuple);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index b82dd17a966..981425f23b6 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -316,22 +316,24 @@ extern BulkInsertState GetBulkInsertState(void);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
-extern void heap_insert(Relation relation, HeapTuple tup, CommandId cid,
- int options, BulkInsertState bistate);
+extern void heap_insert(Relation relation, HeapTuple tup, TransactionId xid,
+ CommandId cid, int options, BulkInsertState bistate);
extern void heap_multi_insert(Relation relation, struct TupleTableSlot **slots,
int ntuples, CommandId cid, int options,
BulkInsertState bistate);
extern TM_Result heap_delete(Relation relation, ItemPointer tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid,
+ Snapshot crosscheck, bool wait,
struct TM_FailureData *tmfd, bool changingPart,
bool wal_logical);
extern void heap_finish_speculative(Relation relation, ItemPointer tid);
extern void heap_abort_speculative(Relation relation, ItemPointer tid);
extern TM_Result heap_update(Relation relation, ItemPointer otid,
- HeapTuple newtup,
+ HeapTuple newtup, TransactionId xid,
CommandId cid, Snapshot crosscheck, bool wait,
struct TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes, bool wal_logical);
+ TU_UpdateIndexes *update_indexes,
+ bool wal_logical);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
index 4a508c57a50..242f8da770a 100644
--- a/src/include/commands/cluster.h
+++ b/src/include/commands/cluster.h
@@ -61,6 +61,8 @@ typedef struct ConcurrentChange
/* See the enum above. */
ConcurrentChangeKind kind;
+ /* Transaction that changes the data. */
+ TransactionId xid;
/*
* The actual tuple.
*
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index 75850334986..3711a7c92b9 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -86,9 +86,6 @@ step change_new
# When applying concurrent data changes, we should see the effects of an
# in-progress subtransaction.
#
-# XXX Not sure this test is useful now - it was designed for the patch that
-# preserves tuple visibility and which therefore modifies
-# TransactionIdIsCurrentTransactionId().
step change_subxact1
{
BEGIN;
@@ -103,7 +100,6 @@ step change_subxact1
# When applying concurrent data changes, we should not see the effects of a
# rolled back subtransaction.
#
-# XXX Is this test useful? See above.
step change_subxact2
{
BEGIN;
--
2.43.0
[application/octet-stream] v21-0003-Refactor-index_concurrently_create_copy-for-use-.patch (4.1K, ../../CADzfLwUgPMLiFkXRnk97ugPqkDfsNJ3TRdw9gjJM=8WB4_nXwQ@mail.gmail.com/3-v21-0003-Refactor-index_concurrently_create_copy-for-use-.patch)
download | inline diff:
From 896f4fc90d128f0a8625f47b82b08eb0da145be7 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Mon, 11 Aug 2025 15:31:34 +0200
Subject: [PATCH v21 3/6] Refactor index_concurrently_create_copy() for use
with REPACK (CONCURRENTLY).
This patch moves the code to index_create_copy() and adds a "concurrently"
parameter so it can be used by REPACK (CONCURRENTLY).
With the CONCURRENTLY option, REPACK cannot simply swap the heap file and
rebuild its indexes. Instead, it needs to build a separate set of indexes
(including system catalog entries) *before* the actual swap, to reduce the
time AccessExclusiveLock needs to be held for.
---
src/backend/catalog/index.c | 36 ++++++++++++++++++++++++++++--------
src/include/catalog/index.h | 3 +++
2 files changed, 31 insertions(+), 8 deletions(-)
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 3063abff9a5..0dee1b1a9d8 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1290,15 +1290,31 @@ index_create(Relation heapRelation,
/*
* index_concurrently_create_copy
*
- * Create concurrently an index based on the definition of the one provided by
- * caller. The index is inserted into catalogs and needs to be built later
- * on. This is called during concurrent reindex processing.
- *
- * "tablespaceOid" is the tablespace to use for this index.
+ * Variant of index_create_copy(), called during concurrent reindex
+ * processing.
*/
Oid
index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId,
Oid tablespaceOid, const char *newName)
+{
+ return index_create_copy(heapRelation, oldIndexId, tablespaceOid, newName,
+ true);
+}
+
+/*
+ * index_create_copy
+ *
+ * Create an index based on the definition of the one provided by caller. The
+ * index is inserted into catalogs and needs to be built later on.
+ *
+ * "tablespaceOid" is the tablespace to use for this index.
+ *
+ * The actual implementation of index_concurrently_create_copy(), reusable for
+ * other purposes.
+ */
+Oid
+index_create_copy(Relation heapRelation, Oid oldIndexId, Oid tablespaceOid,
+ const char *newName, bool concurrently)
{
Relation indexRelation;
IndexInfo *oldInfo,
@@ -1317,6 +1333,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId,
List *indexColNames = NIL;
List *indexExprs = NIL;
List *indexPreds = NIL;
+ int flags = 0;
indexRelation = index_open(oldIndexId, RowExclusiveLock);
@@ -1325,9 +1342,9 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId,
/*
* Concurrent build of an index with exclusion constraints is not
- * supported.
+ * supported. If !concurrently, ii_ExclusinOps is currently not needed.
*/
- if (oldInfo->ii_ExclusionOps != NULL)
+ if (oldInfo->ii_ExclusionOps != NULL && concurrently)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("concurrent index creation for exclusion constraints is not supported")));
@@ -1435,6 +1452,9 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId,
stattargets[i].isnull = isnull;
}
+ if (concurrently)
+ flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT;
+
/*
* Now create the new index.
*
@@ -1458,7 +1478,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId,
indcoloptions->values,
stattargets,
reloptionsDatum,
- INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT,
+ flags,
0,
true, /* allow table to be a system catalog? */
false, /* is_internal? */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 4daa8bef5ee..063a891351a 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -99,6 +99,9 @@ extern Oid index_concurrently_create_copy(Relation heapRelation,
Oid oldIndexId,
Oid tablespaceOid,
const char *newName);
+extern Oid index_create_copy(Relation heapRelation, Oid oldIndexId,
+ Oid tablespaceOid, const char *newName,
+ bool concurrently);
extern void index_concurrently_build(Oid heapRelationId,
Oid indexRelationId);
--
2.43.0
[application/octet-stream] v21-0005-Add-CONCURRENTLY-option-to-REPACK-command.patch (147.2K, ../../CADzfLwUgPMLiFkXRnk97ugPqkDfsNJ3TRdw9gjJM=8WB4_nXwQ@mail.gmail.com/4-v21-0005-Add-CONCURRENTLY-option-to-REPACK-command.patch)
download | inline diff:
From a9411b077bc121215b230556be5a114d5effd847 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
Date: Sat, 30 Aug 2025 19:13:38 +0200
Subject: [PATCH v21 5/6] Add CONCURRENTLY option to REPACK command.
The REPACK command copies the relation data into a new file, creates new
indexes and eventually swaps the files. To make sure that the old file does
not change during the copying, the relation is locked in an exclusive mode,
which prevents applications from both reading and writing. (To keep the data
consistent, we'd only need to prevent the applications from writing, but even
reading needs to be blocked before we can swap the files - otherwise some
applications could continue using the old file. Since we should not request a
stronger lock without releasing the weaker one first, we acquire the exclusive
lock in the beginning and keep it till the end of the processing.)
This patch introduces an alternative workflow, which only requires the
exclusive lock when the relation (and index) files are being swapped.
(Supposedly, the swapping should be pretty fast.) On the other hand, when we
copy the data to the new file, we allow applications to read from the relation
and even to write to it.
First, we scan the relation using a "historic snapshot", and insert all the
tuples satisfying this snapshot into the new file.
Second, logical decoding is used to capture the data changes done by
applications during the copying (i.e. changes that do not satisfy the historic
snapshot mentioned above), and those are applied to the new file before we
acquire the exclusive lock that we need to swap the files. (Of course, more
data changes can take place while we are waiting for the lock - these will be
applied to the new file after we have acquired the lock, before we swap the
files.)
Since the logical decoding system, during its startup, waits until all the
transactions which already have XID assigned have finished, there is a risk of
deadlock if a transaction that already changed anything in the database tries
to acquire a conflicting lock on the table REPACK CONCURRENTLY is working
on. As an example, consider transaction running CREATE INDEX command on the
table that is being REPACKed CONCURRENTLY. On the other hand, DML commands
(INSERT, UPDATE, DELETE) are not a problem as their lock does not conflict
with REPACK CONCURRENTLY.
The current approach is that we accept the risk. If we tried to avoid it, it'd
be necessary to unlock the table before the logical decoding is setup and lock
it again afterwards. Such temporary unlocking would imply re-checking if the
table still meets all the requirements for REPACK CONCURRENTLY.
Like the existing implementation of REPACK, the variant with the CONCURRENTLY
option also requires an extra space for the new relation and index files
(which coexist with the old files for some time). In addition, the
CONCURRENTLY option might introduce a lag in releasing WAL segments for
archiving / recycling. This is due to the decoding of the data changes done by
applications concurrently. When copying the table contents into the new file,
we check the lag periodically. If it exceeds the size of a WAL segment, we
decode all the available WAL before resuming the copying. (Of course, the
changes are not applied until the whole table contents is copied.) A
background worker might be a better approach for the decoding - let's consider
implementing it in the future.
The WAL records produced by running DML commands on the new relation do not
contain enough information to be processed by the logical decoding system. All
we need from the new relation is the file (relfilenode), while the actual
relation is eventually dropped. Thus there is no point in replaying the DMLs
anywhere.
Author: Antonin Houska <ah@cybertec.at>
---
doc/src/sgml/monitoring.sgml | 37 +-
doc/src/sgml/mvcc.sgml | 12 +-
doc/src/sgml/ref/repack.sgml | 129 +-
src/Makefile | 1 +
src/backend/access/heap/heapam.c | 34 +-
src/backend/access/heap/heapam_handler.c | 219 ++-
src/backend/access/heap/rewriteheap.c | 6 +-
src/backend/access/transam/xact.c | 11 +-
src/backend/catalog/system_views.sql | 30 +-
src/backend/commands/cluster.c | 1677 +++++++++++++++--
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 1 +
src/backend/commands/vacuum.c | 12 +-
src/backend/meson.build | 1 +
src/backend/replication/logical/decode.c | 83 +
src/backend/replication/logical/snapbuild.c | 21 +
.../replication/pgoutput_repack/Makefile | 32 +
.../replication/pgoutput_repack/meson.build | 18 +
.../pgoutput_repack/pgoutput_repack.c | 288 +++
src/backend/storage/ipc/ipci.c | 1 +
.../storage/lmgr/generate-lwlocknames.pl | 2 +-
src/backend/utils/cache/relcache.c | 1 +
src/backend/utils/time/snapmgr.c | 3 +-
src/bin/psql/tab-complete.in.c | 25 +-
src/include/access/heapam.h | 9 +-
src/include/access/heapam_xlog.h | 2 +
src/include/access/tableam.h | 10 +
src/include/commands/cluster.h | 91 +-
src/include/commands/progress.h | 23 +-
src/include/replication/snapbuild.h | 1 +
src/include/storage/lockdefs.h | 4 +-
src/include/utils/snapmgr.h | 2 +
src/test/modules/injection_points/Makefile | 5 +-
.../injection_points/expected/repack.out | 113 ++
.../modules/injection_points/logical.conf | 1 +
src/test/modules/injection_points/meson.build | 4 +
.../injection_points/specs/repack.spec | 143 ++
src/test/regress/expected/rules.out | 29 +-
src/tools/pgindent/typedefs.list | 4 +
39 files changed, 2816 insertions(+), 271 deletions(-)
create mode 100644 src/backend/replication/pgoutput_repack/Makefile
create mode 100644 src/backend/replication/pgoutput_repack/meson.build
create mode 100644 src/backend/replication/pgoutput_repack/pgoutput_repack.c
create mode 100644 src/test/modules/injection_points/expected/repack.out
create mode 100644 src/test/modules/injection_points/logical.conf
create mode 100644 src/test/modules/injection_points/specs/repack.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12e103d319d..61c0197555f 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -6074,14 +6074,35 @@ FROM pg_stat_get_backend_idset() AS backendid;
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>heap_tuples_written</structfield> <type>bigint</type>
+ <structfield>heap_tuples_inserted</structfield> <type>bigint</type>
</para>
<para>
- Number of heap tuples written.
+ Number of heap tuples inserted.
This counter only advances when the phase is
<literal>seq scanning heap</literal>,
- <literal>index scanning heap</literal>
- or <literal>writing new heap</literal>.
+ <literal>index scanning heap</literal>,
+ <literal>writing new heap</literal>
+ or <literal>catch-up</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_tuples_updated</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of heap tuples updated.
+ This counter only advances when the phase is <literal>catch-up</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_tuples_deleted</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of heap tuples deleted.
+ This counter only advances when the phase is <literal>catch-up</literal>.
</para></entry>
</row>
@@ -6162,6 +6183,14 @@ FROM pg_stat_get_backend_idset() AS backendid;
<command>REPACK</command> is currently writing the new heap.
</entry>
</row>
+ <row>
+ <entry><literal>catch-up</literal></entry>
+ <entry>
+ <command>REPACK CONCURRENTLY</command> is currently processing the DML
+ commands that other transactions executed during any of the preceding
+ phase.
+ </entry>
+ </row>
<row>
<entry><literal>swapping relation files</literal></entry>
<entry>
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
index 049ee75a4ba..0f5c34af542 100644
--- a/doc/src/sgml/mvcc.sgml
+++ b/doc/src/sgml/mvcc.sgml
@@ -1833,15 +1833,17 @@ SELECT pg_advisory_lock(q.id) FROM
<title>Caveats</title>
<para>
- Some DDL commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link> and the
- table-rewriting forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link>, are not
+ Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the
+ table-rewriting forms of <link linkend="sql-altertable"><command>ALTER
+ TABLE</command></link> and <command>REPACK</command> with
+ the <literal>CONCURRENTLY</literal> option, are not
MVCC-safe. This means that after the truncation or rewrite commits, the
table will appear empty to concurrent transactions, if they are using a
- snapshot taken before the DDL command committed. This will only be an
+ snapshot taken before the command committed. This will only be an
issue for a transaction that did not access the table in question
- before the DDL command started — any transaction that has done so
+ before the command started — any transaction that has done so
would hold at least an <literal>ACCESS SHARE</literal> table lock,
- which would block the DDL command until that transaction completes.
+ which would block the truncating or rewriting command until that transaction completes.
So these commands will not cause any apparent inconsistency in the
table contents for successive queries on the target table, but they
could cause visible inconsistency between the contents of the target
diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml
index fd9d89f8aaa..ff5ce48de55 100644
--- a/doc/src/sgml/ref/repack.sgml
+++ b/doc/src/sgml/ref/repack.sgml
@@ -27,6 +27,7 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
ANALYSE | ANALYZE
+ CONCURRENTLY
</synopsis>
</refsynopsisdiv>
@@ -49,7 +50,8 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
processes every table and materialized view in the current database that
the current user has the <literal>MAINTAIN</literal> privilege on. This
form of <command>REPACK</command> cannot be executed inside a transaction
- block.
+ block. Also, this form is not allowed if
+ the <literal>CONCURRENTLY</literal> option is used.
</para>
<para>
@@ -62,7 +64,8 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
When a table is being repacked, an <literal>ACCESS EXCLUSIVE</literal> lock
is acquired on it. This prevents any other database operations (both reads
and writes) from operating on the table until the <command>REPACK</command>
- is finished.
+ is finished. If you want to keep the table accessible during the repacking,
+ consider using the <literal>CONCURRENTLY</literal> option.
</para>
<refsect2 id="sql-repack-notes-on-clustering" xreflabel="Notes on Clustering">
@@ -179,6 +182,128 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>CONCURRENTLY</literal></term>
+ <listitem>
+ <para>
+ Allow other transactions to use the table while it is being repacked.
+ </para>
+
+ <para>
+ Internally, <command>REPACK</command> copies the contents of the table
+ (ignoring dead tuples) into a new file, sorted by the specified index,
+ and also creates a new file for each index. Then it swaps the old and
+ new files for the table and all the indexes, and deletes the old
+ files. The <literal>ACCESS EXCLUSIVE</literal> lock is needed to make
+ sure that the old files do not change during the processing because the
+ changes would get lost due to the swap.
+ </para>
+
+ <para>
+ With the <literal>CONCURRENTLY</literal> option, the <literal>ACCESS
+ EXCLUSIVE</literal> lock is only acquired to swap the table and index
+ files. The data changes that took place during the creation of the new
+ table and index files are captured using logical decoding
+ (<xref linkend="logicaldecoding"/>) and applied before
+ the <literal>ACCESS EXCLUSIVE</literal> lock is requested. Thus the lock
+ is typically held only for the time needed to swap the files, which
+ should be pretty short. However, the time might still be noticeable if
+ too many data changes have been done to the table while
+ <command>REPACK</command> was waiting for the lock: those changes must
+ be processed just before the files are swapped, while the
+ <literal>ACCESS EXCLUSIVE</literal> lock is being held.
+ </para>
+
+ <para>
+ Note that <command>REPACK</command> with the
+ the <literal>CONCURRENTLY</literal> option does not try to order the
+ rows inserted into the table after the repacking started. Also
+ note <command>REPACK</command> might fail to complete due to DDL
+ commands executed on the table by other transactions during the
+ repacking.
+ </para>
+
+ <note>
+ <para>
+ In addition to the temporary space requirements explained in
+ <xref linkend="sql-repack-notes-on-resources"/>,
+ the <literal>CONCURRENTLY</literal> option can add to the usage of
+ temporary space a bit more. The reason is that other transactions can
+ perform DML operations which cannot be applied to the new file until
+ <command>REPACK</command> has copied all the tuples from the old
+ file. Thus the tuples inserted into the old file during the copying are
+ also stored separately in a temporary file, so they can eventually be
+ applied to the new file.
+ </para>
+
+ <para>
+ Furthermore, the data changes performed during the copying are
+ extracted from <link linkend="wal">write-ahead log</link> (WAL), and
+ this extraction (decoding) only takes place when certain amount of WAL
+ has been written. Therefore, WAL removal can be delayed by this
+ threshold. Currently the threshold is equal to the value of
+ the <link linkend="guc-wal-segment-size"><varname>wal_segment_size</varname></link>
+ configuration parameter.
+ </para>
+ </note>
+
+ <para>
+ The <literal>CONCURRENTLY</literal> option cannot be used in the
+ following cases:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The table is <literal>UNLOGGED</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The table is partitioned.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The table is a system catalog or a <acronym>TOAST</acronym> table.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <command>REPACK</command> is executed inside a transaction block.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <link linkend="guc-wal-level"><varname>wal_level</varname></link>
+ configuration parameter is less than <literal>logical</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+ configuration parameter does not allow for creation of an additional
+ replication slot.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <warning>
+ <para>
+ <command>REPACK</command> with the <literal>CONCURRENTLY</literal>
+ option is not MVCC-safe, see <xref linkend="mvcc-caveats"/> for
+ details.
+ </para>
+ </warning>
+
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>VERBOSE</literal></term>
<listitem>
diff --git a/src/Makefile b/src/Makefile
index 2f31a2f20a7..b18c9a14ffa 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -23,6 +23,7 @@ SUBDIRS = \
interfaces \
backend/replication/libpqwalreceiver \
backend/replication/pgoutput \
+ backend/replication/pgoutput_repack \
fe_utils \
bin \
pl \
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e3e7307ef5f..f9a4fe3faed 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -60,7 +60,8 @@ static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tuple,
- bool all_visible_cleared, bool new_all_visible_cleared);
+ bool all_visible_cleared, bool new_all_visible_cleared,
+ bool wal_logical);
#ifdef USE_ASSERT_CHECKING
static void check_lock_if_inplace_updateable_rel(Relation relation,
ItemPointer otid,
@@ -2780,7 +2781,7 @@ xmax_infomask_changed(uint16 new_infomask, uint16 old_infomask)
TM_Result
heap_delete(Relation relation, ItemPointer tid,
CommandId cid, Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, bool changingPart)
+ TM_FailureData *tmfd, bool changingPart, bool wal_logical)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
@@ -3027,7 +3028,8 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = wal_logical ?
+ ExtractReplicaIdentity(relation, &tp, true, &old_key_copied) : NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3117,6 +3119,15 @@ l1:
xlrec.flags |= XLH_DELETE_CONTAINS_OLD_KEY;
}
+ /*
+ * Unlike UPDATE, DELETE is decoded even if there is no old key, so it
+ * does not help to clear both XLH_DELETE_CONTAINS_OLD_TUPLE and
+ * XLH_DELETE_CONTAINS_OLD_KEY. Thus we need an extra flag. TODO
+ * Consider not decoding tuples w/o the old tuple/key instead.
+ */
+ if (!wal_logical)
+ xlrec.flags |= XLH_DELETE_NO_LOGICAL;
+
XLogBeginInsert();
XLogRegisterData(&xlrec, SizeOfHeapDelete);
@@ -3209,7 +3220,8 @@ simple_heap_delete(Relation relation, ItemPointer tid)
result = heap_delete(relation, tid,
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, false /* changingPart */ );
+ &tmfd, false, /* changingPart */
+ true /* wal_logical */ );
switch (result)
{
case TM_SelfModified:
@@ -3250,7 +3262,7 @@ TM_Result
heap_update(Relation relation, ItemPointer otid, HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TU_UpdateIndexes *update_indexes, bool wal_logical)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
@@ -4143,7 +4155,8 @@ l2:
newbuf, &oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
- all_visible_cleared_new);
+ all_visible_cleared_new,
+ wal_logical);
if (newbuf != buffer)
{
PageSetLSN(BufferGetPage(newbuf), recptr);
@@ -4501,7 +4514,8 @@ simple_heap_update(Relation relation, ItemPointer otid, HeapTuple tup,
result = heap_update(relation, otid, tup,
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode, update_indexes,
+ true /* wal_logical */ );
switch (result)
{
case TM_SelfModified:
@@ -8842,7 +8856,8 @@ static XLogRecPtr
log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup, HeapTuple newtup,
HeapTuple old_key_tuple,
- bool all_visible_cleared, bool new_all_visible_cleared)
+ bool all_visible_cleared, bool new_all_visible_cleared,
+ bool wal_logical)
{
xl_heap_update xlrec;
xl_heap_header xlhdr;
@@ -8853,7 +8868,8 @@ log_heap_update(Relation reln, Buffer oldbuf,
suffixlen = 0;
XLogRecPtr recptr;
Page page = BufferGetPage(newbuf);
- bool need_tuple_data = RelationIsLogicallyLogged(reln);
+ bool need_tuple_data = RelationIsLogicallyLogged(reln) &&
+ wal_logical;
bool init;
int bufflags;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 79f9de5d760..d03084768e0 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -33,6 +33,7 @@
#include "catalog/index.h"
#include "catalog/storage.h"
#include "catalog/storage_xlog.h"
+#include "commands/cluster.h"
#include "commands/progress.h"
#include "executor/executor.h"
#include "miscadmin.h"
@@ -309,7 +310,8 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
* the storage itself is cleaning the dead tuples by itself, it is the
* time to call the index tuple deletion also.
*/
- return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart);
+ return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart,
+ true);
}
@@ -328,7 +330,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
tuple->t_tableOid = slot->tts_tableOid;
result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ tmfd, lockmode, update_indexes, true);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -685,13 +687,15 @@ static void
heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
Relation OldIndex, bool use_sort,
TransactionId OldestXmin,
+ Snapshot snapshot,
+ LogicalDecodingContext *decoding_ctx,
TransactionId *xid_cutoff,
MultiXactId *multi_cutoff,
double *num_tuples,
double *tups_vacuumed,
double *tups_recently_dead)
{
- RewriteState rwstate;
+ RewriteState rwstate = NULL;
IndexScanDesc indexScan;
TableScanDesc tableScan;
HeapScanDesc heapScan;
@@ -705,6 +709,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
bool *isnull;
BufferHeapTupleTableSlot *hslot;
BlockNumber prev_cblock = InvalidBlockNumber;
+ bool concurrent = snapshot != NULL;
+ XLogRecPtr end_of_wal_prev = GetFlushRecPtr(NULL);
/* Remember if it's a system catalog */
is_system_catalog = IsSystemRelation(OldHeap);
@@ -720,9 +726,12 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
values = (Datum *) palloc(natts * sizeof(Datum));
isnull = (bool *) palloc(natts * sizeof(bool));
- /* Initialize the rewrite operation */
- rwstate = begin_heap_rewrite(OldHeap, NewHeap, OldestXmin, *xid_cutoff,
- *multi_cutoff);
+ /*
+ * Initialize the rewrite operation.
+ */
+ if (!concurrent)
+ rwstate = begin_heap_rewrite(OldHeap, NewHeap, OldestXmin,
+ *xid_cutoff, *multi_cutoff);
/* Set up sorting if wanted */
@@ -737,6 +746,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
* Prepare to scan the OldHeap. To ensure we see recently-dead tuples
* that still need to be copied, we scan with SnapshotAny and use
* HeapTupleSatisfiesVacuum for the visibility test.
+ *
+ * In the CONCURRENTLY case, we do regular MVCC visibility tests, using
+ * the snapshot passed by the caller.
*/
if (OldIndex != NULL && !use_sort)
{
@@ -753,7 +765,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
tableScan = NULL;
heapScan = NULL;
- indexScan = index_beginscan(OldHeap, OldIndex, SnapshotAny, NULL, 0, 0);
+ indexScan = index_beginscan(OldHeap, OldIndex,
+ snapshot ? snapshot : SnapshotAny,
+ NULL, 0, 0);
index_rescan(indexScan, NULL, 0, NULL, 0);
}
else
@@ -762,7 +776,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SEQ_SCAN_HEAP);
- tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL);
+ tableScan = table_beginscan(OldHeap,
+ snapshot ? snapshot : SnapshotAny,
+ 0, (ScanKey) NULL);
heapScan = (HeapScanDesc) tableScan;
indexScan = NULL;
@@ -785,6 +801,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
HeapTuple tuple;
Buffer buf;
bool isdead;
+ HTSV_Result vis;
CHECK_FOR_INTERRUPTS();
@@ -837,70 +854,84 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
tuple = ExecFetchSlotHeapTuple(slot, false, NULL);
buf = hslot->buffer;
- LockBuffer(buf, BUFFER_LOCK_SHARE);
-
- switch (HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf))
+ /*
+ * Regarding CONCURRENTLY, see the comments on MVCC snapshot above.
+ */
+ if (!concurrent)
{
- case HEAPTUPLE_DEAD:
- /* Definitely dead */
- isdead = true;
- break;
- case HEAPTUPLE_RECENTLY_DEAD:
- *tups_recently_dead += 1;
- /* fall through */
- case HEAPTUPLE_LIVE:
- /* Live or recently dead, must copy it */
- isdead = false;
- break;
- case HEAPTUPLE_INSERT_IN_PROGRESS:
+ LockBuffer(buf, BUFFER_LOCK_SHARE);
- /*
- * Since we hold exclusive lock on the relation, normally the
- * only way to see this is if it was inserted earlier in our
- * own transaction. However, it can happen in system
- * catalogs, since we tend to release write lock before commit
- * there. Give a warning if neither case applies; but in any
- * case we had better copy it.
- */
- if (!is_system_catalog &&
- !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple->t_data)))
- elog(WARNING, "concurrent insert in progress within table \"%s\"",
- RelationGetRelationName(OldHeap));
- /* treat as live */
- isdead = false;
- break;
- case HEAPTUPLE_DELETE_IN_PROGRESS:
+ switch ((vis = HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf)))
+ {
+ case HEAPTUPLE_DEAD:
+ /* Definitely dead */
+ isdead = true;
+ break;
+ case HEAPTUPLE_RECENTLY_DEAD:
+ *tups_recently_dead += 1;
+ /* fall through */
+ case HEAPTUPLE_LIVE:
+ /* Live or recently dead, must copy it */
+ isdead = false;
+ break;
+ case HEAPTUPLE_INSERT_IN_PROGRESS:
- /*
- * Similar situation to INSERT_IN_PROGRESS case.
- */
- if (!is_system_catalog &&
- !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetUpdateXid(tuple->t_data)))
- elog(WARNING, "concurrent delete in progress within table \"%s\"",
- RelationGetRelationName(OldHeap));
- /* treat as recently dead */
- *tups_recently_dead += 1;
- isdead = false;
- break;
- default:
- elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
- isdead = false; /* keep compiler quiet */
- break;
- }
+ /*
+ * As long as we hold exclusive lock on the relation,
+ * normally the only way to see this is if it was inserted
+ * earlier in our own transaction. However, it can happen
+ * in system catalogs, since we tend to release write lock
+ * before commit there. Also, there's no exclusive lock
+ * during concurrent processing. Give a warning if neither
+ * case applies; but in any case we had better copy it.
+ */
+ if (!is_system_catalog && !concurrent &&
+ !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple->t_data)))
+ elog(WARNING, "concurrent insert in progress within table \"%s\"",
+ RelationGetRelationName(OldHeap));
+ /* treat as live */
+ isdead = false;
+ break;
+ case HEAPTUPLE_DELETE_IN_PROGRESS:
- LockBuffer(buf, BUFFER_LOCK_UNLOCK);
+ /*
+ * Similar situation to INSERT_IN_PROGRESS case.
+ */
+ if (!is_system_catalog && !concurrent &&
+ !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetUpdateXid(tuple->t_data)))
+ elog(WARNING, "concurrent delete in progress within table \"%s\"",
+ RelationGetRelationName(OldHeap));
+ /* treat as recently dead */
+ *tups_recently_dead += 1;
+ isdead = false;
+ break;
+ default:
+ elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
+ isdead = false; /* keep compiler quiet */
+ break;
+ }
- if (isdead)
- {
- *tups_vacuumed += 1;
- /* heap rewrite module still needs to see it... */
- if (rewrite_heap_dead_tuple(rwstate, tuple))
+ if (isdead)
{
- /* A previous recently-dead tuple is now known dead */
*tups_vacuumed += 1;
- *tups_recently_dead -= 1;
+ /* heap rewrite module still needs to see it... */
+ if (rewrite_heap_dead_tuple(rwstate, tuple))
+ {
+ /* A previous recently-dead tuple is now known dead */
+ *tups_vacuumed += 1;
+ *tups_recently_dead -= 1;
+ }
+
+ LockBuffer(buf, BUFFER_LOCK_UNLOCK);
+ continue;
}
- continue;
+
+ /*
+ * In the concurrent case, we have a copy of the tuple, so we
+ * don't worry whether the source tuple will be deleted / updated
+ * after we release the lock.
+ */
+ LockBuffer(buf, BUFFER_LOCK_UNLOCK);
}
*num_tuples += 1;
@@ -919,7 +950,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
{
const int ct_index[] = {
PROGRESS_REPACK_HEAP_TUPLES_SCANNED,
- PROGRESS_REPACK_HEAP_TUPLES_WRITTEN
+ PROGRESS_REPACK_HEAP_TUPLES_INSERTED
};
int64 ct_val[2];
@@ -934,6 +965,31 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
ct_val[1] = *num_tuples;
pgstat_progress_update_multi_param(2, ct_index, ct_val);
}
+
+ /*
+ * Process the WAL produced by the load, as well as by other
+ * transactions, so that the replication slot can advance and WAL does
+ * not pile up. Use wal_segment_size as a threshold so that we do not
+ * introduce the decoding overhead too often.
+ *
+ * Of course, we must not apply the changes until the initial load has
+ * completed.
+ *
+ * Note that our insertions into the new table should not be decoded
+ * as we (intentionally) do not write the logical decoding specific
+ * information to WAL.
+ */
+ if (concurrent)
+ {
+ XLogRecPtr end_of_wal;
+
+ end_of_wal = GetFlushRecPtr(NULL);
+ if ((end_of_wal - end_of_wal_prev) > wal_segment_size)
+ {
+ repack_decode_concurrent_changes(decoding_ctx, end_of_wal);
+ end_of_wal_prev = end_of_wal;
+ }
+ }
}
if (indexScan != NULL)
@@ -977,7 +1033,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
values, isnull,
rwstate);
/* Report n_tuples */
- pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_WRITTEN,
+ pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
n_tuples);
}
@@ -985,7 +1041,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
}
/* Write out any remaining tuples, and fsync if needed */
- end_heap_rewrite(rwstate);
+ if (rwstate)
+ end_heap_rewrite(rwstate);
/* Clean up */
pfree(values);
@@ -2376,6 +2433,10 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
* SET WITHOUT OIDS.
*
* So, we must reconstruct the tuple from component Datums.
+ *
+ * If rwstate=NULL, use simple_heap_insert() instead of rewriting - in that
+ * case we still need to deform/form the tuple. TODO Shouldn't we rename the
+ * function, as might not do any rewrite?
*/
static void
reform_and_rewrite_tuple(HeapTuple tuple,
@@ -2398,8 +2459,28 @@ reform_and_rewrite_tuple(HeapTuple tuple,
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
- /* The heap rewrite module does the rest */
- rewrite_heap_tuple(rwstate, tuple, copiedTuple);
+ if (rwstate)
+ /* The heap rewrite module does the rest */
+ rewrite_heap_tuple(rwstate, tuple, copiedTuple);
+ else
+ {
+ /*
+ * Insert tuple when processing REPACK CONCURRENTLY.
+ *
+ * rewriteheap.c is not used in the CONCURRENTLY case because it'd be
+ * difficult to do the same in the catch-up phase (as the logical
+ * decoding does not provide us with sufficient visibility
+ * information). Thus we must use heap_insert() both during the
+ * catch-up and here.
+ *
+ * The following is like simple_heap_insert() except that we pass the
+ * flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps
+ * the relation files, it drops this relation, so no logical
+ * replication subscription should need the data.
+ */
+ heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true),
+ HEAP_INSERT_NO_LOGICAL, NULL);
+ }
heap_freetuple(copiedTuple);
}
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index e6d2b5fced1..6aa2ed214f2 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -617,9 +617,9 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
int options = HEAP_INSERT_SKIP_FSM;
/*
- * While rewriting the heap for VACUUM FULL / CLUSTER, make sure data
- * for the TOAST table are not logically decoded. The main heap is
- * WAL-logged as XLOG FPI records, which are not logically decoded.
+ * While rewriting the heap for REPACK, make sure data for the TOAST
+ * table are not logically decoded. The main heap is WAL-logged as
+ * XLOG FPI records, which are not logically decoded.
*/
options |= HEAP_INSERT_NO_LOGICAL;
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b46e7e9c2a6..5670f2bfbde 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -215,6 +215,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ bool internal; /* for a subxact: launched internally? */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -4735,6 +4736,7 @@ BeginInternalSubTransaction(const char *name)
/* Normal subtransaction start */
PushTransaction();
s = CurrentTransactionState; /* changed by push */
+ s->internal = true;
/*
* Savepoint names, like the TransactionState block itself, live
@@ -5251,7 +5253,13 @@ AbortSubTransaction(void)
LWLockReleaseAll();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Internal subtransacion might be used by an user command, in which case
+ * the command outlives the subtransaction.
+ */
+ if (!s->internal)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
@@ -5468,6 +5476,7 @@ PushTransaction(void)
s->parallelModeLevel = 0;
s->parallelChildXact = (p->parallelModeLevel != 0 || p->parallelChildXact);
s->topXidLogged = false;
+ s->internal = false;
CurrentTransactionState = s;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index b2b7b10c2be..a92ac78ad9e 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1266,16 +1266,17 @@ CREATE VIEW pg_stat_progress_cluster AS
WHEN 2 THEN 'index scanning heap'
WHEN 3 THEN 'sorting tuples'
WHEN 4 THEN 'writing new heap'
- WHEN 5 THEN 'swapping relation files'
- WHEN 6 THEN 'rebuilding index'
- WHEN 7 THEN 'performing final cleanup'
+ -- 5 is 'catch-up', but that should not appear here.
+ WHEN 6 THEN 'swapping relation files'
+ WHEN 7 THEN 'rebuilding index'
+ WHEN 8 THEN 'performing final cleanup'
END AS phase,
CAST(S.param3 AS oid) AS cluster_index_relid,
S.param4 AS heap_tuples_scanned,
S.param5 AS heap_tuples_written,
- S.param6 AS heap_blks_total,
- S.param7 AS heap_blks_scanned,
- S.param8 AS index_rebuild_count
+ S.param8 AS heap_blks_total,
+ S.param9 AS heap_blks_scanned,
+ S.param10 AS index_rebuild_count
FROM pg_stat_get_progress_info('CLUSTER') AS S
LEFT JOIN pg_database D ON S.datid = D.oid;
@@ -1291,16 +1292,19 @@ CREATE VIEW pg_stat_progress_repack AS
WHEN 2 THEN 'index scanning heap'
WHEN 3 THEN 'sorting tuples'
WHEN 4 THEN 'writing new heap'
- WHEN 5 THEN 'swapping relation files'
- WHEN 6 THEN 'rebuilding index'
- WHEN 7 THEN 'performing final cleanup'
+ WHEN 5 THEN 'catch-up'
+ WHEN 6 THEN 'swapping relation files'
+ WHEN 7 THEN 'rebuilding index'
+ WHEN 8 THEN 'performing final cleanup'
END AS phase,
CAST(S.param3 AS oid) AS repack_index_relid,
S.param4 AS heap_tuples_scanned,
- S.param5 AS heap_tuples_written,
- S.param6 AS heap_blks_total,
- S.param7 AS heap_blks_scanned,
- S.param8 AS index_rebuild_count
+ S.param5 AS heap_tuples_inserted,
+ S.param6 AS heap_tuples_updated,
+ S.param7 AS heap_tuples_deleted,
+ S.param8 AS heap_blks_total,
+ S.param9 AS heap_blks_scanned,
+ S.param10 AS index_rebuild_count
FROM pg_stat_get_progress_info('REPACK') AS S
LEFT JOIN pg_database D ON S.datid = D.oid;
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 8b64f9e6795..61224a3adf2 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -25,6 +25,10 @@
#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "access/xlog.h"
+#include "access/xlog_internal.h"
+#include "access/xloginsert.h"
+#include "access/xlogutils.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
@@ -32,6 +36,7 @@
#include "catalog/namespace.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_control.h"
#include "catalog/pg_inherits.h"
#include "catalog/toasting.h"
#include "commands/cluster.h"
@@ -39,15 +44,21 @@
#include "commands/progress.h"
#include "commands/tablecmds.h"
#include "commands/vacuum.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "optimizer/optimizer.h"
#include "pgstat.h"
+#include "replication/decode.h"
+#include "replication/logical.h"
+#include "replication/snapbuild.h"
#include "storage/bufmgr.h"
+#include "storage/ipc.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "utils/acl.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
+#include "utils/injection_point.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
@@ -67,13 +78,45 @@ typedef struct
Oid indexOid;
} RelToCluster;
+/*
+ * The following definitions are used for concurrent processing.
+ */
+
+/*
+ * The locators are used to avoid logical decoding of data that we do not need
+ * for our table.
+ */
+RelFileLocator repacked_rel_locator = {.relNumber = InvalidOid};
+RelFileLocator repacked_rel_toast_locator = {.relNumber = InvalidOid};
+
+/*
+ * Everything we need to call ExecInsertIndexTuples().
+ */
+typedef struct IndexInsertState
+{
+ ResultRelInfo *rri;
+ EState *estate;
+
+ Relation ident_index;
+} IndexInsertState;
+
+/* The WAL segment being decoded. */
+static XLogSegNo repack_current_segment = 0;
+
+
static bool cluster_rel_recheck(RepackCommand cmd, Relation OldHeap,
- Oid indexOid, Oid userid, int options);
+ Oid indexOid, Oid userid, LOCKMODE lmode,
+ int options);
+static void check_repack_concurrently_requirements(Relation rel);
static void rebuild_relation(RepackCommand cmd, bool usingindex,
- Relation OldHeap, Relation index, bool verbose);
+ Relation OldHeap, Relation index, Oid userid,
+ bool verbose, bool concurrent);
static void copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
- bool verbose, bool *pSwapToastByContent,
- TransactionId *pFreezeXid, MultiXactId *pCutoffMulti);
+ Snapshot snapshot, LogicalDecodingContext *decoding_ctx,
+ bool verbose,
+ bool *pSwapToastByContent,
+ TransactionId *pFreezeXid,
+ MultiXactId *pCutoffMulti);
static List *get_tables_to_repack(RepackCommand cmd, bool usingindex,
MemoryContext permcxt);
static List *get_tables_to_repack_partitioned(RepackCommand cmd,
@@ -81,12 +124,61 @@ static List *get_tables_to_repack_partitioned(RepackCommand cmd,
Oid relid, bool rel_is_index);
static bool cluster_is_permitted_for_relation(RepackCommand cmd,
Oid relid, Oid userid);
+
+static void begin_concurrent_repack(Relation rel);
+static void end_concurrent_repack(void);
+static LogicalDecodingContext *setup_logical_decoding(Oid relid,
+ const char *slotname,
+ TupleDesc tupdesc);
+static HeapTuple get_changed_tuple(char *change);
+static void apply_concurrent_changes(RepackDecodingState *dstate,
+ Relation rel, ScanKey key, int nkeys,
+ IndexInsertState *iistate);
+static void apply_concurrent_insert(Relation rel, ConcurrentChange *change,
+ HeapTuple tup, IndexInsertState *iistate,
+ TupleTableSlot *index_slot);
+static void apply_concurrent_update(Relation rel, HeapTuple tup,
+ HeapTuple tup_target,
+ ConcurrentChange *change,
+ IndexInsertState *iistate,
+ TupleTableSlot *index_slot);
+static void apply_concurrent_delete(Relation rel, HeapTuple tup_target,
+ ConcurrentChange *change);
+static HeapTuple find_target_tuple(Relation rel, ScanKey key, int nkeys,
+ HeapTuple tup_key,
+ IndexInsertState *iistate,
+ TupleTableSlot *ident_slot,
+ IndexScanDesc *scan_p);
+static void process_concurrent_changes(LogicalDecodingContext *ctx,
+ XLogRecPtr end_of_wal,
+ Relation rel_dst,
+ Relation rel_src,
+ ScanKey ident_key,
+ int ident_key_nentries,
+ IndexInsertState *iistate);
+static IndexInsertState *get_index_insert_state(Relation relation,
+ Oid ident_index_id);
+static ScanKey build_identity_key(Oid ident_idx_oid, Relation rel_src,
+ int *nentries);
+static void free_index_insert_state(IndexInsertState *iistate);
+static void cleanup_logical_decoding(LogicalDecodingContext *ctx);
+static void rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
+ Relation cl_index,
+ LogicalDecodingContext *ctx,
+ bool swap_toast_by_content,
+ TransactionId frozenXid,
+ MultiXactId cutoffMulti);
+static List *build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes);
static Relation process_single_relation(RepackStmt *stmt,
+ LOCKMODE lockmode,
+ bool isTopLevel,
ClusterParams *params);
static Oid determine_clustered_index(Relation rel, bool usingindex,
const char *indexname);
+#define REPL_PLUGIN_NAME "pgoutput_repack"
+
static const char *
RepackCommandAsString(RepackCommand cmd)
{
@@ -95,7 +187,7 @@ RepackCommandAsString(RepackCommand cmd)
case REPACK_COMMAND_REPACK:
return "REPACK";
case REPACK_COMMAND_VACUUMFULL:
- return "VACUUM";
+ return "VACUUM (FULL)";
case REPACK_COMMAND_CLUSTER:
return "CLUSTER";
}
@@ -132,6 +224,7 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
ClusterParams params = {0};
Relation rel = NULL;
MemoryContext repack_context;
+ LOCKMODE lockmode;
List *rtcs;
/* Parse option list */
@@ -142,6 +235,16 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
else if (strcmp(opt->defname, "analyze") == 0 ||
strcmp(opt->defname, "analyse") == 0)
params.options |= defGetBoolean(opt) ? CLUOPT_ANALYZE : 0;
+ else if (strcmp(opt->defname, "concurrently") == 0 &&
+ defGetBoolean(opt))
+ {
+ if (stmt->command != REPACK_COMMAND_REPACK)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY option not supported for %s",
+ RepackCommandAsString(stmt->command)));
+ params.options |= CLUOPT_CONCURRENT;
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -151,13 +254,25 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
parser_errposition(pstate, opt->location)));
}
+ /*
+ * Determine the lock mode expected by cluster_rel().
+ *
+ * In the exclusive case, we obtain AccessExclusiveLock right away to
+ * avoid lock-upgrade hazard in the single-transaction case. In the
+ * CONCURRENTLY case, the AccessExclusiveLock will only be used at the end
+ * of processing, supposedly for very short time. Until then, we'll have
+ * to unlock the relation temporarily, so there's no lock-upgrade hazard.
+ */
+ lockmode = (params.options & CLUOPT_CONCURRENT) == 0 ?
+ AccessExclusiveLock : ShareUpdateExclusiveLock;
+
/*
* If a single relation is specified, process it and we're done ... unless
* the relation is a partitioned table, in which case we fall through.
*/
if (stmt->relation != NULL)
{
- rel = process_single_relation(stmt, ¶ms);
+ rel = process_single_relation(stmt, lockmode, isTopLevel, ¶ms);
if (rel == NULL)
return;
}
@@ -169,10 +284,29 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
errmsg("cannot ANALYZE multiple tables"));
/*
- * By here, we know we are in a multi-table situation. In order to avoid
- * holding locks for too long, we want to process each table in its own
- * transaction. This forces us to disallow running inside a user
- * transaction block.
+ * By here, we know we are in a multi-table situation.
+ *
+ * Concurrent processing is currently considered rather special (e.g. in
+ * terms of resources consumed) so it is not performed in bulk.
+ */
+ if (params.options & CLUOPT_CONCURRENT)
+ {
+ if (rel != NULL)
+ {
+ Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
+ ereport(ERROR,
+ errmsg("REPACK CONCURRENTLY not supported for partitioned tables"),
+ errhint("Consider running the command for individual partitions."));
+ }
+ else
+ ereport(ERROR,
+ errmsg("REPACK CONCURRENTLY requires explicit table name"));
+ }
+
+ /*
+ * In order to avoid holding locks for too long, we want to process each
+ * table in its own transaction. This forces us to disallow running
+ * inside a user transaction block.
*/
PreventInTransactionBlock(isTopLevel, RepackCommandAsString(stmt->command));
@@ -252,7 +386,7 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
* Open the target table, coping with the case where it has been
* dropped.
*/
- rel = try_table_open(rtc->tableOid, AccessExclusiveLock);
+ rel = try_table_open(rtc->tableOid, lockmode);
if (rel == NULL)
{
CommitTransactionCommand();
@@ -264,7 +398,7 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
/* Process this table */
cluster_rel(stmt->command, stmt->usingindex,
- rel, rtc->indexOid, ¶ms);
+ rel, rtc->indexOid, ¶ms, isTopLevel);
/* cluster_rel closes the relation, but keeps lock */
PopActiveSnapshot();
@@ -293,22 +427,55 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
* If indexOid is InvalidOid, the table will be rewritten in physical order
* instead of index order.
*
+ * Note that, in the concurrent case, the function releases the lock at some
+ * point, in order to get AccessExclusiveLock for the final steps (i.e. to
+ * swap the relation files). To make things simpler, the caller should expect
+ * OldHeap to be closed on return, regardless CLUOPT_CONCURRENT. (The
+ * AccessExclusiveLock is kept till the end of the transaction.)
+ *
* 'cmd' indicates which command is being executed, to be used for error
* messages.
*/
void
cluster_rel(RepackCommand cmd, bool usingindex,
- Relation OldHeap, Oid indexOid, ClusterParams *params)
+ Relation OldHeap, Oid indexOid, ClusterParams *params,
+ bool isTopLevel)
{
Oid tableOid = RelationGetRelid(OldHeap);
+ Relation index;
+ LOCKMODE lmode;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
bool verbose = ((params->options & CLUOPT_VERBOSE) != 0);
bool recheck = ((params->options & CLUOPT_RECHECK) != 0);
- Relation index;
+ bool concurrent = ((params->options & CLUOPT_CONCURRENT) != 0);
- Assert(CheckRelationLockedByMe(OldHeap, AccessExclusiveLock, false));
+ /*
+ * Check that the correct lock is held. The lock mode is
+ * AccessExclusiveLock for normal processing and ShareUpdateExclusiveLock
+ * for concurrent processing (so that SELECT, INSERT, UPDATE and DELETE
+ * commands work, but cluster_rel() cannot be called concurrently for the
+ * same relation).
+ */
+ lmode = !concurrent ? AccessExclusiveLock : ShareUpdateExclusiveLock;
+
+ /* There are specific requirements on concurrent processing. */
+ if (concurrent)
+ {
+ /*
+ * Make sure we have no XID assigned, otherwise call of
+ * setup_logical_decoding() can cause a deadlock.
+ *
+ * The existence of transaction block actually does not imply that XID
+ * was already assigned, but it very likely is. We might want to check
+ * the result of GetCurrentTransactionIdIfAny() instead, but that
+ * would be less clear from user's perspective.
+ */
+ PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
+
+ check_repack_concurrently_requirements(OldHeap);
+ }
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -351,11 +518,13 @@ cluster_rel(RepackCommand cmd, bool usingindex,
* If this is a single-transaction CLUSTER, we can skip these tests. We
* *must* skip the one on indisclustered since it would reject an attempt
* to cluster a not-previously-clustered index.
+ *
+ * XXX move [some of] these comments to where the RECHECK flag is
+ * determined?
*/
- if (recheck)
- if (!cluster_rel_recheck(cmd, OldHeap, indexOid, save_userid,
- params->options))
- goto out;
+ if (recheck && !cluster_rel_recheck(cmd, OldHeap, indexOid, save_userid,
+ lmode, params->options))
+ goto out;
/*
* We allow repacking shared catalogs only when not using an index. It
@@ -369,6 +538,12 @@ cluster_rel(RepackCommand cmd, bool usingindex,
errmsg("cannot run \"%s\" on a shared catalog",
RepackCommandAsString(cmd))));
+ /*
+ * The CONCURRENTLY case should have been rejected earlier because it does
+ * not support system catalogs.
+ */
+ Assert(!(OldHeap->rd_rel->relisshared && concurrent));
+
/*
* Don't process temp tables of other backends ... their local buffer
* manager is not going to cope.
@@ -404,7 +579,7 @@ cluster_rel(RepackCommand cmd, bool usingindex,
if (OidIsValid(indexOid))
{
/* verify the index is good and lock it */
- check_index_is_clusterable(OldHeap, indexOid, AccessExclusiveLock);
+ check_index_is_clusterable(OldHeap, indexOid, lmode);
/* also open it */
index = index_open(indexOid, NoLock);
}
@@ -421,7 +596,9 @@ cluster_rel(RepackCommand cmd, bool usingindex,
if (OldHeap->rd_rel->relkind == RELKIND_MATVIEW &&
!RelationIsPopulated(OldHeap))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ if (index)
+ index_close(index, lmode);
+ relation_close(OldHeap, lmode);
goto out;
}
@@ -434,11 +611,35 @@ cluster_rel(RepackCommand cmd, bool usingindex,
* invalid, because we move tuples around. Promote them to relation
* locks. Predicate locks on indexes will be promoted when they are
* reindexed.
+ *
+ * During concurrent processing, the heap as well as its indexes stay in
+ * operation, so we postpone this step until they are locked using
+ * AccessExclusiveLock near the end of the processing.
*/
- TransferPredicateLocksToHeapRelation(OldHeap);
+ if (!concurrent)
+ TransferPredicateLocksToHeapRelation(OldHeap);
/* rebuild_relation does all the dirty work */
- rebuild_relation(cmd, usingindex, OldHeap, index, verbose);
+ PG_TRY();
+ {
+ /*
+ * For concurrent processing, make sure that our logical decoding
+ * ignores data changes of other tables than the one we are
+ * processing.
+ */
+ if (concurrent)
+ begin_concurrent_repack(OldHeap);
+
+ rebuild_relation(cmd, usingindex, OldHeap, index, save_userid,
+ verbose, concurrent);
+ }
+ PG_FINALLY();
+ {
+ if (concurrent)
+ end_concurrent_repack();
+ }
+ PG_END_TRY();
+
/* rebuild_relation closes OldHeap, and index if valid */
out:
@@ -457,14 +658,14 @@ out:
*/
static bool
cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
- Oid userid, int options)
+ Oid userid, LOCKMODE lmode, int options)
{
Oid tableOid = RelationGetRelid(OldHeap);
/* Check that the user still has privileges for the relation */
if (!cluster_is_permitted_for_relation(cmd, tableOid, userid))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
@@ -478,7 +679,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
*/
if (RELATION_IS_OTHER_TEMP(OldHeap))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
@@ -489,7 +690,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
*/
if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(indexOid)))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
@@ -500,7 +701,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
if ((options & CLUOPT_RECHECK_ISCLUSTERED) != 0 &&
!get_index_isclustered(indexOid))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
}
@@ -641,19 +842,89 @@ mark_index_clustered(Relation rel, Oid indexOid, bool is_internal)
table_close(pg_index, RowExclusiveLock);
}
+/*
+ * Check if the CONCURRENTLY option is legal for the relation.
+ */
+static void
+check_repack_concurrently_requirements(Relation rel)
+{
+ char relpersistence,
+ replident;
+ Oid ident_idx;
+
+ /* Data changes in system relations are not logically decoded. */
+ if (IsCatalogRelation(rel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("REPACK CONCURRENTLY is not supported for catalog relations.")));
+
+ /*
+ * reorderbuffer.c does not seem to handle processing of TOAST relation
+ * alone.
+ */
+ if (IsToastRelation(rel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("REPACK CONCURRENTLY is not supported for TOAST relations, unless the main relation is repacked too.")));
+
+ relpersistence = rel->rd_rel->relpersistence;
+ if (relpersistence != RELPERSISTENCE_PERMANENT)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("REPACK CONCURRENTLY is only allowed for permanent relations.")));
+
+ /* With NOTHING, WAL does not contain the old tuple. */
+ replident = rel->rd_rel->relreplident;
+ if (replident == REPLICA_IDENTITY_NOTHING)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("Relation \"%s\" has insufficient replication identity.",
+ RelationGetRelationName(rel))));
+
+ /*
+ * Identity index is not set if the replica identity is FULL, but PK might
+ * exist in such a case.
+ */
+ ident_idx = RelationGetReplicaIndex(rel);
+ if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
+ ident_idx = rel->rd_pkindex;
+ if (!OidIsValid(ident_idx))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot process relation \"%s\"",
+ RelationGetRelationName(rel)),
+ (errhint("Relation \"%s\" has no identity index.",
+ RelationGetRelationName(rel)))));
+}
+
+
/*
* rebuild_relation: rebuild an existing relation in index or physical order
*
- * OldHeap: table to rebuild.
+ * OldHeap: table to rebuild. See cluster_rel() for comments on the required
+ * lock strength.
+ *
* index: index to cluster by, or NULL to rewrite in physical order.
*
- * On entry, heap and index (if one is given) must be open, and
- * AccessExclusiveLock held on them.
- * On exit, they are closed, but locks on them are not released.
+ * On entry, heap and index (if one is given) must be open, and the
+ * appropriate lock held on them -- AccessExclusiveLock for exclusive
+ * processing and ShareUpdateExclusiveLock for concurrent processing.
+ *
+ * On exit, they are closed, but still locked with AccessExclusiveLock. (The
+ * function handles the lock upgrade if 'concurrent' is true.)
*/
static void
rebuild_relation(RepackCommand cmd, bool usingindex,
- Relation OldHeap, Relation index, bool verbose)
+ Relation OldHeap, Relation index, Oid userid,
+ bool verbose, bool concurrent)
{
Oid tableOid = RelationGetRelid(OldHeap);
Oid accessMethod = OldHeap->rd_rel->relam;
@@ -661,13 +932,55 @@ rebuild_relation(RepackCommand cmd, bool usingindex,
Oid OIDNewHeap;
Relation NewHeap;
char relpersistence;
- bool is_system_catalog;
bool swap_toast_by_content;
TransactionId frozenXid;
MultiXactId cutoffMulti;
+ NameData slotname;
+ LogicalDecodingContext *ctx = NULL;
+ Snapshot snapshot = NULL;
+#if USE_ASSERT_CHECKING
+ LOCKMODE lmode;
- Assert(CheckRelationLockedByMe(OldHeap, AccessExclusiveLock, false) &&
- (index == NULL || CheckRelationLockedByMe(index, AccessExclusiveLock, false)));
+ lmode = concurrent ? ShareUpdateExclusiveLock : AccessExclusiveLock;
+
+ Assert(CheckRelationLockedByMe(OldHeap, lmode, false));
+ Assert(!usingindex || CheckRelationLockedByMe(index, lmode, false));
+#endif
+
+ if (concurrent)
+ {
+ TupleDesc tupdesc;
+
+ /*
+ * REPACK CONCURRENTLY is not allowed in a transaction block, so this
+ * should never fire.
+ */
+ Assert(GetTopTransactionIdIfAny() == InvalidTransactionId);
+
+ /*
+ * A single backend should not execute multiple REPACK commands at a
+ * time, so use PID to make the slot unique.
+ */
+ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid);
+
+ tupdesc = CreateTupleDescCopy(RelationGetDescr(OldHeap));
+
+ /*
+ * Prepare to capture the concurrent data changes.
+ *
+ * Note that this call waits for all transactions with XID already
+ * assigned to finish. If some of those transactions is waiting for a
+ * lock conflicting with ShareUpdateExclusiveLock on our table (e.g.
+ * it runs CREATE INDEX), we can end up in a deadlock. Not sure this
+ * risk is worth unlocking/locking the table (and its clustering
+ * index) and checking again if its still eligible for REPACK
+ * CONCURRENTLY.
+ */
+ ctx = setup_logical_decoding(tableOid, NameStr(slotname), tupdesc);
+
+ snapshot = SnapBuildInitialSnapshotForRepack(ctx->snapshot_builder);
+ PushActiveSnapshot(snapshot);
+ }
/* for CLUSTER or REPACK USING INDEX, mark the index as the one to use */
if (usingindex)
@@ -675,7 +988,6 @@ rebuild_relation(RepackCommand cmd, bool usingindex,
/* Remember info about rel before closing OldHeap */
relpersistence = OldHeap->rd_rel->relpersistence;
- is_system_catalog = IsSystemRelation(OldHeap);
/*
* Create the transient table that will receive the re-ordered data.
@@ -691,30 +1003,67 @@ rebuild_relation(RepackCommand cmd, bool usingindex,
NewHeap = table_open(OIDNewHeap, NoLock);
/* Copy the heap data into the new table in the desired order */
- copy_table_data(NewHeap, OldHeap, index, verbose,
+ copy_table_data(NewHeap, OldHeap, index, snapshot, ctx, verbose,
&swap_toast_by_content, &frozenXid, &cutoffMulti);
+ /* The historic snapshot won't be needed anymore. */
+ if (snapshot)
+ PopActiveSnapshot();
- /* Close relcache entries, but keep lock until transaction commit */
- table_close(OldHeap, NoLock);
- if (index)
- index_close(index, NoLock);
-
- /*
- * Close the new relation so it can be dropped as soon as the storage is
- * swapped. The relation is not visible to others, so no need to unlock it
- * explicitly.
- */
- table_close(NewHeap, NoLock);
-
- /*
- * Swap the physical files of the target and transient tables, then
- * rebuild the target's indexes and throw away the transient table.
- */
- finish_heap_swap(tableOid, OIDNewHeap, is_system_catalog,
- swap_toast_by_content, false, true,
- frozenXid, cutoffMulti,
- relpersistence);
+ if (concurrent)
+ {
+ /*
+ * Push a snapshot that we will use to find old versions of rows when
+ * processing concurrent UPDATE and DELETE commands. (That snapshot
+ * should also be used by index expressions.)
+ */
+ PushActiveSnapshot(GetTransactionSnapshot());
+
+ /*
+ * Make sure we can find the tuples just inserted when applying DML
+ * commands on top of those.
+ */
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+
+ rebuild_relation_finish_concurrent(NewHeap, OldHeap, index,
+ ctx, swap_toast_by_content,
+ frozenXid, cutoffMulti);
+ PopActiveSnapshot();
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+
+ /* Done with decoding. */
+ cleanup_logical_decoding(ctx);
+ ReplicationSlotRelease();
+ ReplicationSlotDrop(NameStr(slotname), false);
+ }
+ else
+ {
+ bool is_system_catalog = IsSystemRelation(OldHeap);
+
+ /* Close relcache entries, but keep lock until transaction commit */
+ table_close(OldHeap, NoLock);
+ if (index)
+ index_close(index, NoLock);
+
+ /*
+ * Close the new relation so it can be dropped as soon as the storage
+ * is swapped. The relation is not visible to others, so no need to
+ * unlock it explicitly.
+ */
+ table_close(NewHeap, NoLock);
+
+ /*
+ * Swap the physical files of the target and transient tables, then
+ * rebuild the target's indexes and throw away the transient table.
+ */
+ finish_heap_swap(tableOid, OIDNewHeap, is_system_catalog,
+ swap_toast_by_content, false, true, true,
+ frozenXid, cutoffMulti,
+ relpersistence);
+ }
}
@@ -849,15 +1198,19 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod,
/*
* Do the physical copying of table data.
*
+ * 'snapshot' and 'decoding_ctx': see table_relation_copy_for_cluster(). Pass
+ * iff concurrent processing is required.
+ *
* There are three output parameters:
* *pSwapToastByContent is set true if toast tables must be swapped by content.
* *pFreezeXid receives the TransactionId used as freeze cutoff point.
* *pCutoffMulti receives the MultiXactId used as a cutoff point.
*/
static void
-copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verbose,
- bool *pSwapToastByContent, TransactionId *pFreezeXid,
- MultiXactId *pCutoffMulti)
+copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
+ Snapshot snapshot, LogicalDecodingContext *decoding_ctx,
+ bool verbose, bool *pSwapToastByContent,
+ TransactionId *pFreezeXid, MultiXactId *pCutoffMulti)
{
Relation relRelation;
HeapTuple reltup;
@@ -875,6 +1228,8 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
PGRUsage ru0;
char *nspname;
+ bool concurrent = snapshot != NULL;
+
pg_rusage_init(&ru0);
/* Store a copy of the namespace name for logging purposes */
@@ -977,8 +1332,48 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
* provided, else plain seqscan.
*/
if (OldIndex != NULL && OldIndex->rd_rel->relam == BTREE_AM_OID)
+ {
+ ResourceOwner oldowner = NULL;
+ ResourceOwner resowner = NULL;
+
+ /*
+ * In the CONCURRENT case, use a dedicated resource owner so we don't
+ * leave any additional locks behind us that we cannot release easily.
+ */
+ if (concurrent)
+ {
+ Assert(CheckRelationLockedByMe(OldHeap, ShareUpdateExclusiveLock,
+ false));
+ Assert(CheckRelationLockedByMe(OldIndex, ShareUpdateExclusiveLock,
+ false));
+
+ resowner = ResourceOwnerCreate(CurrentResourceOwner,
+ "plan_cluster_use_sort");
+ oldowner = CurrentResourceOwner;
+ CurrentResourceOwner = resowner;
+ }
+
use_sort = plan_cluster_use_sort(RelationGetRelid(OldHeap),
RelationGetRelid(OldIndex));
+
+ if (concurrent)
+ {
+ CurrentResourceOwner = oldowner;
+
+ /*
+ * We are primarily concerned about locks, but if the planner
+ * happened to allocate any other resources, we should release
+ * them too because we're going to delete the whole resowner.
+ */
+ ResourceOwnerRelease(resowner, RESOURCE_RELEASE_BEFORE_LOCKS,
+ false, false);
+ ResourceOwnerRelease(resowner, RESOURCE_RELEASE_LOCKS,
+ false, false);
+ ResourceOwnerRelease(resowner, RESOURCE_RELEASE_AFTER_LOCKS,
+ false, false);
+ ResourceOwnerDelete(resowner);
+ }
+ }
else
use_sort = false;
@@ -1007,7 +1402,9 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
* values (e.g. because the AM doesn't use freezing).
*/
table_relation_copy_for_cluster(OldHeap, NewHeap, OldIndex, use_sort,
- cutoffs.OldestXmin, &cutoffs.FreezeLimit,
+ cutoffs.OldestXmin, snapshot,
+ decoding_ctx,
+ &cutoffs.FreezeLimit,
&cutoffs.MultiXactCutoff,
&num_tuples, &tups_vacuumed,
&tups_recently_dead);
@@ -1016,7 +1413,11 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
*pFreezeXid = cutoffs.FreezeLimit;
*pCutoffMulti = cutoffs.MultiXactCutoff;
- /* Reset rd_toastoid just to be tidy --- it shouldn't be looked at again */
+ /*
+ * Reset rd_toastoid just to be tidy --- it shouldn't be looked at again.
+ * In the CONCURRENTLY case, we need to set it again before applying the
+ * concurrent changes.
+ */
NewHeap->rd_toastoid = InvalidOid;
num_pages = RelationGetNumberOfBlocks(NewHeap);
@@ -1474,14 +1875,13 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
bool swap_toast_by_content,
bool check_constraints,
bool is_internal,
+ bool reindex,
TransactionId frozenXid,
MultiXactId cutoffMulti,
char newrelpersistence)
{
ObjectAddress object;
Oid mapped_tables[4];
- int reindex_flags;
- ReindexParams reindex_params = {0};
int i;
/* Report that we are now swapping relation files */
@@ -1507,39 +1907,47 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
if (is_system_catalog)
CacheInvalidateCatalog(OIDOldHeap);
- /*
- * Rebuild each index on the relation (but not the toast table, which is
- * all-new at this point). It is important to do this before the DROP
- * step because if we are processing a system catalog that will be used
- * during DROP, we want to have its indexes available. There is no
- * advantage to the other order anyway because this is all transactional,
- * so no chance to reclaim disk space before commit. We do not need a
- * final CommandCounterIncrement() because reindex_relation does it.
- *
- * Note: because index_build is called via reindex_relation, it will never
- * set indcheckxmin true for the indexes. This is OK even though in some
- * sense we are building new indexes rather than rebuilding existing ones,
- * because the new heap won't contain any HOT chains at all, let alone
- * broken ones, so it can't be necessary to set indcheckxmin.
- */
- reindex_flags = REINDEX_REL_SUPPRESS_INDEX_USE;
- if (check_constraints)
- reindex_flags |= REINDEX_REL_CHECK_CONSTRAINTS;
+ if (reindex)
+ {
+ int reindex_flags;
+ ReindexParams reindex_params = {0};
- /*
- * Ensure that the indexes have the same persistence as the parent
- * relation.
- */
- if (newrelpersistence == RELPERSISTENCE_UNLOGGED)
- reindex_flags |= REINDEX_REL_FORCE_INDEXES_UNLOGGED;
- else if (newrelpersistence == RELPERSISTENCE_PERMANENT)
- reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
+ /*
+ * Rebuild each index on the relation (but not the toast table, which
+ * is all-new at this point). It is important to do this before the
+ * DROP step because if we are processing a system catalog that will
+ * be used during DROP, we want to have its indexes available. There
+ * is no advantage to the other order anyway because this is all
+ * transactional, so no chance to reclaim disk space before commit. We
+ * do not need a final CommandCounterIncrement() because
+ * reindex_relation does it.
+ *
+ * Note: because index_build is called via reindex_relation, it will
+ * never set indcheckxmin true for the indexes. This is OK even
+ * though in some sense we are building new indexes rather than
+ * rebuilding existing ones, because the new heap won't contain any
+ * HOT chains at all, let alone broken ones, so it can't be necessary
+ * to set indcheckxmin.
+ */
+ reindex_flags = REINDEX_REL_SUPPRESS_INDEX_USE;
+ if (check_constraints)
+ reindex_flags |= REINDEX_REL_CHECK_CONSTRAINTS;
- /* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ /*
+ * Ensure that the indexes have the same persistence as the parent
+ * relation.
+ */
+ if (newrelpersistence == RELPERSISTENCE_UNLOGGED)
+ reindex_flags |= REINDEX_REL_FORCE_INDEXES_UNLOGGED;
+ else if (newrelpersistence == RELPERSISTENCE_PERMANENT)
+ reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
- reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
+ /* Report that we are now reindexing relations */
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+
+ reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
+ }
/* Report that we are now doing clean up */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
@@ -1881,7 +2289,8 @@ cluster_is_permitted_for_relation(RepackCommand cmd, Oid relid, Oid userid)
* resolve in this case.
*/
static Relation
-process_single_relation(RepackStmt *stmt, ClusterParams *params)
+process_single_relation(RepackStmt *stmt, LOCKMODE lockmode, bool isTopLevel,
+ ClusterParams *params)
{
Relation rel;
Oid tableOid;
@@ -1890,13 +2299,9 @@ process_single_relation(RepackStmt *stmt, ClusterParams *params)
Assert(stmt->command == REPACK_COMMAND_CLUSTER ||
stmt->command == REPACK_COMMAND_REPACK);
- /*
- * Find, lock, and check permissions on the table. We obtain
- * AccessExclusiveLock right away to avoid lock-upgrade hazard in the
- * single-transaction case.
- */
+ /* Find, lock, and check permissions on the table. */
tableOid = RangeVarGetRelidExtended(stmt->relation,
- AccessExclusiveLock,
+ lockmode,
0,
RangeVarCallbackMaintainsTable,
NULL);
@@ -1922,26 +2327,17 @@ process_single_relation(RepackStmt *stmt, ClusterParams *params)
return rel;
else
{
- Oid indexOid;
+ Oid indexOid = InvalidOid;
- indexOid = determine_clustered_index(rel, stmt->usingindex,
- stmt->indexname);
- if (OidIsValid(indexOid))
- check_index_is_clusterable(rel, indexOid, AccessExclusiveLock);
- cluster_rel(stmt->command, stmt->usingindex, rel, indexOid, params);
-
- /* Do an analyze, if requested */
- if (params->options & CLUOPT_ANALYZE)
+ if (stmt->usingindex)
{
- VacuumParams vac_params = {0};
-
- vac_params.options |= VACOPT_ANALYZE;
- if (params->options & CLUOPT_VERBOSE)
- vac_params.options |= VACOPT_VERBOSE;
- analyze_rel(RelationGetRelid(rel), NULL, vac_params, NIL, true,
- NULL);
+ indexOid = determine_clustered_index(rel, stmt->usingindex,
+ stmt->indexname);
+ check_index_is_clusterable(rel, indexOid, lockmode);
}
+ cluster_rel(stmt->command, stmt->usingindex, rel, indexOid,
+ params, isTopLevel);
return NULL;
}
}
@@ -1998,3 +2394,1048 @@ determine_clustered_index(Relation rel, bool usingindex, const char *indexname)
return indexOid;
}
+
+
+/*
+ * Call this function before REPACK CONCURRENTLY starts to setup logical
+ * decoding. It makes sure that other users of the table put enough
+ * information into WAL.
+ *
+ * The point is that at various places we expect that the table we're
+ * processing is treated like a system catalog. For example, we need to be
+ * able to scan it using a "historic snapshot" anytime during the processing
+ * (as opposed to scanning only at the start point of the decoding, as logical
+ * replication does during initial table synchronization), in order to apply
+ * concurrent UPDATE / DELETE commands.
+ *
+ * Note that TOAST table needs no attention here as it's not scanned using
+ * historic snapshot.
+ */
+static void
+begin_concurrent_repack(Relation rel)
+{
+ Oid toastrelid;
+
+ /* Avoid logical decoding of other relations by this backend. */
+ repacked_rel_locator = rel->rd_locator;
+ toastrelid = rel->rd_rel->reltoastrelid;
+ if (OidIsValid(toastrelid))
+ {
+ Relation toastrel;
+
+ /* Avoid logical decoding of other TOAST relations. */
+ toastrel = table_open(toastrelid, AccessShareLock);
+ repacked_rel_toast_locator = toastrel->rd_locator;
+ table_close(toastrel, AccessShareLock);
+ }
+}
+
+/*
+ * Call this when done with REPACK CONCURRENTLY.
+ */
+static void
+end_concurrent_repack(void)
+{
+ /*
+ * Restore normal function of (future) logical decoding for this backend.
+ */
+ repacked_rel_locator.relNumber = InvalidOid;
+ repacked_rel_toast_locator.relNumber = InvalidOid;
+}
+
+/*
+ * This function is much like pg_create_logical_replication_slot() except that
+ * the new slot is neither released (if anyone else could read changes from
+ * our slot, we could miss changes other backends do while we copy the
+ * existing data into temporary table), nor persisted (it's easier to handle
+ * crash by restarting all the work from scratch).
+ */
+static LogicalDecodingContext *
+setup_logical_decoding(Oid relid, const char *slotname, TupleDesc tupdesc)
+{
+ LogicalDecodingContext *ctx;
+ RepackDecodingState *dstate;
+
+ /*
+ * Check if we can use logical decoding.
+ */
+ CheckSlotPermissions();
+ CheckLogicalDecodingRequirements();
+
+ /* RS_TEMPORARY so that the slot gets cleaned up on ERROR. */
+ ReplicationSlotCreate(slotname, true, RS_TEMPORARY, false, false, false);
+
+ /*
+ * Neither prepare_write nor do_write callback nor update_progress is
+ * useful for us.
+ */
+ ctx = CreateInitDecodingContext(REPL_PLUGIN_NAME,
+ NIL,
+ true,
+ InvalidXLogRecPtr,
+ XL_ROUTINE(.page_read = read_local_xlog_page,
+ .segment_open = wal_segment_open,
+ .segment_close = wal_segment_close),
+ NULL, NULL, NULL);
+
+ /*
+ * We don't have control on setting fast_forward, so at least check it.
+ */
+ Assert(!ctx->fast_forward);
+
+ DecodingContextFindStartpoint(ctx);
+
+ /* Some WAL records should have been read. */
+ Assert(ctx->reader->EndRecPtr != InvalidXLogRecPtr);
+
+ XLByteToSeg(ctx->reader->EndRecPtr, repack_current_segment,
+ wal_segment_size);
+
+ /*
+ * Setup structures to store decoded changes.
+ */
+ dstate = palloc0(sizeof(RepackDecodingState));
+ dstate->relid = relid;
+ dstate->tstore = tuplestore_begin_heap(false, false,
+ maintenance_work_mem);
+
+ dstate->tupdesc = tupdesc;
+
+ /* Initialize the descriptor to store the changes ... */
+ dstate->tupdesc_change = CreateTemplateTupleDesc(1);
+
+ TupleDescInitEntry(dstate->tupdesc_change, 1, NULL, BYTEAOID, -1, 0);
+ /* ... as well as the corresponding slot. */
+ dstate->tsslot = MakeSingleTupleTableSlot(dstate->tupdesc_change,
+ &TTSOpsMinimalTuple);
+
+ dstate->resowner = ResourceOwnerCreate(CurrentResourceOwner,
+ "logical decoding");
+
+ ctx->output_writer_private = dstate;
+ return ctx;
+}
+
+/*
+ * Retrieve tuple from ConcurrentChange structure.
+ *
+ * The input data starts with the structure but it might not be appropriately
+ * aligned.
+ */
+static HeapTuple
+get_changed_tuple(char *change)
+{
+ HeapTupleData tup_data;
+ HeapTuple result;
+ char *src;
+
+ /*
+ * Ensure alignment before accessing the fields. (This is why we can't use
+ * heap_copytuple() instead of this function.)
+ */
+ src = change + offsetof(ConcurrentChange, tup_data);
+ memcpy(&tup_data, src, sizeof(HeapTupleData));
+
+ result = (HeapTuple) palloc(HEAPTUPLESIZE + tup_data.t_len);
+ memcpy(result, &tup_data, sizeof(HeapTupleData));
+ result->t_data = (HeapTupleHeader) ((char *) result + HEAPTUPLESIZE);
+ src = change + SizeOfConcurrentChange;
+ memcpy(result->t_data, src, result->t_len);
+
+ return result;
+}
+
+/*
+ * Decode logical changes from the WAL sequence up to end_of_wal.
+ */
+void
+repack_decode_concurrent_changes(LogicalDecodingContext *ctx,
+ XLogRecPtr end_of_wal)
+{
+ RepackDecodingState *dstate;
+ ResourceOwner resowner_old;
+
+ /*
+ * Invalidate the "present" cache before moving to "(recent) history".
+ */
+ InvalidateSystemCaches();
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+ resowner_old = CurrentResourceOwner;
+ CurrentResourceOwner = dstate->resowner;
+
+ PG_TRY();
+ {
+ while (ctx->reader->EndRecPtr < end_of_wal)
+ {
+ XLogRecord *record;
+ XLogSegNo segno_new;
+ char *errm = NULL;
+ XLogRecPtr end_lsn;
+
+ record = XLogReadRecord(ctx->reader, &errm);
+ if (errm)
+ elog(ERROR, "%s", errm);
+
+ if (record != NULL)
+ LogicalDecodingProcessRecord(ctx, ctx->reader);
+
+ /*
+ * If WAL segment boundary has been crossed, inform the decoding
+ * system that the catalog_xmin can advance. (We can confirm more
+ * often, but a filling a single WAL segment should not take much
+ * time.)
+ */
+ end_lsn = ctx->reader->EndRecPtr;
+ XLByteToSeg(end_lsn, segno_new, wal_segment_size);
+ if (segno_new != repack_current_segment)
+ {
+ LogicalConfirmReceivedLocation(end_lsn);
+ elog(DEBUG1, "REPACK: confirmed receive location %X/%X",
+ (uint32) (end_lsn >> 32), (uint32) end_lsn);
+ repack_current_segment = segno_new;
+ }
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ InvalidateSystemCaches();
+ CurrentResourceOwner = resowner_old;
+ }
+ PG_CATCH();
+ {
+ /* clear all timetravel entries */
+ InvalidateSystemCaches();
+ CurrentResourceOwner = resowner_old;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Apply changes that happened during the initial load.
+ *
+ * Scan key is passed by caller, so it does not have to be constructed
+ * multiple times. Key entries have all fields initialized, except for
+ * sk_argument.
+ */
+static void
+apply_concurrent_changes(RepackDecodingState *dstate, Relation rel,
+ ScanKey key, int nkeys, IndexInsertState *iistate)
+{
+ TupleTableSlot *index_slot,
+ *ident_slot;
+ HeapTuple tup_old = NULL;
+
+ if (dstate->nchanges == 0)
+ return;
+
+ /* TupleTableSlot is needed to pass the tuple to ExecInsertIndexTuples(). */
+ index_slot = MakeSingleTupleTableSlot(dstate->tupdesc, &TTSOpsHeapTuple);
+
+ /* A slot to fetch tuples from identity index. */
+ ident_slot = table_slot_create(rel, NULL);
+
+ while (tuplestore_gettupleslot(dstate->tstore, true, false,
+ dstate->tsslot))
+ {
+ bool shouldFree;
+ HeapTuple tup_change,
+ tup,
+ tup_exist;
+ char *change_raw,
+ *src;
+ ConcurrentChange change;
+ bool isnull[1];
+ Datum values[1];
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Get the change from the single-column tuple. */
+ tup_change = ExecFetchSlotHeapTuple(dstate->tsslot, false, &shouldFree);
+ heap_deform_tuple(tup_change, dstate->tupdesc_change, values, isnull);
+ Assert(!isnull[0]);
+
+ /* Make sure we access aligned data. */
+ change_raw = (char *) DatumGetByteaP(values[0]);
+ src = (char *) VARDATA(change_raw);
+ memcpy(&change, src, SizeOfConcurrentChange);
+
+ /* TRUNCATE change contains no tuple, so process it separately. */
+ if (change.kind == CHANGE_TRUNCATE)
+ {
+ /*
+ * All the things that ExecuteTruncateGuts() does (such as firing
+ * triggers or handling the DROP_CASCADE behavior) should have
+ * taken place on the source relation. Thus we only do the actual
+ * truncation of the new relation (and its indexes).
+ */
+ heap_truncate_one_rel(rel);
+
+ pfree(tup_change);
+ continue;
+ }
+
+ /*
+ * Extract the tuple from the change. The tuple is copied here because
+ * it might be assigned to 'tup_old', in which case it needs to
+ * survive into the next iteration.
+ */
+ tup = get_changed_tuple(src);
+
+ if (change.kind == CHANGE_UPDATE_OLD)
+ {
+ Assert(tup_old == NULL);
+ tup_old = tup;
+ }
+ else if (change.kind == CHANGE_INSERT)
+ {
+ Assert(tup_old == NULL);
+
+ apply_concurrent_insert(rel, &change, tup, iistate, index_slot);
+
+ pfree(tup);
+ }
+ else if (change.kind == CHANGE_UPDATE_NEW ||
+ change.kind == CHANGE_DELETE)
+ {
+ IndexScanDesc ind_scan = NULL;
+ HeapTuple tup_key;
+
+ if (change.kind == CHANGE_UPDATE_NEW)
+ {
+ tup_key = tup_old != NULL ? tup_old : tup;
+ }
+ else
+ {
+ Assert(tup_old == NULL);
+ tup_key = tup;
+ }
+
+ /*
+ * Find the tuple to be updated or deleted.
+ */
+ tup_exist = find_target_tuple(rel, key, nkeys, tup_key,
+ iistate, ident_slot, &ind_scan);
+ if (tup_exist == NULL)
+ elog(ERROR, "Failed to find target tuple");
+
+ if (change.kind == CHANGE_UPDATE_NEW)
+ apply_concurrent_update(rel, tup, tup_exist, &change, iistate,
+ index_slot);
+ else
+ apply_concurrent_delete(rel, tup_exist, &change);
+
+ if (tup_old != NULL)
+ {
+ pfree(tup_old);
+ tup_old = NULL;
+ }
+
+ pfree(tup);
+ index_endscan(ind_scan);
+ }
+ else
+ elog(ERROR, "Unrecognized kind of change: %d", change.kind);
+
+ /*
+ * If a change was applied now, increment CID for next writes and
+ * update the snapshot so it sees the changes we've applied so far.
+ */
+ if (change.kind != CHANGE_UPDATE_OLD)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ /* TTSOpsMinimalTuple has .get_heap_tuple==NULL. */
+ Assert(shouldFree);
+ pfree(tup_change);
+ }
+
+ tuplestore_clear(dstate->tstore);
+ dstate->nchanges = 0;
+
+ /* Cleanup. */
+ ExecDropSingleTupleTableSlot(index_slot);
+ ExecDropSingleTupleTableSlot(ident_slot);
+}
+
+static void
+apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup,
+ IndexInsertState *iistate, TupleTableSlot *index_slot)
+{
+ List *recheck;
+
+
+ /*
+ * Like simple_heap_insert(), but make sure that the INSERT is not
+ * logically decoded - see reform_and_rewrite_tuple() for more
+ * information.
+ */
+ heap_insert(rel, tup, GetCurrentCommandId(true), HEAP_INSERT_NO_LOGICAL,
+ NULL);
+
+ /*
+ * Update indexes.
+ *
+ * In case functions in the index need the active snapshot and caller
+ * hasn't set one.
+ */
+ ExecStoreHeapTuple(tup, index_slot, false);
+ recheck = ExecInsertIndexTuples(iistate->rri,
+ index_slot,
+ iistate->estate,
+ false, /* update */
+ false, /* noDupErr */
+ NULL, /* specConflict */
+ NIL, /* arbiterIndexes */
+ false /* onlySummarizing */
+ );
+
+ /*
+ * If recheck is required, it must have been preformed on the source
+ * relation by now. (All the logical changes we process here are already
+ * committed.)
+ */
+ list_free(recheck);
+
+ pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED, 1);
+}
+
+static void
+apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
+ ConcurrentChange *change, IndexInsertState *iistate,
+ TupleTableSlot *index_slot)
+{
+ LockTupleMode lockmode;
+ TM_FailureData tmfd;
+ TU_UpdateIndexes update_indexes;
+ TM_Result res;
+ List *recheck;
+
+ /*
+ * Write the new tuple into the new heap. ('tup' gets the TID assigned
+ * here.)
+ *
+ * Do it like in simple_heap_update(), except for 'wal_logical' (and
+ * except for 'wait').
+ */
+ res = heap_update(rel, &tup_target->t_self, tup,
+ GetCurrentCommandId(true),
+ InvalidSnapshot,
+ false, /* no wait - only we are doing changes */
+ &tmfd, &lockmode, &update_indexes,
+ false /* wal_logical */ );
+ if (res != TM_Ok)
+ ereport(ERROR, (errmsg("failed to apply concurrent UPDATE")));
+
+ ExecStoreHeapTuple(tup, index_slot, false);
+
+ if (update_indexes != TU_None)
+ {
+ recheck = ExecInsertIndexTuples(iistate->rri,
+ index_slot,
+ iistate->estate,
+ true, /* update */
+ false, /* noDupErr */
+ NULL, /* specConflict */
+ NIL, /* arbiterIndexes */
+ /* onlySummarizing */
+ update_indexes == TU_Summarizing);
+ list_free(recheck);
+ }
+
+ pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_UPDATED, 1);
+}
+
+static void
+apply_concurrent_delete(Relation rel, HeapTuple tup_target,
+ ConcurrentChange *change)
+{
+ TM_Result res;
+ TM_FailureData tmfd;
+
+ /*
+ * Delete tuple from the new heap.
+ *
+ * Do it like in simple_heap_delete(), except for 'wal_logical' (and
+ * except for 'wait').
+ */
+ res = heap_delete(rel, &tup_target->t_self, GetCurrentCommandId(true),
+ InvalidSnapshot, false,
+ &tmfd,
+ false, /* no wait - only we are doing changes */
+ false /* wal_logical */ );
+
+ if (res != TM_Ok)
+ ereport(ERROR, (errmsg("failed to apply concurrent DELETE")));
+
+ pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_DELETED, 1);
+}
+
+/*
+ * Find the tuple to be updated or deleted.
+ *
+ * 'key' is a pre-initialized scan key, into which the function will put the
+ * key values.
+ *
+ * 'tup_key' is a tuple containing the key values for the scan.
+ *
+ * On exit,'*scan_p' contains the scan descriptor used. The caller must close
+ * it when he no longer needs the tuple returned.
+ */
+static HeapTuple
+find_target_tuple(Relation rel, ScanKey key, int nkeys, HeapTuple tup_key,
+ IndexInsertState *iistate,
+ TupleTableSlot *ident_slot, IndexScanDesc *scan_p)
+{
+ IndexScanDesc scan;
+ Form_pg_index ident_form;
+ int2vector *ident_indkey;
+ HeapTuple result = NULL;
+
+ /* XXX no instrumentation for now */
+ scan = index_beginscan(rel, iistate->ident_index, GetActiveSnapshot(),
+ NULL, nkeys, 0);
+ *scan_p = scan;
+ index_rescan(scan, key, nkeys, NULL, 0);
+
+ /* Info needed to retrieve key values from heap tuple. */
+ ident_form = iistate->ident_index->rd_index;
+ ident_indkey = &ident_form->indkey;
+
+ /* Use the incoming tuple to finalize the scan key. */
+ for (int i = 0; i < scan->numberOfKeys; i++)
+ {
+ ScanKey entry;
+ bool isnull;
+ int16 attno_heap;
+
+ entry = &scan->keyData[i];
+ attno_heap = ident_indkey->values[i];
+ entry->sk_argument = heap_getattr(tup_key,
+ attno_heap,
+ rel->rd_att,
+ &isnull);
+ Assert(!isnull);
+ }
+ if (index_getnext_slot(scan, ForwardScanDirection, ident_slot))
+ {
+ bool shouldFree;
+
+ result = ExecFetchSlotHeapTuple(ident_slot, false, &shouldFree);
+ /* TTSOpsBufferHeapTuple has .get_heap_tuple != NULL. */
+ Assert(!shouldFree);
+ }
+
+ return result;
+}
+
+/*
+ * Decode and apply concurrent changes.
+ *
+ * Pass rel_src iff its reltoastrelid is needed.
+ */
+static void
+process_concurrent_changes(LogicalDecodingContext *ctx, XLogRecPtr end_of_wal,
+ Relation rel_dst, Relation rel_src, ScanKey ident_key,
+ int ident_key_nentries, IndexInsertState *iistate)
+{
+ RepackDecodingState *dstate;
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_CATCH_UP);
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+
+ repack_decode_concurrent_changes(ctx, end_of_wal);
+
+ if (dstate->nchanges == 0)
+ return;
+
+ PG_TRY();
+ {
+ /*
+ * Make sure that TOAST values can eventually be accessed via the old
+ * relation - see comment in copy_table_data().
+ */
+ if (rel_src)
+ rel_dst->rd_toastoid = rel_src->rd_rel->reltoastrelid;
+
+ apply_concurrent_changes(dstate, rel_dst, ident_key,
+ ident_key_nentries, iistate);
+ }
+ PG_FINALLY();
+ {
+ if (rel_src)
+ rel_dst->rd_toastoid = InvalidOid;
+ }
+ PG_END_TRY();
+}
+
+static IndexInsertState *
+get_index_insert_state(Relation relation, Oid ident_index_id)
+{
+ EState *estate;
+ int i;
+ IndexInsertState *result;
+
+ result = (IndexInsertState *) palloc0(sizeof(IndexInsertState));
+ estate = CreateExecutorState();
+
+ result->rri = (ResultRelInfo *) palloc(sizeof(ResultRelInfo));
+ InitResultRelInfo(result->rri, relation, 0, 0, 0);
+ ExecOpenIndices(result->rri, false);
+
+ /*
+ * Find the relcache entry of the identity index so that we spend no extra
+ * effort to open / close it.
+ */
+ for (i = 0; i < result->rri->ri_NumIndices; i++)
+ {
+ Relation ind_rel;
+
+ ind_rel = result->rri->ri_IndexRelationDescs[i];
+ if (ind_rel->rd_id == ident_index_id)
+ result->ident_index = ind_rel;
+ }
+ if (result->ident_index == NULL)
+ elog(ERROR, "Failed to open identity index");
+
+ /* Only initialize fields needed by ExecInsertIndexTuples(). */
+ result->estate = estate;
+
+ return result;
+}
+
+/*
+ * Build scan key to process logical changes.
+ */
+static ScanKey
+build_identity_key(Oid ident_idx_oid, Relation rel_src, int *nentries)
+{
+ Relation ident_idx_rel;
+ Form_pg_index ident_idx;
+ int n,
+ i;
+ ScanKey result;
+
+ Assert(OidIsValid(ident_idx_oid));
+ ident_idx_rel = index_open(ident_idx_oid, AccessShareLock);
+ ident_idx = ident_idx_rel->rd_index;
+ n = ident_idx->indnatts;
+ result = (ScanKey) palloc(sizeof(ScanKeyData) * n);
+ for (i = 0; i < n; i++)
+ {
+ ScanKey entry;
+ int16 relattno;
+ Form_pg_attribute att;
+ Oid opfamily,
+ opcintype,
+ opno,
+ opcode;
+
+ entry = &result[i];
+ relattno = ident_idx->indkey.values[i];
+ if (relattno >= 1)
+ {
+ TupleDesc desc;
+
+ desc = rel_src->rd_att;
+ att = TupleDescAttr(desc, relattno - 1);
+ }
+ else
+ elog(ERROR, "Unexpected attribute number %d in index", relattno);
+
+ opfamily = ident_idx_rel->rd_opfamily[i];
+ opcintype = ident_idx_rel->rd_opcintype[i];
+ opno = get_opfamily_member(opfamily, opcintype, opcintype,
+ BTEqualStrategyNumber);
+
+ if (!OidIsValid(opno))
+ elog(ERROR, "Failed to find = operator for type %u", opcintype);
+
+ opcode = get_opcode(opno);
+ if (!OidIsValid(opcode))
+ elog(ERROR, "Failed to find = operator for operator %u", opno);
+
+ /* Initialize everything but argument. */
+ ScanKeyInit(entry,
+ i + 1,
+ BTEqualStrategyNumber, opcode,
+ (Datum) NULL);
+ entry->sk_collation = att->attcollation;
+ }
+ index_close(ident_idx_rel, AccessShareLock);
+
+ *nentries = n;
+ return result;
+}
+
+static void
+free_index_insert_state(IndexInsertState *iistate)
+{
+ ExecCloseIndices(iistate->rri);
+ FreeExecutorState(iistate->estate);
+ pfree(iistate->rri);
+ pfree(iistate);
+}
+
+static void
+cleanup_logical_decoding(LogicalDecodingContext *ctx)
+{
+ RepackDecodingState *dstate;
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+
+ ExecDropSingleTupleTableSlot(dstate->tsslot);
+ FreeTupleDesc(dstate->tupdesc_change);
+ FreeTupleDesc(dstate->tupdesc);
+ tuplestore_end(dstate->tstore);
+
+ FreeDecodingContext(ctx);
+}
+
+/*
+ * The final steps of rebuild_relation() for concurrent processing.
+ *
+ * On entry, NewHeap is locked in AccessExclusiveLock mode. OldHeap and its
+ * clustering index (if one is passed) are still locked in a mode that allows
+ * concurrent data changes. On exit, both tables and their indexes are closed,
+ * but locked in AccessExclusiveLock mode.
+ */
+static void
+rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
+ Relation cl_index,
+ LogicalDecodingContext *ctx,
+ bool swap_toast_by_content,
+ TransactionId frozenXid,
+ MultiXactId cutoffMulti)
+{
+ LOCKMODE lockmode_old PG_USED_FOR_ASSERTS_ONLY;
+ List *ind_oids_new;
+ Oid old_table_oid = RelationGetRelid(OldHeap);
+ Oid new_table_oid = RelationGetRelid(NewHeap);
+ List *ind_oids_old = RelationGetIndexList(OldHeap);
+ ListCell *lc,
+ *lc2;
+ char relpersistence;
+ bool is_system_catalog;
+ Oid ident_idx_old,
+ ident_idx_new;
+ IndexInsertState *iistate;
+ ScanKey ident_key;
+ int ident_key_nentries;
+ XLogRecPtr wal_insert_ptr,
+ end_of_wal;
+ char dummy_rec_data = '\0';
+ Relation *ind_refs,
+ *ind_refs_p;
+ int nind;
+
+ /* Like in cluster_rel(). */
+ lockmode_old = ShareUpdateExclusiveLock;
+ Assert(CheckRelationLockedByMe(OldHeap, lockmode_old, false));
+ Assert(cl_index == NULL ||
+ CheckRelationLockedByMe(cl_index, lockmode_old, false));
+ /* This is expected from the caller. */
+ Assert(CheckRelationLockedByMe(NewHeap, AccessExclusiveLock, false));
+
+ ident_idx_old = RelationGetReplicaIndex(OldHeap);
+
+ /*
+ * Unlike the exclusive case, we build new indexes for the new relation
+ * rather than swapping the storage and reindexing the old relation. The
+ * point is that the index build can take some time, so we do it before we
+ * get AccessExclusiveLock on the old heap and therefore we cannot swap
+ * the heap storage yet.
+ *
+ * index_create() will lock the new indexes using AccessExclusiveLock - no
+ * need to change that.
+ *
+ * We assume that ShareUpdateExclusiveLock on the table prevents anyone
+ * from dropping the existing indexes or adding new ones, so the lists of
+ * old and new indexes should match at the swap time. On the other hand we
+ * do not block ALTER INDEX commands that do not require table lock (e.g.
+ * ALTER INDEX ... SET ...).
+ *
+ * XXX Should we check a the end of our work if another transaction
+ * executed such a command and issue a NOTICE that we might have discarded
+ * its effects? (For example, someone changes storage parameter after we
+ * have created the new index, the new value of that parameter is lost.)
+ * Alternatively, we can lock all the indexes now in a mode that blocks
+ * all the ALTER INDEX commands (ShareUpdateExclusiveLock ?), and keep
+ * them locked till the end of the transactions. That might increase the
+ * risk of deadlock during the lock upgrade below, however SELECT / DML
+ * queries should not be involved in such a deadlock.
+ */
+ ind_oids_new = build_new_indexes(NewHeap, OldHeap, ind_oids_old);
+
+ /*
+ * Processing shouldn't start w/o valid identity index.
+ */
+ Assert(OidIsValid(ident_idx_old));
+
+ /* Find "identity index" on the new relation. */
+ ident_idx_new = InvalidOid;
+ forboth(lc, ind_oids_old, lc2, ind_oids_new)
+ {
+ Oid ind_old = lfirst_oid(lc);
+ Oid ind_new = lfirst_oid(lc2);
+
+ if (ident_idx_old == ind_old)
+ {
+ ident_idx_new = ind_new;
+ break;
+ }
+ }
+ if (!OidIsValid(ident_idx_new))
+
+ /*
+ * Should not happen, given our lock on the old relation.
+ */
+ ereport(ERROR,
+ (errmsg("Identity index missing on the new relation")));
+
+ /* Executor state to update indexes. */
+ iistate = get_index_insert_state(NewHeap, ident_idx_new);
+
+ /*
+ * Build scan key that we'll use to look for rows to be updated / deleted
+ * during logical decoding.
+ */
+ ident_key = build_identity_key(ident_idx_new, OldHeap, &ident_key_nentries);
+
+ /*
+ * During testing, wait for another backend to perform concurrent data
+ * changes which we will process below.
+ */
+ INJECTION_POINT("repack-concurrently-before-lock", NULL);
+
+ /*
+ * Flush all WAL records inserted so far (possibly except for the last
+ * incomplete page, see GetInsertRecPtr), to minimize the amount of data
+ * we need to flush while holding exclusive lock on the source table.
+ */
+ wal_insert_ptr = GetInsertRecPtr();
+ XLogFlush(wal_insert_ptr);
+ end_of_wal = GetFlushRecPtr(NULL);
+
+ /*
+ * Apply concurrent changes first time, to minimize the time we need to
+ * hold AccessExclusiveLock. (Quite some amount of WAL could have been
+ * written during the data copying and index creation.)
+ */
+ process_concurrent_changes(ctx, end_of_wal, NewHeap,
+ swap_toast_by_content ? OldHeap : NULL,
+ ident_key, ident_key_nentries, iistate);
+
+ /*
+ * Acquire AccessExclusiveLock on the table, its TOAST relation (if there
+ * is one), all its indexes, so that we can swap the files.
+ *
+ * Before that, unlock the index temporarily to avoid deadlock in case
+ * another transaction is trying to lock it while holding the lock on the
+ * table.
+ */
+ if (cl_index)
+ {
+ index_close(cl_index, ShareUpdateExclusiveLock);
+ cl_index = NULL;
+ }
+ /* For the same reason, unlock TOAST relation. */
+ if (OldHeap->rd_rel->reltoastrelid)
+ LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
+ /* Finally lock the table */
+ LockRelationOid(old_table_oid, AccessExclusiveLock);
+
+ /*
+ * Lock all indexes now, not only the clustering one: all indexes need to
+ * have their files swapped. While doing that, store their relation
+ * references in an array, to handle predicate locks below.
+ */
+ ind_refs_p = ind_refs = palloc_array(Relation, list_length(ind_oids_old));
+ nind = 0;
+ foreach(lc, ind_oids_old)
+ {
+ Oid ind_oid;
+ Relation index;
+
+ ind_oid = lfirst_oid(lc);
+ index = index_open(ind_oid, AccessExclusiveLock);
+
+ /*
+ * TODO 1) Do we need to check if ALTER INDEX was executed since the
+ * new index was created in build_new_indexes()? 2) Specifically for
+ * the clustering index, should check_index_is_clusterable() be called
+ * here? (Not sure about the latter: ShareUpdateExclusiveLock on the
+ * table probably blocks all commands that affect the result of
+ * check_index_is_clusterable().)
+ */
+ *ind_refs_p = index;
+ ind_refs_p++;
+ nind++;
+ }
+
+ /*
+ * In addition, lock the OldHeap's TOAST relation exclusively - again, the
+ * lock is needed to swap the files.
+ */
+ if (OidIsValid(OldHeap->rd_rel->reltoastrelid))
+ LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
+
+ /*
+ * Tuples and pages of the old heap will be gone, but the heap will stay.
+ */
+ TransferPredicateLocksToHeapRelation(OldHeap);
+ /* The same for indexes. */
+ for (int i = 0; i < nind; i++)
+ {
+ Relation index = ind_refs[i];
+
+ TransferPredicateLocksToHeapRelation(index);
+
+ /*
+ * References to indexes on the old relation are not needed anymore,
+ * however locks stay till the end of the transaction.
+ */
+ index_close(index, NoLock);
+ }
+ pfree(ind_refs);
+
+ /*
+ * Flush anything we see in WAL, to make sure that all changes committed
+ * while we were waiting for the exclusive lock are available for
+ * decoding. This should not be necessary if all backends had
+ * synchronous_commit set, but we can't rely on this setting.
+ *
+ * Unfortunately, GetInsertRecPtr() may lag behind the actual insert
+ * position, and GetLastImportantRecPtr() points at the start of the last
+ * record rather than at the end. Thus the simplest way to determine the
+ * insert position is to insert a dummy record and use its LSN.
+ *
+ * XXX Consider using GetLastImportantRecPtr() and adding the size of the
+ * last record (plus the total size of all the page headers the record
+ * spans)?
+ */
+ XLogBeginInsert();
+ XLogRegisterData(&dummy_rec_data, 1);
+ wal_insert_ptr = XLogInsert(RM_XLOG_ID, XLOG_NOOP);
+ XLogFlush(wal_insert_ptr);
+ end_of_wal = GetFlushRecPtr(NULL);
+
+ /* Apply the concurrent changes again. */
+ process_concurrent_changes(ctx, end_of_wal, NewHeap,
+ swap_toast_by_content ? OldHeap : NULL,
+ ident_key, ident_key_nentries, iistate);
+
+ /* Remember info about rel before closing OldHeap */
+ relpersistence = OldHeap->rd_rel->relpersistence;
+ is_system_catalog = IsSystemRelation(OldHeap);
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+
+ /*
+ * Even ShareUpdateExclusiveLock should have prevented others from
+ * creating / dropping indexes (even using the CONCURRENTLY option), so we
+ * do not need to check whether the lists match.
+ */
+ forboth(lc, ind_oids_old, lc2, ind_oids_new)
+ {
+ Oid ind_old = lfirst_oid(lc);
+ Oid ind_new = lfirst_oid(lc2);
+ Oid mapped_tables[4];
+
+ /* Zero out possible results from swapped_relation_files */
+ memset(mapped_tables, 0, sizeof(mapped_tables));
+
+ swap_relation_files(ind_old, ind_new,
+ (old_table_oid == RelationRelationId),
+ swap_toast_by_content,
+ true,
+ InvalidTransactionId,
+ InvalidMultiXactId,
+ mapped_tables);
+
+#ifdef USE_ASSERT_CHECKING
+
+ /*
+ * Concurrent processing is not supported for system relations, so
+ * there should be no mapped tables.
+ */
+ for (int i = 0; i < 4; i++)
+ Assert(mapped_tables[i] == 0);
+#endif
+ }
+
+ /* The new indexes must be visible for deletion. */
+ CommandCounterIncrement();
+
+ /* Close the old heap but keep lock until transaction commit. */
+ table_close(OldHeap, NoLock);
+ /* Close the new heap. (We didn't have to open its indexes). */
+ table_close(NewHeap, NoLock);
+
+ /* Cleanup what we don't need anymore. (And close the identity index.) */
+ pfree(ident_key);
+ free_index_insert_state(iistate);
+
+ /*
+ * Swap the relations and their TOAST relations and TOAST indexes. This
+ * also drops the new relation and its indexes.
+ *
+ * (System catalogs are currently not supported.)
+ */
+ Assert(!is_system_catalog);
+ finish_heap_swap(old_table_oid, new_table_oid,
+ is_system_catalog,
+ swap_toast_by_content,
+ false, true, false,
+ frozenXid, cutoffMulti,
+ relpersistence);
+}
+
+/*
+ * Build indexes on NewHeap according to those on OldHeap.
+ *
+ * OldIndexes is the list of index OIDs on OldHeap.
+ *
+ * A list of OIDs of the corresponding indexes created on NewHeap is
+ * returned. The order of items does match, so we can use these arrays to swap
+ * index storage.
+ */
+static List *
+build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
+{
+ ListCell *lc;
+ List *result = NIL;
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+
+ foreach(lc, OldIndexes)
+ {
+ Oid ind_oid,
+ ind_oid_new;
+ char *newName;
+ Relation ind;
+
+ ind_oid = lfirst_oid(lc);
+ ind = index_open(ind_oid, AccessShareLock);
+
+ newName = ChooseRelationName(get_rel_name(ind_oid),
+ NULL,
+ "repacknew",
+ get_rel_namespace(ind->rd_index->indrelid),
+ false);
+ ind_oid_new = index_create_copy(NewHeap, ind_oid,
+ ind->rd_rel->reltablespace, newName,
+ false);
+ result = lappend_oid(result, ind_oid_new);
+
+ index_close(ind, AccessShareLock);
+ }
+
+ return result;
+}
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 188e26f0e6e..71b73c21ebf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -904,7 +904,7 @@ refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
static void
refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersistence)
{
- finish_heap_swap(matviewOid, OIDNewHeap, false, false, true, true,
+ finish_heap_swap(matviewOid, OIDNewHeap, false, false, true, true, true,
RecentXmin, ReadNextMultiXactId(), relpersistence);
}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 082a3575d62..c79f5b1dc0f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5989,6 +5989,7 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode,
finish_heap_swap(tab->relid, OIDNewHeap,
false, false, true,
!OidIsValid(tab->newTableSpace),
+ true,
RecentXmin,
ReadNextMultiXactId(),
persistence);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 8863ad0e8bd..6de9d0ba39d 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -125,7 +125,7 @@ static void vac_truncate_clog(TransactionId frozenXID,
TransactionId lastSaneFrozenXid,
MultiXactId lastSaneMinMulti);
static bool vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
- BufferAccessStrategy bstrategy);
+ BufferAccessStrategy bstrategy, bool isTopLevel);
static double compute_parallel_delay(void);
static VacOptValue get_vacoptval_from_boolean(DefElem *def);
static bool vac_tid_reaped(ItemPointer itemptr, void *state);
@@ -633,7 +633,8 @@ vacuum(List *relations, const VacuumParams params, BufferAccessStrategy bstrateg
if (params.options & VACOPT_VACUUM)
{
- if (!vacuum_rel(vrel->oid, vrel->relation, params, bstrategy))
+ if (!vacuum_rel(vrel->oid, vrel->relation, params, bstrategy,
+ isTopLevel))
continue;
}
@@ -1997,7 +1998,7 @@ vac_truncate_clog(TransactionId frozenXID,
*/
static bool
vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
- BufferAccessStrategy bstrategy)
+ BufferAccessStrategy bstrategy, bool isTopLevel)
{
LOCKMODE lmode;
Relation rel;
@@ -2288,7 +2289,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
cluster_rel(REPACK_COMMAND_VACUUMFULL, false, rel, InvalidOid,
- &cluster_params);
+ &cluster_params, isTopLevel);
/* cluster_rel closes the relation, but keeps lock */
rel = NULL;
@@ -2331,7 +2332,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
toast_vacuum_params.options |= VACOPT_PROCESS_MAIN;
toast_vacuum_params.toast_parent = relid;
- vacuum_rel(toast_relid, NULL, toast_vacuum_params, bstrategy);
+ vacuum_rel(toast_relid, NULL, toast_vacuum_params, bstrategy,
+ isTopLevel);
}
/*
diff --git a/src/backend/meson.build b/src/backend/meson.build
index b831a541652..5c148131217 100644
--- a/src/backend/meson.build
+++ b/src/backend/meson.build
@@ -194,5 +194,6 @@ pg_test_mod_args = pg_mod_args + {
subdir('jit/llvm')
subdir('replication/libpqwalreceiver')
subdir('replication/pgoutput')
+subdir('replication/pgoutput_repack')
subdir('snowball')
subdir('utils/mb/conversion_procs')
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index cc03f0706e9..5dc4ae58ffe 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -33,6 +33,7 @@
#include "access/xlogreader.h"
#include "access/xlogrecord.h"
#include "catalog/pg_control.h"
+#include "commands/cluster.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/message.h"
@@ -472,6 +473,88 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
TransactionId xid = XLogRecGetXid(buf->record);
SnapBuild *builder = ctx->snapshot_builder;
+ /*
+ * If the change is not intended for logical decoding, do not even
+ * establish transaction for it - REPACK CONCURRENTLY is the typical use
+ * case.
+ *
+ * First, check if REPACK CONCURRENTLY is being performed by this backend.
+ * If so, only decode data changes of the table that it is processing, and
+ * the changes of its TOAST relation.
+ *
+ * (TOAST locator should not be set unless the main is.)
+ */
+ Assert(!OidIsValid(repacked_rel_toast_locator.relNumber) ||
+ OidIsValid(repacked_rel_locator.relNumber));
+
+ if (OidIsValid(repacked_rel_locator.relNumber))
+ {
+ XLogReaderState *r = buf->record;
+ RelFileLocator locator;
+
+ /* Not all records contain the block. */
+ if (XLogRecGetBlockTagExtended(r, 0, &locator, NULL, NULL, NULL) &&
+ !RelFileLocatorEquals(locator, repacked_rel_locator) &&
+ (!OidIsValid(repacked_rel_toast_locator.relNumber) ||
+ !RelFileLocatorEquals(locator, repacked_rel_toast_locator)))
+ return;
+ }
+
+ /*
+ * Second, skip records which do not contain sufficient information for
+ * the decoding.
+ *
+ * The problem we solve here is that REPACK CONCURRENTLY generates WAL
+ * when doing changes in the new table. Those changes should not be useful
+ * for any other user (such as logical replication subscription) because
+ * the new table will eventually be dropped (after REPACK CONCURRENTLY has
+ * assigned its file to the "old table").
+ */
+ switch (info)
+ {
+ case XLOG_HEAP_INSERT:
+ {
+ xl_heap_insert *rec;
+
+ rec = (xl_heap_insert *) XLogRecGetData(buf->record);
+
+ /*
+ * This does happen when 1) raw_heap_insert marks the TOAST
+ * record as HEAP_INSERT_NO_LOGICAL, 2) REPACK CONCURRENTLY
+ * replays inserts performed by other backends.
+ */
+ if ((rec->flags & XLH_INSERT_CONTAINS_NEW_TUPLE) == 0)
+ return;
+
+ break;
+ }
+
+ case XLOG_HEAP_HOT_UPDATE:
+ case XLOG_HEAP_UPDATE:
+ {
+ xl_heap_update *rec;
+
+ rec = (xl_heap_update *) XLogRecGetData(buf->record);
+ if ((rec->flags &
+ (XLH_UPDATE_CONTAINS_NEW_TUPLE |
+ XLH_UPDATE_CONTAINS_OLD_TUPLE |
+ XLH_UPDATE_CONTAINS_OLD_KEY)) == 0)
+ return;
+
+ break;
+ }
+
+ case XLOG_HEAP_DELETE:
+ {
+ xl_heap_delete *rec;
+
+ rec = (xl_heap_delete *) XLogRecGetData(buf->record);
+ if (rec->flags & XLH_DELETE_NO_LOGICAL)
+ return;
+ break;
+ }
+ }
+
ReorderBufferProcessXid(ctx->reorder, xid, buf->origptr);
/*
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index a2f1803622c..d69229905a2 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -486,6 +486,27 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
return SnapBuildMVCCFromHistoric(snap, true);
}
+/*
+ * Build an MVCC snapshot for the initial data load performed by REPACK
+ * CONCURRENTLY command.
+ *
+ * The snapshot will only be used to scan one particular relation, which is
+ * treated like a catalog (therefore ->building_full_snapshot is not
+ * important), and the caller should already have a replication slot setup (so
+ * we do not set MyProc->xmin). XXX Do we yet need to add some restrictions?
+ */
+Snapshot
+SnapBuildInitialSnapshotForRepack(SnapBuild *builder)
+{
+ Snapshot snap;
+
+ Assert(builder->state == SNAPBUILD_CONSISTENT);
+ Assert(builder->building_full_snapshot);
+
+ snap = SnapBuildBuildSnapshot(builder);
+ return SnapBuildMVCCFromHistoric(snap, false);
+}
+
/*
* Turn a historic MVCC snapshot into an ordinary MVCC snapshot.
*
diff --git a/src/backend/replication/pgoutput_repack/Makefile b/src/backend/replication/pgoutput_repack/Makefile
new file mode 100644
index 00000000000..4efeb713b70
--- /dev/null
+++ b/src/backend/replication/pgoutput_repack/Makefile
@@ -0,0 +1,32 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for src/backend/replication/pgoutput_repack
+#
+# IDENTIFICATION
+# src/backend/replication/pgoutput_repack
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/replication/pgoutput_repack
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ $(WIN32RES) \
+ pgoutput_repack.o
+PGFILEDESC = "pgoutput_repack - logical replication output plugin for REPACK command"
+NAME = pgoutput_repack
+
+all: all-shared-lib
+
+include $(top_srcdir)/src/Makefile.shlib
+
+install: all installdirs install-lib
+
+installdirs: installdirs-lib
+
+uninstall: uninstall-lib
+
+clean distclean: clean-lib
+ rm -f $(OBJS)
diff --git a/src/backend/replication/pgoutput_repack/meson.build b/src/backend/replication/pgoutput_repack/meson.build
new file mode 100644
index 00000000000..133e865a4a0
--- /dev/null
+++ b/src/backend/replication/pgoutput_repack/meson.build
@@ -0,0 +1,18 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+pgoutput_repack_sources = files(
+ 'pgoutput_repack.c',
+)
+
+if host_system == 'windows'
+ pgoutput_repack_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pgoutput_repack',
+ '--FILEDESC', 'pgoutput_repack - logical replication output plugin for REPACK command',])
+endif
+
+pgoutput_repack = shared_module('pgoutput_repack',
+ pgoutput_repack_sources,
+ kwargs: pg_mod_args,
+)
+
+backend_targets += pgoutput_repack
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
new file mode 100644
index 00000000000..687fbbc59bb
--- /dev/null
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -0,0 +1,288 @@
+/*-------------------------------------------------------------------------
+ *
+ * pgoutput_cluster.c
+ * Logical Replication output plugin for REPACK command
+ *
+ * Copyright (c) 2012-2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/pgoutput_cluster/pgoutput_cluster.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/heaptoast.h"
+#include "commands/cluster.h"
+#include "replication/snapbuild.h"
+
+PG_MODULE_MAGIC;
+
+static void plugin_startup(LogicalDecodingContext *ctx,
+ OutputPluginOptions *opt, bool is_init);
+static void plugin_shutdown(LogicalDecodingContext *ctx);
+static void plugin_begin_txn(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn);
+static void plugin_commit_txn(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr commit_lsn);
+static void plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ Relation rel, ReorderBufferChange *change);
+static void plugin_truncate(struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, int nrelations,
+ Relation relations[],
+ ReorderBufferChange *change);
+static void store_change(LogicalDecodingContext *ctx,
+ ConcurrentChangeKind kind, HeapTuple tuple);
+
+void
+_PG_output_plugin_init(OutputPluginCallbacks *cb)
+{
+ AssertVariableIsOfType(&_PG_output_plugin_init, LogicalOutputPluginInit);
+
+ cb->startup_cb = plugin_startup;
+ cb->begin_cb = plugin_begin_txn;
+ cb->change_cb = plugin_change;
+ cb->truncate_cb = plugin_truncate;
+ cb->commit_cb = plugin_commit_txn;
+ cb->shutdown_cb = plugin_shutdown;
+}
+
+
+/* initialize this plugin */
+static void
+plugin_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
+ bool is_init)
+{
+ ctx->output_plugin_private = NULL;
+
+ /* Probably unnecessary, as we don't use the SQL interface ... */
+ opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+
+ if (ctx->output_plugin_options != NIL)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("This plugin does not expect any options")));
+ }
+}
+
+static void
+plugin_shutdown(LogicalDecodingContext *ctx)
+{
+}
+
+/*
+ * As we don't release the slot during processing of particular table, there's
+ * no room for SQL interface, even for debugging purposes. Therefore we need
+ * neither OutputPluginPrepareWrite() nor OutputPluginWrite() in the plugin
+ * callbacks. (Although we might want to write custom callbacks, this API
+ * seems to be unnecessarily generic for our purposes.)
+ */
+
+/* BEGIN callback */
+static void
+plugin_begin_txn(LogicalDecodingContext *ctx, ReorderBufferTXN *txn)
+{
+}
+
+/* COMMIT callback */
+static void
+plugin_commit_txn(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr commit_lsn)
+{
+}
+
+/*
+ * Callback for individual changed tuples
+ */
+static void
+plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ Relation relation, ReorderBufferChange *change)
+{
+ RepackDecodingState *dstate;
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+
+ /* Only interested in one particular relation. */
+ if (relation->rd_id != dstate->relid)
+ return;
+
+ /* Decode entry depending on its type */
+ switch (change->action)
+ {
+ case REORDER_BUFFER_CHANGE_INSERT:
+ {
+ HeapTuple newtuple;
+
+ newtuple = change->data.tp.newtuple != NULL ?
+ change->data.tp.newtuple : NULL;
+
+ /*
+ * Identity checks in the main function should have made this
+ * impossible.
+ */
+ if (newtuple == NULL)
+ elog(ERROR, "Incomplete insert info.");
+
+ store_change(ctx, CHANGE_INSERT, newtuple);
+ }
+ break;
+ case REORDER_BUFFER_CHANGE_UPDATE:
+ {
+ HeapTuple oldtuple,
+ newtuple;
+
+ oldtuple = change->data.tp.oldtuple != NULL ?
+ change->data.tp.oldtuple : NULL;
+ newtuple = change->data.tp.newtuple != NULL ?
+ change->data.tp.newtuple : NULL;
+
+ if (newtuple == NULL)
+ elog(ERROR, "Incomplete update info.");
+
+ if (oldtuple != NULL)
+ store_change(ctx, CHANGE_UPDATE_OLD, oldtuple);
+
+ store_change(ctx, CHANGE_UPDATE_NEW, newtuple);
+ }
+ break;
+ case REORDER_BUFFER_CHANGE_DELETE:
+ {
+ HeapTuple oldtuple;
+
+ oldtuple = change->data.tp.oldtuple ?
+ change->data.tp.oldtuple : NULL;
+
+ if (oldtuple == NULL)
+ elog(ERROR, "Incomplete delete info.");
+
+ store_change(ctx, CHANGE_DELETE, oldtuple);
+ }
+ break;
+ default:
+ /* Should not come here */
+ Assert(false);
+ break;
+ }
+}
+
+static void
+plugin_truncate(struct LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ int nrelations, Relation relations[],
+ ReorderBufferChange *change)
+{
+ RepackDecodingState *dstate;
+ int i;
+ Relation relation = NULL;
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+
+ /* Find the relation we are processing. */
+ for (i = 0; i < nrelations; i++)
+ {
+ relation = relations[i];
+
+ if (RelationGetRelid(relation) == dstate->relid)
+ break;
+ }
+
+ /* Is this truncation of another relation? */
+ if (i == nrelations)
+ return;
+
+ store_change(ctx, CHANGE_TRUNCATE, NULL);
+}
+
+/* Store concurrent data change. */
+static void
+store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
+ HeapTuple tuple)
+{
+ RepackDecodingState *dstate;
+ char *change_raw;
+ ConcurrentChange change;
+ bool flattened = false;
+ Size size;
+ Datum values[1];
+ bool isnull[1];
+ char *dst,
+ *dst_start;
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+
+ size = MAXALIGN(VARHDRSZ) + SizeOfConcurrentChange;
+
+ if (tuple)
+ {
+ /*
+ * ReorderBufferCommit() stores the TOAST chunks in its private memory
+ * context and frees them after having called apply_change().
+ * Therefore we need flat copy (including TOAST) that we eventually
+ * copy into the memory context which is available to
+ * decode_concurrent_changes().
+ */
+ if (HeapTupleHasExternal(tuple))
+ {
+ /*
+ * toast_flatten_tuple_to_datum() might be more convenient but we
+ * don't want the decompression it does.
+ */
+ tuple = toast_flatten_tuple(tuple, dstate->tupdesc);
+ flattened = true;
+ }
+
+ size += tuple->t_len;
+ }
+
+ /* XXX Isn't there any function / macro to do this? */
+ if (size >= 0x3FFFFFFF)
+ elog(ERROR, "Change is too big.");
+
+ /* Construct the change. */
+ change_raw = (char *) palloc0(size);
+ SET_VARSIZE(change_raw, size);
+
+ /*
+ * Since the varlena alignment might not be sufficient for the structure,
+ * set the fields in a local instance and remember where it should
+ * eventually be copied.
+ */
+ change.kind = kind;
+ dst_start = (char *) VARDATA(change_raw);
+
+ /* No other information is needed for TRUNCATE. */
+ if (change.kind == CHANGE_TRUNCATE)
+ {
+ memcpy(dst_start, &change, SizeOfConcurrentChange);
+ goto store;
+ }
+
+ /*
+ * Copy the tuple.
+ *
+ * CAUTION: change->tup_data.t_data must be fixed on retrieval!
+ */
+ memcpy(&change.tup_data, tuple, sizeof(HeapTupleData));
+ dst = dst_start + SizeOfConcurrentChange;
+ memcpy(dst, tuple->t_data, tuple->t_len);
+
+ /* The data has been copied. */
+ if (flattened)
+ pfree(tuple);
+
+store:
+ /* Copy the structure so it can be stored. */
+ memcpy(dst_start, &change, SizeOfConcurrentChange);
+
+ /* Store as tuple of 1 bytea column. */
+ values[0] = PointerGetDatum(change_raw);
+ isnull[0] = false;
+ tuplestore_putvalues(dstate->tstore, dstate->tupdesc_change,
+ values, isnull);
+
+ /* Accounting. */
+ dstate->nchanges++;
+
+ /* Cleanup. */
+ pfree(change_raw);
+}
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 2fa045e6b0f..e9ddf39500c 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -25,6 +25,7 @@
#include "access/xlogprefetcher.h"
#include "access/xlogrecovery.h"
#include "commands/async.h"
+#include "commands/cluster.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
diff --git a/src/backend/storage/lmgr/generate-lwlocknames.pl b/src/backend/storage/lmgr/generate-lwlocknames.pl
index cd3e43c448a..519f3953638 100644
--- a/src/backend/storage/lmgr/generate-lwlocknames.pl
+++ b/src/backend/storage/lmgr/generate-lwlocknames.pl
@@ -162,7 +162,7 @@ while (<$lwlocklist>)
die
"$wait_event_lwlocks[$lwlock_count] defined in wait_event_names.txt but "
- . " missing from lwlocklist.h"
+ . "missing from lwlocklist.h"
if $lwlock_count < scalar @wait_event_lwlocks;
die
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6fe268a8eec..d27a4c30548 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -64,6 +64,7 @@
#include "catalog/pg_type.h"
#include "catalog/schemapg.h"
#include "catalog/storage.h"
+#include "commands/cluster.h"
#include "commands/policy.h"
#include "commands/publicationcmds.h"
#include "commands/trigger.h"
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index bc7840052fe..6d46537cbe8 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -213,7 +213,6 @@ static List *exportedSnapshots = NIL;
/* Prototypes for local functions */
static void UnregisterSnapshotNoOwner(Snapshot snapshot);
-static void FreeSnapshot(Snapshot snapshot);
static void SnapshotResetXmin(void);
/* ResourceOwner callbacks to track snapshot references */
@@ -657,7 +656,7 @@ CopySnapshot(Snapshot snapshot)
* FreeSnapshot
* Free the memory associated with a snapshot.
*/
-static void
+void
FreeSnapshot(Snapshot snapshot)
{
Assert(snapshot->regd_count == 0);
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 59ff6e0923b..528fb08154a 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -4998,18 +4998,27 @@ match_previous_words(int pattern_id,
}
/* REPACK */
- else if (Matches("REPACK"))
+ else if (Matches("REPACK") || Matches("REPACK", "(*)"))
+ COMPLETE_WITH_SCHEMA_QUERY_PLUS(Query_for_list_of_clusterables,
+ "CONCURRENTLY");
+ else if (Matches("REPACK", "CONCURRENTLY"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_clusterables);
- else if (Matches("REPACK", "(*)"))
+ else if (Matches("REPACK", "(*)", "CONCURRENTLY"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_clusterables);
- /* If we have REPACK <sth>, then add "USING INDEX" */
- else if (Matches("REPACK", MatchAnyExcept("(")))
+ /* If we have REPACK [ CONCURRENTLY ] <sth>, then add "USING INDEX" */
+ else if (Matches("REPACK", MatchAnyExcept("(|CONCURRENTLY")) ||
+ Matches("REPACK", "CONCURRENTLY", MatchAnyExcept("(")))
COMPLETE_WITH("USING INDEX");
- /* If we have REPACK (*) <sth>, then add "USING INDEX" */
- else if (Matches("REPACK", "(*)", MatchAny))
+ /* If we have REPACK (*) [ CONCURRENTLY ] <sth>, then add "USING INDEX" */
+ else if (Matches("REPACK", "(*)", MatchAnyExcept("CONCURRENTLY")) ||
+ Matches("REPACK", "(*)", "CONCURRENTLY", MatchAnyExcept("(")))
COMPLETE_WITH("USING INDEX");
- /* If we have REPACK <sth> USING, then add the index as well */
- else if (Matches("REPACK", MatchAny, "USING", "INDEX"))
+
+ /*
+ * Complete ... [ (*) ] [ CONCURRENTLY ] <sth> USING INDEX, with a list of
+ * indexes for <sth>.
+ */
+ else if (TailMatches(MatchAnyExcept("(|CONCURRENTLY"), "USING", "INDEX"))
{
set_completion_reference(prev3_wd);
COMPLETE_WITH_SCHEMA_QUERY(Query_for_index_of_table);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index a2bd5a897f8..b82dd17a966 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -323,14 +323,15 @@ extern void heap_multi_insert(Relation relation, struct TupleTableSlot **slots,
BulkInsertState bistate);
extern TM_Result heap_delete(Relation relation, ItemPointer tid,
CommandId cid, Snapshot crosscheck, bool wait,
- struct TM_FailureData *tmfd, bool changingPart);
+ struct TM_FailureData *tmfd, bool changingPart,
+ bool wal_logical);
extern void heap_finish_speculative(Relation relation, ItemPointer tid);
extern void heap_abort_speculative(Relation relation, ItemPointer tid);
extern TM_Result heap_update(Relation relation, ItemPointer otid,
HeapTuple newtup,
CommandId cid, Snapshot crosscheck, bool wait,
struct TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ TU_UpdateIndexes *update_indexes, bool wal_logical);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -411,6 +412,10 @@ extern HTSV_Result HeapTupleSatisfiesVacuumHorizon(HeapTuple htup, Buffer buffer
TransactionId *dead_after);
extern void HeapTupleSetHintBits(HeapTupleHeader tuple, Buffer buffer,
uint16 infomask, TransactionId xid);
+extern bool HeapTupleMVCCInserted(HeapTuple htup, Snapshot snapshot,
+ Buffer buffer);
+extern bool HeapTupleMVCCNotDeleted(HeapTuple htup, Snapshot snapshot,
+ Buffer buffer);
extern bool HeapTupleHeaderIsOnlyLocked(HeapTupleHeader tuple);
extern bool HeapTupleIsSurelyDead(HeapTuple htup,
struct GlobalVisState *vistest);
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index 277df6b3cf0..8d4af07f840 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -104,6 +104,8 @@
#define XLH_DELETE_CONTAINS_OLD_KEY (1<<2)
#define XLH_DELETE_IS_SUPER (1<<3)
#define XLH_DELETE_IS_PARTITION_MOVE (1<<4)
+/* See heap_delete() */
+#define XLH_DELETE_NO_LOGICAL (1<<5)
/* convenience macro for checking whether any form of old tuple was logged */
#define XLH_DELETE_CONTAINS_OLD \
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1c9e802a6b1..289b64edfd9 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -22,6 +22,7 @@
#include "access/xact.h"
#include "commands/vacuum.h"
#include "executor/tuptable.h"
+#include "replication/logical.h"
#include "storage/read_stream.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -623,6 +624,8 @@ typedef struct TableAmRoutine
Relation OldIndex,
bool use_sort,
TransactionId OldestXmin,
+ Snapshot snapshot,
+ LogicalDecodingContext *decoding_ctx,
TransactionId *xid_cutoff,
MultiXactId *multi_cutoff,
double *num_tuples,
@@ -1627,6 +1630,10 @@ table_relation_copy_data(Relation rel, const RelFileLocator *newrlocator)
* not needed for the relation's AM
* - *xid_cutoff - ditto
* - *multi_cutoff - ditto
+ * - snapshot - if != NULL, ignore data changes done by transactions that this
+ * (MVCC) snapshot considers still in-progress or in the future.
+ * - decoding_ctx - logical decoding context, to capture concurrent data
+ * changes.
*
* Output parameters:
* - *xid_cutoff - rel's new relfrozenxid value, may be invalid
@@ -1639,6 +1646,8 @@ table_relation_copy_for_cluster(Relation OldTable, Relation NewTable,
Relation OldIndex,
bool use_sort,
TransactionId OldestXmin,
+ Snapshot snapshot,
+ LogicalDecodingContext *decoding_ctx,
TransactionId *xid_cutoff,
MultiXactId *multi_cutoff,
double *num_tuples,
@@ -1647,6 +1656,7 @@ table_relation_copy_for_cluster(Relation OldTable, Relation NewTable,
{
OldTable->rd_tableam->relation_copy_for_cluster(OldTable, NewTable, OldIndex,
use_sort, OldestXmin,
+ snapshot, decoding_ctx,
xid_cutoff, multi_cutoff,
num_tuples, tups_vacuumed,
tups_recently_dead);
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
index 890998d84bb..4a508c57a50 100644
--- a/src/include/commands/cluster.h
+++ b/src/include/commands/cluster.h
@@ -13,10 +13,15 @@
#ifndef CLUSTER_H
#define CLUSTER_H
+#include "nodes/execnodes.h"
#include "nodes/parsenodes.h"
#include "parser/parse_node.h"
+#include "replication/logical.h"
#include "storage/lock.h"
+#include "storage/relfilelocator.h"
#include "utils/relcache.h"
+#include "utils/resowner.h"
+#include "utils/tuplestore.h"
/* flag bits for ClusterParams->options */
@@ -25,6 +30,8 @@
#define CLUOPT_RECHECK_ISCLUSTERED 0x04 /* recheck relation state for
* indisclustered */
#define CLUOPT_ANALYZE 0x08 /* do an ANALYZE */
+#define CLUOPT_CONCURRENT 0x08 /* allow concurrent data changes */
+
/* options for CLUSTER */
typedef struct ClusterParams
@@ -33,14 +40,95 @@ typedef struct ClusterParams
} ClusterParams;
+/*
+ * The following definitions are used by REPACK CONCURRENTLY.
+ */
+
+extern RelFileLocator repacked_rel_locator;
+extern RelFileLocator repacked_rel_toast_locator;
+
+typedef enum
+{
+ CHANGE_INSERT,
+ CHANGE_UPDATE_OLD,
+ CHANGE_UPDATE_NEW,
+ CHANGE_DELETE,
+ CHANGE_TRUNCATE
+} ConcurrentChangeKind;
+
+typedef struct ConcurrentChange
+{
+ /* See the enum above. */
+ ConcurrentChangeKind kind;
+
+ /*
+ * The actual tuple.
+ *
+ * The tuple data follows the ConcurrentChange structure. Before use make
+ * sure the tuple is correctly aligned (ConcurrentChange can be stored as
+ * bytea) and that tuple->t_data is fixed.
+ */
+ HeapTupleData tup_data;
+} ConcurrentChange;
+
+#define SizeOfConcurrentChange (offsetof(ConcurrentChange, tup_data) + \
+ sizeof(HeapTupleData))
+
+/*
+ * Logical decoding state.
+ *
+ * Here we store the data changes that we decode from WAL while the table
+ * contents is being copied to a new storage. Also the necessary metadata
+ * needed to apply these changes to the table is stored here.
+ */
+typedef struct RepackDecodingState
+{
+ /* The relation whose changes we're decoding. */
+ Oid relid;
+
+ /*
+ * Decoded changes are stored here. Although we try to avoid excessive
+ * batches, it can happen that the changes need to be stored to disk. The
+ * tuplestore does this transparently.
+ */
+ Tuplestorestate *tstore;
+
+ /* The current number of changes in tstore. */
+ double nchanges;
+
+ /*
+ * Descriptor to store the ConcurrentChange structure serialized (bytea).
+ * We can't store the tuple directly because tuplestore only supports
+ * minimum tuple and we may need to transfer OID system column from the
+ * output plugin. Also we need to transfer the change kind, so it's better
+ * to put everything in the structure than to use 2 tuplestores "in
+ * parallel".
+ */
+ TupleDesc tupdesc_change;
+
+ /* Tuple descriptor needed to update indexes. */
+ TupleDesc tupdesc;
+
+ /* Slot to retrieve data from tstore. */
+ TupleTableSlot *tsslot;
+
+ ResourceOwner resowner;
+} RepackDecodingState;
+
+
+
extern void ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel);
extern void cluster_rel(RepackCommand command, bool usingindex,
- Relation OldHeap, Oid indexOid, ClusterParams *params);
+ Relation OldHeap, Oid indexOid, ClusterParams *params,
+ bool isTopLevel);
extern void check_index_is_clusterable(Relation OldHeap, Oid indexOid,
LOCKMODE lockmode);
extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal);
+extern void repack_decode_concurrent_changes(LogicalDecodingContext *ctx,
+ XLogRecPtr end_of_wal);
+
extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod,
char relpersistence, LOCKMODE lockmode);
extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
@@ -48,6 +136,7 @@ extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
bool swap_toast_by_content,
bool check_constraints,
bool is_internal,
+ bool reindex,
TransactionId frozenXid,
MultiXactId cutoffMulti,
char newrelpersistence);
diff --git a/src/include/commands/progress.h b/src/include/commands/progress.h
index 5b6639c114c..93917ad5544 100644
--- a/src/include/commands/progress.h
+++ b/src/include/commands/progress.h
@@ -59,18 +59,20 @@
/*
* Progress parameters for REPACK.
*
- * Note: Since REPACK shares some code with CLUSTER, these values are also
- * used by CLUSTER. (CLUSTER is now deprecated, so it makes little sense to
- * introduce a separate set of constants.)
+ * Note: Since REPACK shares some code with CLUSTER, (some of) these values
+ * are also used by CLUSTER. (CLUSTER is now deprecated, so it makes little
+ * sense to introduce a separate set of constants.)
*/
#define PROGRESS_REPACK_COMMAND 0
#define PROGRESS_REPACK_PHASE 1
#define PROGRESS_REPACK_INDEX_RELID 2
#define PROGRESS_REPACK_HEAP_TUPLES_SCANNED 3
-#define PROGRESS_REPACK_HEAP_TUPLES_WRITTEN 4
-#define PROGRESS_REPACK_TOTAL_HEAP_BLKS 5
-#define PROGRESS_REPACK_HEAP_BLKS_SCANNED 6
-#define PROGRESS_REPACK_INDEX_REBUILD_COUNT 7
+#define PROGRESS_REPACK_HEAP_TUPLES_INSERTED 4
+#define PROGRESS_REPACK_HEAP_TUPLES_UPDATED 5
+#define PROGRESS_REPACK_HEAP_TUPLES_DELETED 6
+#define PROGRESS_REPACK_TOTAL_HEAP_BLKS 7
+#define PROGRESS_REPACK_HEAP_BLKS_SCANNED 8
+#define PROGRESS_REPACK_INDEX_REBUILD_COUNT 9
/*
* Phases of repack (as advertised via PROGRESS_REPACK_PHASE).
@@ -79,9 +81,10 @@
#define PROGRESS_REPACK_PHASE_INDEX_SCAN_HEAP 2
#define PROGRESS_REPACK_PHASE_SORT_TUPLES 3
#define PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP 4
-#define PROGRESS_REPACK_PHASE_SWAP_REL_FILES 5
-#define PROGRESS_REPACK_PHASE_REBUILD_INDEX 6
-#define PROGRESS_REPACK_PHASE_FINAL_CLEANUP 7
+#define PROGRESS_REPACK_PHASE_CATCH_UP 5
+#define PROGRESS_REPACK_PHASE_SWAP_REL_FILES 6
+#define PROGRESS_REPACK_PHASE_REBUILD_INDEX 7
+#define PROGRESS_REPACK_PHASE_FINAL_CLEANUP 8
/*
* Commands of PROGRESS_REPACK
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index 6d4d2d1814c..802fc4b0823 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -73,6 +73,7 @@ extern void FreeSnapshotBuilder(SnapBuild *builder);
extern void SnapBuildSnapDecRefcount(Snapshot snap);
extern Snapshot SnapBuildInitialSnapshot(SnapBuild *builder);
+extern Snapshot SnapBuildInitialSnapshotForRepack(SnapBuild *builder);
extern Snapshot SnapBuildMVCCFromHistoric(Snapshot snapshot, bool in_place);
extern const char *SnapBuildExportSnapshot(SnapBuild *builder);
extern void SnapBuildClearExportedSnapshot(void);
diff --git a/src/include/storage/lockdefs.h b/src/include/storage/lockdefs.h
index 7f3ba0352f6..2739327b0da 100644
--- a/src/include/storage/lockdefs.h
+++ b/src/include/storage/lockdefs.h
@@ -36,8 +36,8 @@ typedef int LOCKMODE;
#define AccessShareLock 1 /* SELECT */
#define RowShareLock 2 /* SELECT FOR UPDATE/FOR SHARE */
#define RowExclusiveLock 3 /* INSERT, UPDATE, DELETE */
-#define ShareUpdateExclusiveLock 4 /* VACUUM (non-FULL), ANALYZE, CREATE
- * INDEX CONCURRENTLY */
+#define ShareUpdateExclusiveLock 4 /* VACUUM (non-exclusive), ANALYZE, CREATE
+ * INDEX CONCURRENTLY, REPACK CONCURRENTLY */
#define ShareLock 5 /* CREATE INDEX (WITHOUT CONCURRENTLY) */
#define ShareRowExclusiveLock 6 /* like EXCLUSIVE MODE, but allows ROW
* SHARE */
diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h
index f65f83c85cd..1f821fd2ccd 100644
--- a/src/include/utils/snapmgr.h
+++ b/src/include/utils/snapmgr.h
@@ -64,6 +64,8 @@ extern Snapshot GetLatestSnapshot(void);
extern void SnapshotSetCommandId(CommandId curcid);
extern Snapshot CopySnapshot(Snapshot snapshot);
+extern void FreeSnapshot(Snapshot snapshot);
+
extern Snapshot GetCatalogSnapshot(Oid relid);
extern Snapshot GetNonHistoricCatalogSnapshot(Oid relid);
extern void InvalidateCatalogSnapshot(void);
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index fc82cd67f6c..f16422175f8 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -11,10 +11,11 @@ EXTENSION = injection_points
DATA = injection_points--1.0.sql
PGFILEDESC = "injection_points - facility for injection points"
-REGRESS = injection_points hashagg reindex_conc vacuum
+# REGRESS = injection_points hashagg reindex_conc vacuum
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace syscache-update-pruned
+ISOLATION = basic inplace syscache-update-pruned repack
+ISOLATION_OPTS = --temp-config $(top_srcdir)/src/test/modules/injection_points/logical.conf
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/repack.out b/src/test/modules/injection_points/expected/repack.out
new file mode 100644
index 00000000000..b575e9052ee
--- /dev/null
+++ b/src/test/modules/injection_points/expected/repack.out
@@ -0,0 +1,113 @@
+Parsed test spec with 2 sessions
+
+starting permutation: wait_before_lock change_existing change_new change_subxact1 change_subxact2 check2 wakeup_before_lock check1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step change_existing:
+ UPDATE repack_test SET i=10 where i=1;
+ UPDATE repack_test SET j=20 where i=2;
+ UPDATE repack_test SET i=30 where i=3;
+ UPDATE repack_test SET i=40 where i=30;
+ DELETE FROM repack_test WHERE i=4;
+
+step change_new:
+ INSERT INTO repack_test(i, j) VALUES (5, 5), (6, 6), (7, 7), (8, 8);
+ UPDATE repack_test SET i=50 where i=5;
+ UPDATE repack_test SET j=60 where i=6;
+ DELETE FROM repack_test WHERE i=7;
+
+step change_subxact1:
+ BEGIN;
+ INSERT INTO repack_test(i, j) VALUES (100, 100);
+ SAVEPOINT s1;
+ UPDATE repack_test SET i=101 where i=100;
+ SAVEPOINT s2;
+ UPDATE repack_test SET i=102 where i=101;
+ COMMIT;
+
+step change_subxact2:
+ BEGIN;
+ SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 110);
+ ROLLBACK TO SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 111);
+ COMMIT;
+
+step check2:
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s2(i, j)
+ SELECT i, j FROM repack_test;
+
+ i| j
+---+---
+ 2| 20
+ 6| 60
+ 8| 8
+ 10| 1
+ 40| 3
+ 50| 5
+102|100
+110|111
+(8 rows)
+
+step wakeup_before_lock:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step check1:
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT count(DISTINCT node) FROM relfilenodes;
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s1(i, j)
+ SELECT i, j FROM repack_test;
+
+ SELECT count(*)
+ FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j)
+ WHERE d1.i ISNULL OR d2.i ISNULL;
+
+count
+-----
+ 2
+(1 row)
+
+ i| j
+---+---
+ 2| 20
+ 6| 60
+ 8| 8
+ 10| 1
+ 40| 3
+ 50| 5
+102|100
+110|111
+(8 rows)
+
+count
+-----
+ 0
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
diff --git a/src/test/modules/injection_points/logical.conf b/src/test/modules/injection_points/logical.conf
new file mode 100644
index 00000000000..c8f264bc6cb
--- /dev/null
+++ b/src/test/modules/injection_points/logical.conf
@@ -0,0 +1 @@
+wal_level = logical
\ No newline at end of file
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 20390d6b4bf..29561103bbf 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -47,9 +47,13 @@ tests += {
'specs': [
'basic',
'inplace',
+ 'repack',
'syscache-update-pruned',
],
'runningcheck': false, # see syscache-update-pruned
+ # 'repack' requires wal_level = 'logical'.
+ 'regress_args': ['--temp-config', files('logical.conf')],
+
},
'tap': {
'env': {
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
new file mode 100644
index 00000000000..75850334986
--- /dev/null
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -0,0 +1,143 @@
+# Prefix the system columns with underscore as they are not allowed as column
+# names.
+setup
+{
+ CREATE EXTENSION injection_points;
+
+ CREATE TABLE repack_test(i int PRIMARY KEY, j int);
+ INSERT INTO repack_test(i, j) VALUES (1, 1), (2, 2), (3, 3), (4, 4);
+
+ CREATE TABLE relfilenodes(node oid);
+
+ CREATE TABLE data_s1(i int, j int);
+ CREATE TABLE data_s2(i int, j int);
+}
+
+teardown
+{
+ DROP TABLE repack_test;
+ DROP EXTENSION injection_points;
+
+ DROP TABLE relfilenodes;
+ DROP TABLE data_s1;
+ DROP TABLE data_s2;
+}
+
+session s1
+setup
+{
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('repack-concurrently-before-lock', 'wait');
+}
+# Perform the initial load and wait for s2 to do some data changes.
+step wait_before_lock
+{
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+}
+# Check the table from the perspective of s1.
+#
+# Besides the contents, we also check that relfilenode has changed.
+
+# Have each session write the contents into a table and use FULL JOIN to check
+# if the outputs are identical.
+step check1
+{
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT count(DISTINCT node) FROM relfilenodes;
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s1(i, j)
+ SELECT i, j FROM repack_test;
+
+ SELECT count(*)
+ FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j)
+ WHERE d1.i ISNULL OR d2.i ISNULL;
+}
+teardown
+{
+ SELECT injection_points_detach('repack-concurrently-before-lock');
+}
+
+session s2
+# Change the existing data. UPDATE changes both key and non-key columns. Also
+# update one row twice to test whether tuple version generated by this session
+# can be found.
+step change_existing
+{
+ UPDATE repack_test SET i=10 where i=1;
+ UPDATE repack_test SET j=20 where i=2;
+ UPDATE repack_test SET i=30 where i=3;
+ UPDATE repack_test SET i=40 where i=30;
+ DELETE FROM repack_test WHERE i=4;
+}
+# Insert new rows and UPDATE / DELETE some of them. Again, update both key and
+# non-key column.
+step change_new
+{
+ INSERT INTO repack_test(i, j) VALUES (5, 5), (6, 6), (7, 7), (8, 8);
+ UPDATE repack_test SET i=50 where i=5;
+ UPDATE repack_test SET j=60 where i=6;
+ DELETE FROM repack_test WHERE i=7;
+}
+
+# When applying concurrent data changes, we should see the effects of an
+# in-progress subtransaction.
+#
+# XXX Not sure this test is useful now - it was designed for the patch that
+# preserves tuple visibility and which therefore modifies
+# TransactionIdIsCurrentTransactionId().
+step change_subxact1
+{
+ BEGIN;
+ INSERT INTO repack_test(i, j) VALUES (100, 100);
+ SAVEPOINT s1;
+ UPDATE repack_test SET i=101 where i=100;
+ SAVEPOINT s2;
+ UPDATE repack_test SET i=102 where i=101;
+ COMMIT;
+}
+
+# When applying concurrent data changes, we should not see the effects of a
+# rolled back subtransaction.
+#
+# XXX Is this test useful? See above.
+step change_subxact2
+{
+ BEGIN;
+ SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 110);
+ ROLLBACK TO SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 111);
+ COMMIT;
+}
+
+# Check the table from the perspective of s2.
+step check2
+{
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s2(i, j)
+ SELECT i, j FROM repack_test;
+}
+step wakeup_before_lock
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
+
+# Test if data changes introduced while one session is performing REPACK
+# CONCURRENTLY find their way into the table.
+permutation
+ wait_before_lock
+ change_existing
+ change_new
+ change_subxact1
+ change_subxact2
+ check2
+ wakeup_before_lock
+ check1
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3a1d1d28282..fe227bd8a30 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1999,17 +1999,17 @@ pg_stat_progress_cluster| SELECT s.pid,
WHEN 2 THEN 'index scanning heap'::text
WHEN 3 THEN 'sorting tuples'::text
WHEN 4 THEN 'writing new heap'::text
- WHEN 5 THEN 'swapping relation files'::text
- WHEN 6 THEN 'rebuilding index'::text
- WHEN 7 THEN 'performing final cleanup'::text
+ WHEN 6 THEN 'swapping relation files'::text
+ WHEN 7 THEN 'rebuilding index'::text
+ WHEN 8 THEN 'performing final cleanup'::text
ELSE NULL::text
END AS phase,
(s.param3)::oid AS cluster_index_relid,
s.param4 AS heap_tuples_scanned,
s.param5 AS heap_tuples_written,
- s.param6 AS heap_blks_total,
- s.param7 AS heap_blks_scanned,
- s.param8 AS index_rebuild_count
+ s.param8 AS heap_blks_total,
+ s.param9 AS heap_blks_scanned,
+ s.param10 AS index_rebuild_count
FROM (pg_stat_get_progress_info('CLUSTER'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20)
LEFT JOIN pg_database d ON ((s.datid = d.oid)));
pg_stat_progress_copy| SELECT s.pid,
@@ -2081,17 +2081,20 @@ pg_stat_progress_repack| SELECT s.pid,
WHEN 2 THEN 'index scanning heap'::text
WHEN 3 THEN 'sorting tuples'::text
WHEN 4 THEN 'writing new heap'::text
- WHEN 5 THEN 'swapping relation files'::text
- WHEN 6 THEN 'rebuilding index'::text
- WHEN 7 THEN 'performing final cleanup'::text
+ WHEN 5 THEN 'catch-up'::text
+ WHEN 6 THEN 'swapping relation files'::text
+ WHEN 7 THEN 'rebuilding index'::text
+ WHEN 8 THEN 'performing final cleanup'::text
ELSE NULL::text
END AS phase,
(s.param3)::oid AS repack_index_relid,
s.param4 AS heap_tuples_scanned,
- s.param5 AS heap_tuples_written,
- s.param6 AS heap_blks_total,
- s.param7 AS heap_blks_scanned,
- s.param8 AS index_rebuild_count
+ s.param5 AS heap_tuples_inserted,
+ s.param6 AS heap_tuples_updated,
+ s.param7 AS heap_tuples_deleted,
+ s.param8 AS heap_blks_total,
+ s.param9 AS heap_blks_scanned,
+ s.param10 AS index_rebuild_count
FROM (pg_stat_get_progress_info('REPACK'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20)
LEFT JOIN pg_database d ON ((s.datid = d.oid)));
pg_stat_progress_vacuum| SELECT s.pid,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 98242e25432..b64ab8dfab4 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -485,6 +485,8 @@ CompressFileHandle
CompressionLocation
CompressorState
ComputeXidHorizonsResult
+ConcurrentChange
+ConcurrentChangeKind
ConditionVariable
ConditionVariableMinimallyPadded
ConditionalStack
@@ -1257,6 +1259,7 @@ IndexElem
IndexFetchHeapData
IndexFetchTableData
IndexInfo
+IndexInsertState
IndexList
IndexOnlyScan
IndexOnlyScanState
@@ -2538,6 +2541,7 @@ ReorderBufferUpdateProgressTxnCB
ReorderTuple
RepOriginId
RepackCommand
+RepackDecodingState
RepackStmt
ReparameterizeForeignPathByChild_function
ReplaceVarsFromTargetList_context
--
2.43.0
[application/octet-stream] v21-0002-Add-REPACK-command.patch (133.3K, ../../CADzfLwUgPMLiFkXRnk97ugPqkDfsNJ3TRdw9gjJM=8WB4_nXwQ@mail.gmail.com/5-v21-0002-Add-REPACK-command.patch)
download | inline diff:
From 40965dfef0f26a92249cda7a956bd03c9358a026 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
Date: Sat, 26 Jul 2025 19:57:26 +0200
Subject: [PATCH v21 2/6] Add REPACK command
REPACK absorbs the functionality of VACUUM FULL and CLUSTER in a single
command. Because this functionality is completely different from
regular VACUUM, having it separate from VACUUM makes it easier for users
to understand; as for CLUSTER, the term is heavily overloaded in the
TI world and even in Postgres itself, so it's good that we can avoid it.
This also adds pg_repackdb, a new utility that can invoke the new
commands. This is heavily based on vacuumdb. We may still change the
implementation, depending on how does Windows like this one.
Author: Antonin Houska <ah@cybertec.at>
Reviewed-by: To fill in
Discussion: https://postgr.es/m/82651.1720540558@antos
Discussion: https://postgr.es/m/202507262156.sb455angijk6@alvherre.pgsql
---
doc/src/sgml/monitoring.sgml | 223 ++++++-
doc/src/sgml/ref/allfiles.sgml | 2 +
doc/src/sgml/ref/cluster.sgml | 97 +--
doc/src/sgml/ref/clusterdb.sgml | 5 +
doc/src/sgml/ref/pg_repackdb.sgml | 479 ++++++++++++++
doc/src/sgml/ref/repack.sgml | 284 +++++++++
doc/src/sgml/ref/vacuum.sgml | 33 +-
doc/src/sgml/reference.sgml | 2 +
src/backend/access/heap/heapam_handler.c | 32 +-
src/backend/catalog/index.c | 2 +-
src/backend/catalog/system_views.sql | 26 +
src/backend/commands/cluster.c | 758 +++++++++++++++--------
src/backend/commands/vacuum.c | 3 +-
src/backend/parser/gram.y | 88 ++-
src/backend/tcop/utility.c | 20 +-
src/backend/utils/adt/pgstatfuncs.c | 2 +
src/bin/psql/tab-complete.in.c | 33 +-
src/bin/scripts/Makefile | 4 +-
src/bin/scripts/meson.build | 2 +
src/bin/scripts/pg_repackdb.c | 226 +++++++
src/bin/scripts/t/103_repackdb.pl | 24 +
src/bin/scripts/vacuuming.c | 60 +-
src/bin/scripts/vacuuming.h | 11 +-
src/include/commands/cluster.h | 8 +-
src/include/commands/progress.h | 61 +-
src/include/nodes/parsenodes.h | 20 +-
src/include/parser/kwlist.h | 1 +
src/include/tcop/cmdtaglist.h | 1 +
src/include/utils/backend_progress.h | 1 +
src/test/regress/expected/cluster.out | 125 +++-
src/test/regress/expected/rules.out | 23 +
src/test/regress/sql/cluster.sql | 59 ++
src/tools/pgindent/typedefs.list | 3 +
33 files changed, 2271 insertions(+), 447 deletions(-)
create mode 100644 doc/src/sgml/ref/pg_repackdb.sgml
create mode 100644 doc/src/sgml/ref/repack.sgml
create mode 100644 src/bin/scripts/pg_repackdb.c
create mode 100644 src/bin/scripts/t/103_repackdb.pl
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3f4a27a736e..12e103d319d 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -405,6 +405,14 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</entry>
</row>
+ <row>
+ <entry><structname>pg_stat_progress_repack</structname><indexterm><primary>pg_stat_progress_repack</primary></indexterm></entry>
+ <entry>One row for each backend running
+ <command>REPACK</command>, showing current progress. See
+ <xref linkend="repack-progress-reporting"/>.
+ </entry>
+ </row>
+
<row>
<entry><structname>pg_stat_progress_basebackup</structname><indexterm><primary>pg_stat_progress_basebackup</primary></indexterm></entry>
<entry>One row for each WAL sender process streaming a base backup,
@@ -5506,7 +5514,8 @@ FROM pg_stat_get_backend_idset() AS backendid;
certain commands during command execution. Currently, the only commands
which support progress reporting are <command>ANALYZE</command>,
<command>CLUSTER</command>,
- <command>CREATE INDEX</command>, <command>VACUUM</command>,
+ <command>CREATE INDEX</command>, <command>REPACK</command>,
+ <command>VACUUM</command>,
<command>COPY</command>,
and <xref linkend="protocol-replication-base-backup"/> (i.e., replication
command that <xref linkend="app-pgbasebackup"/> issues to take
@@ -5965,6 +5974,218 @@ FROM pg_stat_get_backend_idset() AS backendid;
</table>
</sect2>
+ <sect2 id="repack-progress-reporting">
+ <title>REPACK Progress Reporting</title>
+
+ <indexterm>
+ <primary>pg_stat_progress_repack</primary>
+ </indexterm>
+
+ <para>
+ Whenever <command>REPACK</command> is running,
+ the <structname>pg_stat_progress_repack</structname> view will contain a
+ row for each backend that is currently running the command. The tables
+ below describe the information that will be reported and provide
+ information about how to interpret it.
+ </para>
+
+ <table id="pg-stat-progress-repack-view" xreflabel="pg_stat_progress_repack">
+ <title><structname>pg_stat_progress_repack</structname> View</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>pid</structfield> <type>integer</type>
+ </para>
+ <para>
+ Process ID of backend.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>datid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of the database to which this backend is connected.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>datname</structfield> <type>name</type>
+ </para>
+ <para>
+ Name of the database to which this backend is connected.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ OID of the table being repacked.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>phase</structfield> <type>text</type>
+ </para>
+ <para>
+ Current processing phase. See <xref linkend="repack-phases"/>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>repack_index_relid</structfield> <type>oid</type>
+ </para>
+ <para>
+ If the table is being scanned using an index, this is the OID of the
+ index being used; otherwise, it is zero.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_tuples_scanned</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of heap tuples scanned.
+ This counter only advances when the phase is
+ <literal>seq scanning heap</literal>,
+ <literal>index scanning heap</literal>
+ or <literal>writing new heap</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_tuples_written</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of heap tuples written.
+ This counter only advances when the phase is
+ <literal>seq scanning heap</literal>,
+ <literal>index scanning heap</literal>
+ or <literal>writing new heap</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_blks_total</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Total number of heap blocks in the table. This number is reported
+ as of the beginning of <literal>seq scanning heap</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_blks_scanned</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of heap blocks scanned. This counter only advances when the
+ phase is <literal>seq scanning heap</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>index_rebuild_count</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of indexes rebuilt. This counter only advances when the phase
+ is <literal>rebuilding index</literal>.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <table id="repack-phases">
+ <title>REPACK Phases</title>
+ <tgroup cols="2">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Phase</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><literal>initializing</literal></entry>
+ <entry>
+ The command is preparing to begin scanning the heap. This phase is
+ expected to be very brief.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>seq scanning heap</literal></entry>
+ <entry>
+ The command is currently scanning the table using a sequential scan.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>index scanning heap</literal></entry>
+ <entry>
+ <command>REPACK</command> is currently scanning the table using an index scan.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>sorting tuples</literal></entry>
+ <entry>
+ <command>REPACK</command> is currently sorting tuples.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>writing new heap</literal></entry>
+ <entry>
+ <command>REPACK</command> is currently writing the new heap.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>swapping relation files</literal></entry>
+ <entry>
+ The command is currently swapping newly-built files into place.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>rebuilding index</literal></entry>
+ <entry>
+ The command is currently rebuilding an index.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>performing final cleanup</literal></entry>
+ <entry>
+ The command is performing final cleanup. When this phase is
+ completed, <command>REPACK</command> will end.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
<sect2 id="copy-progress-reporting">
<title>COPY Progress Reporting</title>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index f5be638867a..eabf92e3536 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -167,6 +167,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY refreshMaterializedView SYSTEM "refresh_materialized_view.sgml">
<!ENTITY reindex SYSTEM "reindex.sgml">
<!ENTITY releaseSavepoint SYSTEM "release_savepoint.sgml">
+<!ENTITY repack SYSTEM "repack.sgml">
<!ENTITY reset SYSTEM "reset.sgml">
<!ENTITY revoke SYSTEM "revoke.sgml">
<!ENTITY rollback SYSTEM "rollback.sgml">
@@ -212,6 +213,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgIsready SYSTEM "pg_isready.sgml">
<!ENTITY pgReceivewal SYSTEM "pg_receivewal.sgml">
<!ENTITY pgRecvlogical SYSTEM "pg_recvlogical.sgml">
+<!ENTITY pgRepackdb SYSTEM "pg_repackdb.sgml">
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
<!ENTITY pgRestore SYSTEM "pg_restore.sgml">
<!ENTITY pgRewind SYSTEM "pg_rewind.sgml">
diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml
index 8811f169ea0..cfcfb65e349 100644
--- a/doc/src/sgml/ref/cluster.sgml
+++ b/doc/src/sgml/ref/cluster.sgml
@@ -33,51 +33,13 @@ CLUSTER [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <r
<title>Description</title>
<para>
- <command>CLUSTER</command> instructs <productname>PostgreSQL</productname>
- to cluster the table specified
- by <replaceable class="parameter">table_name</replaceable>
- based on the index specified by
- <replaceable class="parameter">index_name</replaceable>. The index must
- already have been defined on
- <replaceable class="parameter">table_name</replaceable>.
+ The <command>CLUSTER</command> command is equivalent to
+ <xref linkend="sql-repack"/> with an <literal>USING INDEX</literal>
+ clause. See there for more details.
</para>
- <para>
- When a table is clustered, it is physically reordered
- based on the index information. Clustering is a one-time operation:
- when the table is subsequently updated, the changes are
- not clustered. That is, no attempt is made to store new or
- updated rows according to their index order. (If one wishes, one can
- periodically recluster by issuing the command again. Also, setting
- the table's <literal>fillfactor</literal> storage parameter to less than
- 100% can aid in preserving cluster ordering during updates, since updated
- rows are kept on the same page if enough space is available there.)
- </para>
-
- <para>
- When a table is clustered, <productname>PostgreSQL</productname>
- remembers which index it was clustered by. The form
- <command>CLUSTER <replaceable class="parameter">table_name</replaceable></command>
- reclusters the table using the same index as before. You can also
- use the <literal>CLUSTER</literal> or <literal>SET WITHOUT CLUSTER</literal>
- forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link> to set the index to be used for
- future cluster operations, or to clear any previous setting.
- </para>
+<!-- Do we need to describe exactly which options map to what? They seem obvious to me. -->
- <para>
- <command>CLUSTER</command> without a
- <replaceable class="parameter">table_name</replaceable> reclusters all the
- previously-clustered tables in the current database that the calling user
- has privileges for. This form of <command>CLUSTER</command> cannot be
- executed inside a transaction block.
- </para>
-
- <para>
- When a table is being clustered, an <literal>ACCESS
- EXCLUSIVE</literal> lock is acquired on it. This prevents any other
- database operations (both reads and writes) from operating on the
- table until the <command>CLUSTER</command> is finished.
- </para>
</refsect1>
<refsect1>
@@ -136,63 +98,12 @@ CLUSTER [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <r
on the table.
</para>
- <para>
- In cases where you are accessing single rows randomly
- within a table, the actual order of the data in the
- table is unimportant. However, if you tend to access some
- data more than others, and there is an index that groups
- them together, you will benefit from using <command>CLUSTER</command>.
- If you are requesting a range of indexed values from a table, or a
- single indexed value that has multiple rows that match,
- <command>CLUSTER</command> will help because once the index identifies the
- table page for the first row that matches, all other rows
- that match are probably already on the same table page,
- and so you save disk accesses and speed up the query.
- </para>
-
- <para>
- <command>CLUSTER</command> can re-sort the table using either an index scan
- on the specified index, or (if the index is a b-tree) a sequential
- scan followed by sorting. It will attempt to choose the method that
- will be faster, based on planner cost parameters and available statistical
- information.
- </para>
-
<para>
While <command>CLUSTER</command> is running, the <xref
linkend="guc-search-path"/> is temporarily changed to <literal>pg_catalog,
pg_temp</literal>.
</para>
- <para>
- When an index scan is used, a temporary copy of the table is created that
- contains the table data in the index order. Temporary copies of each
- index on the table are created as well. Therefore, you need free space on
- disk at least equal to the sum of the table size and the index sizes.
- </para>
-
- <para>
- When a sequential scan and sort is used, a temporary sort file is
- also created, so that the peak temporary space requirement is as much
- as double the table size, plus the index sizes. This method is often
- faster than the index scan method, but if the disk space requirement is
- intolerable, you can disable this choice by temporarily setting <xref
- linkend="guc-enable-sort"/> to <literal>off</literal>.
- </para>
-
- <para>
- It is advisable to set <xref linkend="guc-maintenance-work-mem"/> to
- a reasonably large value (but not more than the amount of RAM you can
- dedicate to the <command>CLUSTER</command> operation) before clustering.
- </para>
-
- <para>
- Because the planner records statistics about the ordering of
- tables, it is advisable to run <link linkend="sql-analyze"><command>ANALYZE</command></link>
- on the newly clustered table.
- Otherwise, the planner might make poor choices of query plans.
- </para>
-
<para>
Because <command>CLUSTER</command> remembers which indexes are clustered,
one can cluster the tables one wants clustered manually the first time,
diff --git a/doc/src/sgml/ref/clusterdb.sgml b/doc/src/sgml/ref/clusterdb.sgml
index 0d2051bf6f1..546c1289c31 100644
--- a/doc/src/sgml/ref/clusterdb.sgml
+++ b/doc/src/sgml/ref/clusterdb.sgml
@@ -64,6 +64,11 @@ PostgreSQL documentation
this utility and via other methods for accessing the server.
</para>
+ <para>
+ <application>clusterdb</application> has been superceded by
+ <application>pg_repackdb</application>.
+ </para>
+
</refsect1>
diff --git a/doc/src/sgml/ref/pg_repackdb.sgml b/doc/src/sgml/ref/pg_repackdb.sgml
new file mode 100644
index 00000000000..32570d071cb
--- /dev/null
+++ b/doc/src/sgml/ref/pg_repackdb.sgml
@@ -0,0 +1,479 @@
+<!--
+doc/src/sgml/ref/pg_repackdb.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgrepackdb">
+ <indexterm zone="app-pgrepackdb">
+ <primary>pg_repackdb</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle><application>pg_repackdb</application></refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_repackdb</refname>
+ <refpurpose>repack and analyze a <productname>PostgreSQL</productname>
+ database</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_repackdb</command>
+ <arg rep="repeat"><replaceable>connection-option</replaceable></arg>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+
+ <arg choice="plain" rep="repeat">
+ <arg choice="opt">
+ <group choice="plain">
+ <arg choice="plain"><option>-t</option></arg>
+ <arg choice="plain"><option>--table</option></arg>
+ </group>
+ <replaceable>table</replaceable>
+ <arg choice="opt">( <replaceable class="parameter">column</replaceable> [,...] )</arg>
+ </arg>
+ </arg>
+
+ <arg choice="opt">
+ <group choice="plain">
+ <arg choice="plain"><replaceable>dbname</replaceable></arg>
+ <arg choice="plain"><option>-a</option></arg>
+ <arg choice="plain"><option>--all</option></arg>
+ </group>
+ </arg>
+ </cmdsynopsis>
+
+ <cmdsynopsis>
+ <command>pg_repackdb</command>
+ <arg rep="repeat"><replaceable>connection-option</replaceable></arg>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+
+ <arg choice="plain" rep="repeat">
+ <arg choice="opt">
+ <group choice="plain">
+ <arg choice="plain"><option>-n</option></arg>
+ <arg choice="plain"><option>--schema</option></arg>
+ </group>
+ <replaceable>schema</replaceable>
+ </arg>
+ </arg>
+
+ <arg choice="opt">
+ <group choice="plain">
+ <arg choice="plain"><replaceable>dbname</replaceable></arg>
+ <arg choice="plain"><option>-a</option></arg>
+ <arg choice="plain"><option>--all</option></arg>
+ </group>
+ </arg>
+ </cmdsynopsis>
+
+ <cmdsynopsis>
+ <command>pg_repackdb</command>
+ <arg rep="repeat"><replaceable>connection-option</replaceable></arg>
+ <arg rep="repeat"><replaceable>option</replaceable></arg>
+
+ <arg choice="plain" rep="repeat">
+ <arg choice="opt">
+ <group choice="plain">
+ <arg choice="plain"><option>-N</option></arg>
+ <arg choice="plain"><option>--exclude-schema</option></arg>
+ </group>
+ <replaceable>schema</replaceable>
+ </arg>
+ </arg>
+
+ <arg choice="opt">
+ <group choice="plain">
+ <arg choice="plain"><replaceable>dbname</replaceable></arg>
+ <arg choice="plain"><option>-a</option></arg>
+ <arg choice="plain"><option>--all</option></arg>
+ </group>
+ </arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <application>pg_repackdb</application> is a utility for repacking a
+ <productname>PostgreSQL</productname> database.
+ <application>pg_repackdb</application> will also generate internal
+ statistics used by the <productname>PostgreSQL</productname> query
+ optimizer.
+ </para>
+
+ <para>
+ <application>pg_repackdb</application> is a wrapper around the SQL
+ command <link linkend="sql-repack"><command>REPACK</command></link> There
+ is no effective difference between repacking and analyzing databases via
+ this utility and via other methods for accessing the server.
+ </para>
+
+ </refsect1>
+
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ <application>pg_repackdb</application> accepts the following command-line arguments:
+ <variablelist>
+ <varlistentry>
+ <term><option>-a</option></term>
+ <term><option>--all</option></term>
+ <listitem>
+ <para>
+ Repack all databases.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option><optional>-d</optional> <replaceable class="parameter">dbname</replaceable></option></term>
+ <term><option><optional>--dbname=</optional><replaceable class="parameter">dbname</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the name of the database to be repacked or analyzed,
+ when <option>-a</option>/<option>--all</option> is not used. If this
+ is not specified, the database name is read from the environment
+ variable <envar>PGDATABASE</envar>. If that is not set, the user name
+ specified for the connection is used.
+ The <replaceable>dbname</replaceable> can be
+ a <link linkend="libpq-connstring">connection string</link>. If so,
+ connection string parameters will override any conflicting command
+ line options.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-e</option></term>
+ <term><option>--echo</option></term>
+ <listitem>
+ <para>
+ Echo the commands that <application>pg_repackdb</application>
+ generates and sends to the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-j <replaceable class="parameter">njobs</replaceable></option></term>
+ <term><option>--jobs=<replaceable class="parameter">njobs</replaceable></option></term>
+ <listitem>
+ <para>
+ Execute the repack or analyze commands in parallel by running
+ <replaceable class="parameter">njobs</replaceable>
+ commands simultaneously. This option may reduce the processing time
+ but it also increases the load on the database server.
+ </para>
+ <para>
+ <application>pg_repackdb</application> will open
+ <replaceable class="parameter">njobs</replaceable> connections to the
+ database, so make sure your <xref linkend="guc-max-connections"/>
+ setting is high enough to accommodate all connections.
+ </para>
+ <para>
+ Note that using this mode might cause deadlock failures if certain
+ system catalogs are processed in parallel.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-n <replaceable class="parameter">schema</replaceable></option></term>
+ <term><option>--schema=<replaceable class="parameter">schema</replaceable></option></term>
+ <listitem>
+ <para>
+ Repack or analyze all tables in
+ <replaceable class="parameter">schema</replaceable> only. Multiple
+ schemas can be repacked by writing multiple <option>-n</option>
+ switches.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-N <replaceable class="parameter">schema</replaceable></option></term>
+ <term><option>--exclude-schema=<replaceable class="parameter">schema</replaceable></option></term>
+ <listitem>
+ <para>
+ Do not repack or analyze any tables in
+ <replaceable class="parameter">schema</replaceable>. Multiple schemas
+ can be excluded by writing multiple <option>-N</option> switches.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-q</option></term>
+ <term><option>--quiet</option></term>
+ <listitem>
+ <para>
+ Do not display progress messages.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-t <replaceable class="parameter">table</replaceable> [ (<replaceable class="parameter">column</replaceable> [,...]) ]</option></term>
+ <term><option>--table=<replaceable class="parameter">table</replaceable> [ (<replaceable class="parameter">column</replaceable> [,...]) ]</option></term>
+ <listitem>
+ <para>
+ Repack or analyze <replaceable class="parameter">table</replaceable>
+ only. Column names can be specified only in conjunction with
+ the <option>--analyze</option> option. Multiple tables can be
+ repacked by writing multiple
+ <option>-t</option> switches.
+ </para>
+ <tip>
+ <para>
+ If you specify columns, you probably have to escape the parentheses
+ from the shell. (See examples below.)
+ </para>
+ </tip>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-v</option></term>
+ <term><option>--verbose</option></term>
+ <listitem>
+ <para>
+ Print detailed information during processing.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-V</option></term>
+ <term><option>--version</option></term>
+ <listitem>
+ <para>
+ Print the <application>pg_repackdb</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-z</option></term>
+ <term><option>--analyze</option></term>
+ <listitem>
+ <para>
+ Also calculate statistics for use by the optimizer.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</option></term>
+ <term><option>--help</option></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_repackdb</application> command line
+ arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ <para>
+ <application>pg_repackdb</application> also accepts
+ the following command-line arguments for connection parameters:
+ <variablelist>
+ <varlistentry>
+ <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
+ <term><option>--host=<replaceable class="parameter">host</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the host name of the machine on which the server
+ is running. If the value begins with a slash, it is used
+ as the directory for the Unix domain socket.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-p <replaceable class="parameter">port</replaceable></option></term>
+ <term><option>--port=<replaceable class="parameter">port</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the TCP port or local Unix domain socket file
+ extension on which the server
+ is listening for connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-U <replaceable class="parameter">username</replaceable></option></term>
+ <term><option>--username=<replaceable class="parameter">username</replaceable></option></term>
+ <listitem>
+ <para>
+ User name to connect as.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-w</option></term>
+ <term><option>--no-password</option></term>
+ <listitem>
+ <para>
+ Never issue a password prompt. If the server requires
+ password authentication and a password is not available by
+ other means such as a <filename>.pgpass</filename> file, the
+ connection attempt will fail. This option can be useful in
+ batch jobs and scripts where no user is present to enter a
+ password.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-W</option></term>
+ <term><option>--password</option></term>
+ <listitem>
+ <para>
+ Force <application>pg_repackdb</application> to prompt for a
+ password before connecting to a database.
+ </para>
+
+ <para>
+ This option is never essential, since
+ <application>pg_repackdb</application> will automatically prompt
+ for a password if the server demands password authentication.
+ However, <application>pg_repackdb</application> will waste a
+ connection attempt finding out that the server wants a password.
+ In some cases it is worth typing <option>-W</option> to avoid the extra
+ connection attempt.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>--maintenance-db=<replaceable class="parameter">dbname</replaceable></option></term>
+ <listitem>
+ <para>
+ When the <option>-a</option>/<option>--all</option> is used, connect
+ to this database to gather the list of databases to repack.
+ If not specified, the <literal>postgres</literal> database will be used,
+ or if that does not exist, <literal>template1</literal> will be used.
+ This can be a <link linkend="libpq-connstring">connection
+ string</link>. If so, connection string parameters will override any
+ conflicting command line options. Also, connection string parameters
+ other than the database name itself will be re-used when connecting
+ to other databases.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </refsect1>
+
+
+ <refsect1>
+ <title>Environment</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><envar>PGDATABASE</envar></term>
+ <term><envar>PGHOST</envar></term>
+ <term><envar>PGPORT</envar></term>
+ <term><envar>PGUSER</envar></term>
+
+ <listitem>
+ <para>
+ Default connection parameters
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><envar>PG_COLOR</envar></term>
+ <listitem>
+ <para>
+ Specifies whether to use color in diagnostic messages. Possible values
+ are <literal>always</literal>, <literal>auto</literal> and
+ <literal>never</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ This utility, like most other <productname>PostgreSQL</productname> utilities,
+ also uses the environment variables supported by <application>libpq</application>
+ (see <xref linkend="libpq-envars"/>).
+ </para>
+
+ </refsect1>
+
+
+ <refsect1>
+ <title>Diagnostics</title>
+
+ <para>
+ In case of difficulty, see
+ <xref linkend="sql-repack"/> and <xref linkend="app-psql"/> for
+ discussions of potential problems and error messages.
+ The database server must be running at the
+ targeted host. Also, any default connection settings and environment
+ variables used by the <application>libpq</application> front-end
+ library will apply.
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ To repack the database <literal>test</literal>:
+<screen>
+<prompt>$ </prompt><userinput>pg_repackdb test</userinput>
+</screen>
+ </para>
+
+ <para>
+ To repack and analyze for the optimizer a database named
+ <literal>bigdb</literal>:
+<screen>
+<prompt>$ </prompt><userinput>pg_repackdb --analyze bigdb</userinput>
+</screen>
+ </para>
+
+ <para>
+ To repack a single table
+ <literal>foo</literal> in a database named
+ <literal>xyzzy</literal>, and analyze a single column
+ <literal>bar</literal> of the table for the optimizer:
+<screen>
+<prompt>$ </prompt><userinput>pg_repackdb --analyze --verbose --table='foo(bar)' xyzzy</userinput>
+</screen></para>
+
+ <para>
+ To repack all tables in the <literal>foo</literal> and <literal>bar</literal> schemas
+ in a database named <literal>xyzzy</literal>:
+<screen>
+<prompt>$ </prompt><userinput>pg_repackdb --schema='foo' --schema='bar' xyzzy</userinput>
+</screen></para>
+
+
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="sql-repack"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml
new file mode 100644
index 00000000000..fd9d89f8aaa
--- /dev/null
+++ b/doc/src/sgml/ref/repack.sgml
@@ -0,0 +1,284 @@
+<!--
+doc/src/sgml/ref/repack.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-repack">
+ <indexterm zone="sql-repack">
+ <primary>REPACK</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>REPACK</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>REPACK</refname>
+ <refpurpose>rewrite a table to reclaim disk space</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_name</replaceable> [ USING INDEX [ <replaceable class="parameter">index_name</replaceable> ] ] ]
+
+<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
+
+ VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
+ ANALYSE | ANALYZE
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>REPACK</command> reclaims storage occupied by dead
+ tuples. Unlike <command>VACUUM</command>, it does so by rewriting the
+ entire contents of the table specified
+ by <replaceable class="parameter">table_name</replaceable> into a new disk
+ file with no extra space (except for the space guaranteed by
+ the <literal>fillfactor</literal> storage parameter), allowing unused space
+ to be returned to the operating system.
+ </para>
+
+ <para>
+ Without
+ a <replaceable class="parameter">table_name</replaceable>, <command>REPACK</command>
+ processes every table and materialized view in the current database that
+ the current user has the <literal>MAINTAIN</literal> privilege on. This
+ form of <command>REPACK</command> cannot be executed inside a transaction
+ block.
+ </para>
+
+ <para>
+ If a <literal>USING INDEX</literal> clause is specified, the rows are
+ physically reordered based on information from an index. Please see the
+ notes on clustering below.
+ </para>
+
+ <para>
+ When a table is being repacked, an <literal>ACCESS EXCLUSIVE</literal> lock
+ is acquired on it. This prevents any other database operations (both reads
+ and writes) from operating on the table until the <command>REPACK</command>
+ is finished.
+ </para>
+
+ <refsect2 id="sql-repack-notes-on-clustering" xreflabel="Notes on Clustering">
+ <title>Notes on Clustering</title>
+
+ <para>
+ If the <literal>USING INDEX</literal> clause is specified, the rows in
+ the table are physically reordered following an index: if an index name
+ is specified in the command, then that index is used; if no index name
+ is specified, then the index that has been configured as the index to
+ cluster on. If no index has been configured in this way, an error is
+ thrown. The index given in the <literal>USING INDEX</literal> clause
+ is configured as the index to cluster on, as well as an index given
+ to the <command>CLUSTER</command> command. An index can be set
+ manually using <command>ALTER TABLE ... CLUSTER ON</command>, and reset
+ with <command>ALTER TABLE ... SET WITHOUT CLUSTER</command>.
+ </para>
+
+ <para>
+ If no table name is specified in <command>REPACK USING INDEX</command>,
+ all tables which have a clustering index defined and which the calling
+ user has privileges for are processed.
+ </para>
+
+ <para>
+ Clustering is a one-time operation: when the table is
+ subsequently updated, the changes are not clustered. That is, no attempt
+ is made to store new or updated rows according to their index order. (If
+ one wishes, one can periodically recluster by issuing the command again.
+ Also, setting the table's <literal>fillfactor</literal> storage parameter
+ to less than 100% can aid in preserving cluster ordering during updates,
+ since updated rows are kept on the same page if enough space is available
+ there.)
+ </para>
+
+ <para>
+ In cases where you are accessing single rows randomly within a table, the
+ actual order of the data in the table is unimportant. However, if you tend
+ to access some data more than others, and there is an index that groups
+ them together, you will benefit from using clustering. If
+ you are requesting a range of indexed values from a table, or a single
+ indexed value that has multiple rows that match,
+ <command>REPACK</command> will help because once the index identifies the
+ table page for the first row that matches, all other rows that match are
+ probably already on the same table page, and so you save disk accesses and
+ speed up the query.
+ </para>
+
+ <para>
+ <command>REPACK</command> can re-sort the table using either an index scan
+ on the specified index (if the index is a b-tree), or a sequential scan
+ followed by sorting. It will attempt to choose the method that will be
+ faster, based on planner cost parameters and available statistical
+ information.
+ </para>
+
+ <para>
+ Because the planner records statistics about the ordering of tables, it is
+ advisable to
+ run <link linkend="sql-analyze"><command>ANALYZE</command></link> on the
+ newly repacked table. Otherwise, the planner might make poor choices of
+ query plans.
+ </para>
+ </refsect2>
+
+ <refsect2 id="sql-repack-notes-on-resources" xreflabel="Notes on Resources">
+ <title>Notes on Resources</title>
+
+ <para>
+ When an index scan or a sequential scan without sort is used, a temporary
+ copy of the table is created that contains the table data in the index
+ order. Temporary copies of each index on the table are created as well.
+ Therefore, you need free space on disk at least equal to the sum of the
+ table size and the index sizes.
+ </para>
+
+ <para>
+ When a sequential scan and sort is used, a temporary sort file is also
+ created, so that the peak temporary space requirement is as much as double
+ the table size, plus the index sizes. This method is often faster than
+ the index scan method, but if the disk space requirement is intolerable,
+ you can disable this choice by temporarily setting
+ <xref linkend="guc-enable-sort"/> to <literal>off</literal>.
+ </para>
+
+ <para>
+ It is advisable to set <xref linkend="guc-maintenance-work-mem"/> to a
+ reasonably large value (but not more than the amount of RAM you can
+ dedicate to the <command>REPACK</command> operation) before repacking.
+ </para>
+ </refsect2>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">table_name</replaceable></term>
+ <listitem>
+ <para>
+ The name (possibly schema-qualified) of a table.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">index_name</replaceable></term>
+ <listitem>
+ <para>
+ The name of an index.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>VERBOSE</literal></term>
+ <listitem>
+ <para>
+ Prints a progress report as each table is repacked
+ at <literal>INFO</literal> level.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>ANALYZE</literal></term>
+ <term><literal>ANALYSE</literal></term>
+ <listitem>
+ <para>
+ Applies <xref linkend="sql-analyze"/> on the table after repacking. This is
+ currently only supported when a single (non-partitioned) table is specified.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">boolean</replaceable></term>
+ <listitem>
+ <para>
+ Specifies whether the selected option should be turned on or off.
+ You can write <literal>TRUE</literal>, <literal>ON</literal>, or
+ <literal>1</literal> to enable the option, and <literal>FALSE</literal>,
+ <literal>OFF</literal>, or <literal>0</literal> to disable it. The
+ <replaceable class="parameter">boolean</replaceable> value can also
+ be omitted, in which case <literal>TRUE</literal> is assumed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Notes</title>
+
+ <para>
+ To repack a table, one must have the <literal>MAINTAIN</literal> privilege
+ on the table.
+ </para>
+
+ <para>
+ While <command>REPACK</command> is running, the <xref
+ linkend="guc-search-path"/> is temporarily changed to <literal>pg_catalog,
+ pg_temp</literal>.
+ </para>
+
+ <para>
+ Each backend running <command>REPACK</command> will report its progress
+ in the <structname>pg_stat_progress_repack</structname> view. See
+ <xref linkend="repack-progress-reporting"/> for details.
+ </para>
+
+ <para>
+ Repacking a partitioned table repacks each of its partitions. If an index
+ is specified, each partition is repacked using the partition of that
+ index. <command>REPACK</command> on a partitioned table cannot be executed
+ inside a transaction block.
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Repack the table <literal>employees</literal>:
+<programlisting>
+REPACK employees;
+</programlisting>
+ </para>
+
+ <para>
+ Repack the table <literal>employees</literal> on the basis of its
+ index <literal>employees_ind</literal> (Since index is used here, this is
+ effectively clustering):
+<programlisting>
+REPACK employees USING INDEX employees_ind;
+</programlisting>
+ </para>
+
+ <para>
+ Repack all tables in the database on which you have
+ the <literal>MAINTAIN</literal> privilege:
+<programlisting>
+REPACK;
+</programlisting></para>
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ There is no <command>REPACK</command> statement in the SQL standard.
+ </para>
+
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index bd5dcaf86a5..062b658cfcd 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -25,7 +25,6 @@ VACUUM [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
- FULL [ <replaceable class="parameter">boolean</replaceable> ]
FREEZE [ <replaceable class="parameter">boolean</replaceable> ]
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
ANALYZE [ <replaceable class="parameter">boolean</replaceable> ]
@@ -39,6 +38,7 @@ VACUUM [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
SKIP_DATABASE_STATS [ <replaceable class="parameter">boolean</replaceable> ]
ONLY_DATABASE_STATS [ <replaceable class="parameter">boolean</replaceable> ]
BUFFER_USAGE_LIMIT <replaceable class="parameter">size</replaceable>
+ FULL [ <replaceable class="parameter">boolean</replaceable> ]
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -95,20 +95,6 @@ VACUUM [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
<title>Parameters</title>
<variablelist>
- <varlistentry>
- <term><literal>FULL</literal></term>
- <listitem>
- <para>
- Selects <quote>full</quote> vacuum, which can reclaim more
- space, but takes much longer and exclusively locks the table.
- This method also requires extra disk space, since it writes a
- new copy of the table and doesn't release the old copy until
- the operation is complete. Usually this should only be used when a
- significant amount of space needs to be reclaimed from within the table.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><literal>FREEZE</literal></term>
<listitem>
@@ -362,6 +348,23 @@ VACUUM [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>FULL</literal></term>
+ <listitem>
+ <para>
+ This option, which is deprecated, makes <command>VACUUM</command>
+ behave like <command>REPACK</command> without a
+ <literal>USING INDEX</literal> clause.
+ This method of compacting the table takes much longer than
+ <command>VACUUM</command> and exclusively locks the table.
+ This method also requires extra disk space, since it writes a
+ new copy of the table and doesn't release the old copy until
+ the operation is complete. Usually this should only be used when a
+ significant amount of space needs to be reclaimed from within the table.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><replaceable class="parameter">boolean</replaceable></term>
<listitem>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index ff85ace83fc..2ee08e21f41 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -195,6 +195,7 @@
&refreshMaterializedView;
&reindex;
&releaseSavepoint;
+ &repack;
&reset;
&revoke;
&rollback;
@@ -257,6 +258,7 @@
&pgIsready;
&pgReceivewal;
&pgRecvlogical;
+ &pgRepackdb;
&pgRestore;
&pgVerifyBackup;
&psqlRef;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bcbac844bb6..79f9de5d760 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -741,13 +741,13 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
if (OldIndex != NULL && !use_sort)
{
const int ci_index[] = {
- PROGRESS_CLUSTER_PHASE,
- PROGRESS_CLUSTER_INDEX_RELID
+ PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_INDEX_RELID
};
int64 ci_val[2];
/* Set phase and OIDOldIndex to columns */
- ci_val[0] = PROGRESS_CLUSTER_PHASE_INDEX_SCAN_HEAP;
+ ci_val[0] = PROGRESS_REPACK_PHASE_INDEX_SCAN_HEAP;
ci_val[1] = RelationGetRelid(OldIndex);
pgstat_progress_update_multi_param(2, ci_index, ci_val);
@@ -759,15 +759,15 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
else
{
/* In scan-and-sort mode and also VACUUM FULL, set phase */
- pgstat_progress_update_param(PROGRESS_CLUSTER_PHASE,
- PROGRESS_CLUSTER_PHASE_SEQ_SCAN_HEAP);
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SEQ_SCAN_HEAP);
tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL);
heapScan = (HeapScanDesc) tableScan;
indexScan = NULL;
/* Set total heap blocks */
- pgstat_progress_update_param(PROGRESS_CLUSTER_TOTAL_HEAP_BLKS,
+ pgstat_progress_update_param(PROGRESS_REPACK_TOTAL_HEAP_BLKS,
heapScan->rs_nblocks);
}
@@ -809,7 +809,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
* is manually updated to the correct value when the table
* scan finishes.
*/
- pgstat_progress_update_param(PROGRESS_CLUSTER_HEAP_BLKS_SCANNED,
+ pgstat_progress_update_param(PROGRESS_REPACK_HEAP_BLKS_SCANNED,
heapScan->rs_nblocks);
break;
}
@@ -825,7 +825,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
*/
if (prev_cblock != heapScan->rs_cblock)
{
- pgstat_progress_update_param(PROGRESS_CLUSTER_HEAP_BLKS_SCANNED,
+ pgstat_progress_update_param(PROGRESS_REPACK_HEAP_BLKS_SCANNED,
(heapScan->rs_cblock +
heapScan->rs_nblocks -
heapScan->rs_startblock
@@ -912,14 +912,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
* In scan-and-sort mode, report increase in number of tuples
* scanned
*/
- pgstat_progress_update_param(PROGRESS_CLUSTER_HEAP_TUPLES_SCANNED,
+ pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_SCANNED,
*num_tuples);
}
else
{
const int ct_index[] = {
- PROGRESS_CLUSTER_HEAP_TUPLES_SCANNED,
- PROGRESS_CLUSTER_HEAP_TUPLES_WRITTEN
+ PROGRESS_REPACK_HEAP_TUPLES_SCANNED,
+ PROGRESS_REPACK_HEAP_TUPLES_WRITTEN
};
int64 ct_val[2];
@@ -952,14 +952,14 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
double n_tuples = 0;
/* Report that we are now sorting tuples */
- pgstat_progress_update_param(PROGRESS_CLUSTER_PHASE,
- PROGRESS_CLUSTER_PHASE_SORT_TUPLES);
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SORT_TUPLES);
tuplesort_performsort(tuplesort);
/* Report that we are now writing new heap */
- pgstat_progress_update_param(PROGRESS_CLUSTER_PHASE,
- PROGRESS_CLUSTER_PHASE_WRITE_NEW_HEAP);
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
for (;;)
{
@@ -977,7 +977,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
values, isnull,
rwstate);
/* Report n_tuples */
- pgstat_progress_update_param(PROGRESS_CLUSTER_HEAP_TUPLES_WRITTEN,
+ pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_WRITTEN,
n_tuples);
}
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c4029a4f3d3..3063abff9a5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -4079,7 +4079,7 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
Assert(!ReindexIsProcessingIndex(indexOid));
/* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_CLUSTER_INDEX_REBUILD_COUNT,
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
i);
i++;
}
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1b3c5a55882..b2b7b10c2be 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1279,6 +1279,32 @@ CREATE VIEW pg_stat_progress_cluster AS
FROM pg_stat_get_progress_info('CLUSTER') AS S
LEFT JOIN pg_database D ON S.datid = D.oid;
+CREATE VIEW pg_stat_progress_repack AS
+ SELECT
+ S.pid AS pid,
+ S.datid AS datid,
+ D.datname AS datname,
+ S.relid AS relid,
+ -- param1 is currently unused
+ CASE S.param2 WHEN 0 THEN 'initializing'
+ WHEN 1 THEN 'seq scanning heap'
+ WHEN 2 THEN 'index scanning heap'
+ WHEN 3 THEN 'sorting tuples'
+ WHEN 4 THEN 'writing new heap'
+ WHEN 5 THEN 'swapping relation files'
+ WHEN 6 THEN 'rebuilding index'
+ WHEN 7 THEN 'performing final cleanup'
+ END AS phase,
+ CAST(S.param3 AS oid) AS repack_index_relid,
+ S.param4 AS heap_tuples_scanned,
+ S.param5 AS heap_tuples_written,
+ S.param6 AS heap_blks_total,
+ S.param7 AS heap_blks_scanned,
+ S.param8 AS index_rebuild_count
+ FROM pg_stat_get_progress_info('REPACK') AS S
+ LEFT JOIN pg_database D ON S.datid = D.oid;
+
+
CREATE VIEW pg_stat_progress_create_index AS
SELECT
S.pid AS pid, S.datid AS datid, D.datname AS datname,
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index b55221d44cd..8b64f9e6795 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -67,18 +67,41 @@ typedef struct
Oid indexOid;
} RelToCluster;
-
-static void cluster_multiple_rels(List *rtcs, ClusterParams *params);
-static void rebuild_relation(Relation OldHeap, Relation index, bool verbose);
+static bool cluster_rel_recheck(RepackCommand cmd, Relation OldHeap,
+ Oid indexOid, Oid userid, int options);
+static void rebuild_relation(RepackCommand cmd, bool usingindex,
+ Relation OldHeap, Relation index, bool verbose);
static void copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
bool verbose, bool *pSwapToastByContent,
TransactionId *pFreezeXid, MultiXactId *pCutoffMulti);
-static List *get_tables_to_cluster(MemoryContext cluster_context);
-static List *get_tables_to_cluster_partitioned(MemoryContext cluster_context,
- Oid indexOid);
-static bool cluster_is_permitted_for_relation(Oid relid, Oid userid);
+static List *get_tables_to_repack(RepackCommand cmd, bool usingindex,
+ MemoryContext permcxt);
+static List *get_tables_to_repack_partitioned(RepackCommand cmd,
+ MemoryContext cluster_context,
+ Oid relid, bool rel_is_index);
+static bool cluster_is_permitted_for_relation(RepackCommand cmd,
+ Oid relid, Oid userid);
+static Relation process_single_relation(RepackStmt *stmt,
+ ClusterParams *params);
+static Oid determine_clustered_index(Relation rel, bool usingindex,
+ const char *indexname);
+static const char *
+RepackCommandAsString(RepackCommand cmd)
+{
+ switch (cmd)
+ {
+ case REPACK_COMMAND_REPACK:
+ return "REPACK";
+ case REPACK_COMMAND_VACUUMFULL:
+ return "VACUUM";
+ case REPACK_COMMAND_CLUSTER:
+ return "CLUSTER";
+ }
+ return "???";
+}
+
/*---------------------------------------------------------------------------
* This cluster code allows for clustering multiple tables at once. Because
* of this, we cannot just run everything on a single transaction, or we
@@ -104,191 +127,155 @@ static bool cluster_is_permitted_for_relation(Oid relid, Oid userid);
*---------------------------------------------------------------------------
*/
void
-cluster(ParseState *pstate, ClusterStmt *stmt, bool isTopLevel)
+ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
{
- ListCell *lc;
ClusterParams params = {0};
- bool verbose = false;
Relation rel = NULL;
- Oid indexOid = InvalidOid;
- MemoryContext cluster_context;
+ MemoryContext repack_context;
List *rtcs;
/* Parse option list */
- foreach(lc, stmt->params)
+ foreach_node(DefElem, opt, stmt->params)
{
- DefElem *opt = (DefElem *) lfirst(lc);
-
if (strcmp(opt->defname, "verbose") == 0)
- verbose = defGetBoolean(opt);
+ params.options |= defGetBoolean(opt) ? CLUOPT_VERBOSE : 0;
+ else if (strcmp(opt->defname, "analyze") == 0 ||
+ strcmp(opt->defname, "analyse") == 0)
+ params.options |= defGetBoolean(opt) ? CLUOPT_ANALYZE : 0;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("unrecognized CLUSTER option \"%s\"",
+ errmsg("unrecognized %s option \"%s\"",
+ RepackCommandAsString(stmt->command),
opt->defname),
parser_errposition(pstate, opt->location)));
}
- params.options = (verbose ? CLUOPT_VERBOSE : 0);
-
+ /*
+ * If a single relation is specified, process it and we're done ... unless
+ * the relation is a partitioned table, in which case we fall through.
+ */
if (stmt->relation != NULL)
{
- /* This is the single-relation case. */
- Oid tableOid;
-
- /*
- * Find, lock, and check permissions on the table. We obtain
- * AccessExclusiveLock right away to avoid lock-upgrade hazard in the
- * single-transaction case.
- */
- tableOid = RangeVarGetRelidExtended(stmt->relation,
- AccessExclusiveLock,
- 0,
- RangeVarCallbackMaintainsTable,
- NULL);
- rel = table_open(tableOid, NoLock);
-
- /*
- * Reject clustering a remote temp table ... their local buffer
- * manager is not going to cope.
- */
- if (RELATION_IS_OTHER_TEMP(rel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot cluster temporary tables of other sessions")));
-
- if (stmt->indexname == NULL)
- {
- ListCell *index;
-
- /* We need to find the index that has indisclustered set. */
- foreach(index, RelationGetIndexList(rel))
- {
- indexOid = lfirst_oid(index);
- if (get_index_isclustered(indexOid))
- break;
- indexOid = InvalidOid;
- }
-
- if (!OidIsValid(indexOid))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("there is no previously clustered index for table \"%s\"",
- stmt->relation->relname)));
- }
- else
- {
- /*
- * The index is expected to be in the same namespace as the
- * relation.
- */
- indexOid = get_relname_relid(stmt->indexname,
- rel->rd_rel->relnamespace);
- if (!OidIsValid(indexOid))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("index \"%s\" for table \"%s\" does not exist",
- stmt->indexname, stmt->relation->relname)));
- }
-
- /* For non-partitioned tables, do what we came here to do. */
- if (rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
- {
- cluster_rel(rel, indexOid, ¶ms);
- /* cluster_rel closes the relation, but keeps lock */
-
+ rel = process_single_relation(stmt, ¶ms);
+ if (rel == NULL)
return;
- }
}
+ /* Don't allow this for now. Maybe we can add support for this later */
+ if (params.options & CLUOPT_ANALYZE)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot ANALYZE multiple tables"));
+
/*
* By here, we know we are in a multi-table situation. In order to avoid
* holding locks for too long, we want to process each table in its own
* transaction. This forces us to disallow running inside a user
* transaction block.
*/
- PreventInTransactionBlock(isTopLevel, "CLUSTER");
+ PreventInTransactionBlock(isTopLevel, RepackCommandAsString(stmt->command));
/* Also, we need a memory context to hold our list of relations */
- cluster_context = AllocSetContextCreate(PortalContext,
- "Cluster",
- ALLOCSET_DEFAULT_SIZES);
+ repack_context = AllocSetContextCreate(PortalContext,
+ "Repack",
+ ALLOCSET_DEFAULT_SIZES);
- /*
- * Either we're processing a partitioned table, or we were not given any
- * table name at all. In either case, obtain a list of relations to
- * process.
- *
- * In the former case, an index name must have been given, so we don't
- * need to recheck its "indisclustered" bit, but we have to check that it
- * is an index that we can cluster on. In the latter case, we set the
- * option bit to have indisclustered verified.
- *
- * Rechecking the relation itself is necessary here in all cases.
- */
params.options |= CLUOPT_RECHECK;
- if (rel != NULL)
+
+ /*
+ * If we don't have a relation yet, determine a relation list. If we do,
+ * then it must be a partitioned table, and we want to process its
+ * partitions.
+ */
+ if (rel == NULL)
{
+ Assert(stmt->indexname == NULL);
+ rtcs = get_tables_to_repack(stmt->command, stmt->usingindex,
+ repack_context);
+ }
+ else
+ {
+ Oid relid;
+ bool rel_is_index;
+
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
- check_index_is_clusterable(rel, indexOid, AccessShareLock);
- rtcs = get_tables_to_cluster_partitioned(cluster_context, indexOid);
- /* close relation, releasing lock on parent table */
+ /*
+ * If an index name was specified, resolve it now and pass it down.
+ */
+ if (stmt->usingindex)
+ {
+ /*
+ * XXX how should this behave? Passing no index to a partitioned
+ * table could be useful to have certain partitions clustered by
+ * some index, and other partitions by a different index.
+ */
+ if (!stmt->indexname)
+ ereport(ERROR,
+ errmsg("there is no previously clustered index for table \"%s\"",
+ RelationGetRelationName(rel)));
+
+ relid = determine_clustered_index(rel, true, stmt->indexname);
+ if (!OidIsValid(relid))
+ elog(ERROR, "unable to determine index to cluster on");
+ /* XXX is this the right place for this check? */
+ check_index_is_clusterable(rel, relid, AccessExclusiveLock);
+ rel_is_index = true;
+ }
+ else
+ {
+ relid = RelationGetRelid(rel);
+ rel_is_index = false;
+ }
+
+ rtcs = get_tables_to_repack_partitioned(stmt->command, repack_context,
+ relid, rel_is_index);
+
+ /* close parent relation, releasing lock on it */
table_close(rel, AccessExclusiveLock);
+ rel = NULL;
}
- else
- {
- rtcs = get_tables_to_cluster(cluster_context);
- params.options |= CLUOPT_RECHECK_ISCLUSTERED;
- }
-
- /* Do the job. */
- cluster_multiple_rels(rtcs, ¶ms);
-
- /* Start a new transaction for the cleanup work. */
- StartTransactionCommand();
-
- /* Clean up working storage */
- MemoryContextDelete(cluster_context);
-}
-
-/*
- * Given a list of relations to cluster, process each of them in a separate
- * transaction.
- *
- * We expect to be in a transaction at start, but there isn't one when we
- * return.
- */
-static void
-cluster_multiple_rels(List *rtcs, ClusterParams *params)
-{
- ListCell *lc;
/* Commit to get out of starting transaction */
PopActiveSnapshot();
CommitTransactionCommand();
/* Cluster the tables, each in a separate transaction */
- foreach(lc, rtcs)
+ Assert(rel == NULL);
+ foreach_ptr(RelToCluster, rtc, rtcs)
{
- RelToCluster *rtc = (RelToCluster *) lfirst(lc);
- Relation rel;
-
/* Start a new transaction for each relation. */
StartTransactionCommand();
+ /*
+ * Open the target table, coping with the case where it has been
+ * dropped.
+ */
+ rel = try_table_open(rtc->tableOid, AccessExclusiveLock);
+ if (rel == NULL)
+ {
+ CommitTransactionCommand();
+ continue;
+ }
+
/* functions in indexes may want a snapshot set */
PushActiveSnapshot(GetTransactionSnapshot());
- rel = table_open(rtc->tableOid, AccessExclusiveLock);
-
/* Process this table */
- cluster_rel(rel, rtc->indexOid, params);
+ cluster_rel(stmt->command, stmt->usingindex,
+ rel, rtc->indexOid, ¶ms);
/* cluster_rel closes the relation, but keeps lock */
PopActiveSnapshot();
CommitTransactionCommand();
}
+
+ /* Start a new transaction for the cleanup work. */
+ StartTransactionCommand();
+
+ /* Clean up working storage */
+ MemoryContextDelete(repack_context);
}
/*
@@ -304,11 +291,14 @@ cluster_multiple_rels(List *rtcs, ClusterParams *params)
* them incrementally while we load the table.
*
* If indexOid is InvalidOid, the table will be rewritten in physical order
- * instead of index order. This is the new implementation of VACUUM FULL,
- * and error messages should refer to the operation as VACUUM not CLUSTER.
+ * instead of index order.
+ *
+ * 'cmd' indicates which command is being executed, to be used for error
+ * messages.
*/
void
-cluster_rel(Relation OldHeap, Oid indexOid, ClusterParams *params)
+cluster_rel(RepackCommand cmd, bool usingindex,
+ Relation OldHeap, Oid indexOid, ClusterParams *params)
{
Oid tableOid = RelationGetRelid(OldHeap);
Oid save_userid;
@@ -323,13 +313,25 @@ cluster_rel(Relation OldHeap, Oid indexOid, ClusterParams *params)
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
- pgstat_progress_start_command(PROGRESS_COMMAND_CLUSTER, tableOid);
- if (OidIsValid(indexOid))
- pgstat_progress_update_param(PROGRESS_CLUSTER_COMMAND,
+ if (cmd == REPACK_COMMAND_REPACK)
+ pgstat_progress_start_command(PROGRESS_COMMAND_REPACK, tableOid);
+ else
+ pgstat_progress_start_command(PROGRESS_COMMAND_CLUSTER, tableOid);
+
+ if (cmd == REPACK_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_COMMAND,
+ PROGRESS_REPACK_COMMAND_REPACK);
+ else if (cmd == REPACK_COMMAND_CLUSTER)
+ {
+ pgstat_progress_update_param(PROGRESS_REPACK_COMMAND,
PROGRESS_CLUSTER_COMMAND_CLUSTER);
+ }
else
- pgstat_progress_update_param(PROGRESS_CLUSTER_COMMAND,
+ {
+ Assert(cmd == REPACK_COMMAND_VACUUMFULL);
+ pgstat_progress_update_param(PROGRESS_REPACK_COMMAND,
PROGRESS_CLUSTER_COMMAND_VACUUM_FULL);
+ }
/*
* Switch to the table owner's userid, so that any index functions are run
@@ -351,63 +353,21 @@ cluster_rel(Relation OldHeap, Oid indexOid, ClusterParams *params)
* to cluster a not-previously-clustered index.
*/
if (recheck)
- {
- /* Check that the user still has privileges for the relation */
- if (!cluster_is_permitted_for_relation(tableOid, save_userid))
- {
- relation_close(OldHeap, AccessExclusiveLock);
+ if (!cluster_rel_recheck(cmd, OldHeap, indexOid, save_userid,
+ params->options))
goto out;
- }
-
- /*
- * Silently skip a temp table for a remote session. Only doing this
- * check in the "recheck" case is appropriate (which currently means
- * somebody is executing a database-wide CLUSTER or on a partitioned
- * table), because there is another check in cluster() which will stop
- * any attempt to cluster remote temp tables by name. There is
- * another check in cluster_rel which is redundant, but we leave it
- * for extra safety.
- */
- if (RELATION_IS_OTHER_TEMP(OldHeap))
- {
- relation_close(OldHeap, AccessExclusiveLock);
- goto out;
- }
-
- if (OidIsValid(indexOid))
- {
- /*
- * Check that the index still exists
- */
- if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(indexOid)))
- {
- relation_close(OldHeap, AccessExclusiveLock);
- goto out;
- }
-
- /*
- * Check that the index is still the one with indisclustered set,
- * if needed.
- */
- if ((params->options & CLUOPT_RECHECK_ISCLUSTERED) != 0 &&
- !get_index_isclustered(indexOid))
- {
- relation_close(OldHeap, AccessExclusiveLock);
- goto out;
- }
- }
- }
/*
- * We allow VACUUM FULL, but not CLUSTER, on shared catalogs. CLUSTER
- * would work in most respects, but the index would only get marked as
- * indisclustered in the current database, leading to unexpected behavior
- * if CLUSTER were later invoked in another database.
+ * We allow repacking shared catalogs only when not using an index. It
+ * would work to use an index in most respects, but the index would only
+ * get marked as indisclustered in the current database, leading to
+ * unexpected behavior if CLUSTER were later invoked in another database.
*/
- if (OidIsValid(indexOid) && OldHeap->rd_rel->relisshared)
+ if (usingindex && OldHeap->rd_rel->relisshared)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot cluster a shared catalog")));
+ errmsg("cannot run \"%s\" on a shared catalog",
+ RepackCommandAsString(cmd))));
/*
* Don't process temp tables of other backends ... their local buffer
@@ -415,21 +375,30 @@ cluster_rel(Relation OldHeap, Oid indexOid, ClusterParams *params)
*/
if (RELATION_IS_OTHER_TEMP(OldHeap))
{
- if (OidIsValid(indexOid))
+ if (cmd == REPACK_COMMAND_CLUSTER)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster temporary tables of other sessions")));
+ else if (cmd == REPACK_COMMAND_REPACK)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot repack temporary tables of other sessions")));
+ }
else
+ {
+ Assert(cmd == REPACK_COMMAND_VACUUMFULL);
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot vacuum temporary tables of other sessions")));
+ }
}
/*
* Also check for active uses of the relation in the current transaction,
* including open scans and pending AFTER trigger events.
*/
- CheckTableNotInUse(OldHeap, OidIsValid(indexOid) ? "CLUSTER" : "VACUUM");
+ CheckTableNotInUse(OldHeap, RepackCommandAsString(cmd));
/* Check heap and index are valid to cluster on */
if (OidIsValid(indexOid))
@@ -469,7 +438,7 @@ cluster_rel(Relation OldHeap, Oid indexOid, ClusterParams *params)
TransferPredicateLocksToHeapRelation(OldHeap);
/* rebuild_relation does all the dirty work */
- rebuild_relation(OldHeap, index, verbose);
+ rebuild_relation(cmd, usingindex, OldHeap, index, verbose);
/* rebuild_relation closes OldHeap, and index if valid */
out:
@@ -482,6 +451,63 @@ out:
pgstat_progress_end_command();
}
+/*
+ * Check if the table (and its index) still meets the requirements of
+ * cluster_rel().
+ */
+static bool
+cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
+ Oid userid, int options)
+{
+ Oid tableOid = RelationGetRelid(OldHeap);
+
+ /* Check that the user still has privileges for the relation */
+ if (!cluster_is_permitted_for_relation(cmd, tableOid, userid))
+ {
+ relation_close(OldHeap, AccessExclusiveLock);
+ return false;
+ }
+
+ /*
+ * Silently skip a temp table for a remote session. Only doing this check
+ * in the "recheck" case is appropriate (which currently means somebody is
+ * executing a database-wide CLUSTER or on a partitioned table), because
+ * there is another check in cluster() which will stop any attempt to
+ * cluster remote temp tables by name. There is another check in
+ * cluster_rel which is redundant, but we leave it for extra safety.
+ */
+ if (RELATION_IS_OTHER_TEMP(OldHeap))
+ {
+ relation_close(OldHeap, AccessExclusiveLock);
+ return false;
+ }
+
+ if (OidIsValid(indexOid))
+ {
+ /*
+ * Check that the index still exists
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(indexOid)))
+ {
+ relation_close(OldHeap, AccessExclusiveLock);
+ return false;
+ }
+
+ /*
+ * Check that the index is still the one with indisclustered set, if
+ * needed.
+ */
+ if ((options & CLUOPT_RECHECK_ISCLUSTERED) != 0 &&
+ !get_index_isclustered(indexOid))
+ {
+ relation_close(OldHeap, AccessExclusiveLock);
+ return false;
+ }
+ }
+
+ return true;
+}
+
/*
* Verify that the specified heap and index are valid to cluster on
*
@@ -626,7 +652,8 @@ mark_index_clustered(Relation rel, Oid indexOid, bool is_internal)
* On exit, they are closed, but locks on them are not released.
*/
static void
-rebuild_relation(Relation OldHeap, Relation index, bool verbose)
+rebuild_relation(RepackCommand cmd, bool usingindex,
+ Relation OldHeap, Relation index, bool verbose)
{
Oid tableOid = RelationGetRelid(OldHeap);
Oid accessMethod = OldHeap->rd_rel->relam;
@@ -642,8 +669,8 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose)
Assert(CheckRelationLockedByMe(OldHeap, AccessExclusiveLock, false) &&
(index == NULL || CheckRelationLockedByMe(index, AccessExclusiveLock, false)));
- if (index)
- /* Mark the correct index as clustered */
+ /* for CLUSTER or REPACK USING INDEX, mark the index as the one to use */
+ if (usingindex)
mark_index_clustered(OldHeap, RelationGetRelid(index), true);
/* Remember info about rel before closing OldHeap */
@@ -1458,8 +1485,8 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
int i;
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_CLUSTER_PHASE,
- PROGRESS_CLUSTER_PHASE_SWAP_REL_FILES);
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1509,14 +1536,14 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_CLUSTER_PHASE,
- PROGRESS_CLUSTER_PHASE_REBUILD_INDEX);
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_CLUSTER_PHASE,
- PROGRESS_CLUSTER_PHASE_FINAL_CLEANUP);
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -1632,69 +1659,137 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
}
}
-
/*
- * Get a list of tables that the current user has privileges on and
- * have indisclustered set. Return the list in a List * of RelToCluster
- * (stored in the specified memory context), each one giving the tableOid
- * and the indexOid on which the table is already clustered.
+ * Determine which relations to process, when REPACK/CLUSTER is called
+ * without specifying a table name. The exact process depends on whether
+ * USING INDEX was given or not, and in any case we only return tables and
+ * materialized views that the current user has privileges to repack/cluster.
+ *
+ * If USING INDEX was given, we scan pg_index to find those that have
+ * indisclustered set; if it was not given, scan pg_class and return all
+ * tables.
+ *
+ * Return it as a list of RelToCluster in the given memory context.
*/
static List *
-get_tables_to_cluster(MemoryContext cluster_context)
+get_tables_to_repack(RepackCommand command, bool usingindex,
+ MemoryContext permcxt)
{
- Relation indRelation;
+ Relation catalog;
TableScanDesc scan;
- ScanKeyData entry;
- HeapTuple indexTuple;
- Form_pg_index index;
+ HeapTuple tuple;
MemoryContext old_context;
List *rtcs = NIL;
- /*
- * Get all indexes that have indisclustered set and that the current user
- * has the appropriate privileges for.
- */
- indRelation = table_open(IndexRelationId, AccessShareLock);
- ScanKeyInit(&entry,
- Anum_pg_index_indisclustered,
- BTEqualStrategyNumber, F_BOOLEQ,
- BoolGetDatum(true));
- scan = table_beginscan_catalog(indRelation, 1, &entry);
- while ((indexTuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
+ if (usingindex)
{
- RelToCluster *rtc;
+ ScanKeyData entry;
- index = (Form_pg_index) GETSTRUCT(indexTuple);
+ catalog = table_open(IndexRelationId, AccessShareLock);
+ ScanKeyInit(&entry,
+ Anum_pg_index_indisclustered,
+ BTEqualStrategyNumber, F_BOOLEQ,
+ BoolGetDatum(true));
+ scan = table_beginscan_catalog(catalog, 1, &entry);
+ while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
+ {
+ RelToCluster *rtc;
+ Form_pg_index index;
- if (!cluster_is_permitted_for_relation(index->indrelid, GetUserId()))
- continue;
+ index = (Form_pg_index) GETSTRUCT(tuple);
- /* Use a permanent memory context for the result list */
- old_context = MemoryContextSwitchTo(cluster_context);
+ /*
+ * XXX I think the only reason there's no test failure here is
+ * that we seldom have clustered indexes that would be affected by
+ * concurrency. Maybe we should also do the
+ * ConditionalLockRelationOid+SearchSysCacheExists dance that we
+ * do below.
+ */
+ if (!cluster_is_permitted_for_relation(command, index->indrelid,
+ GetUserId()))
+ continue;
- rtc = (RelToCluster *) palloc(sizeof(RelToCluster));
- rtc->tableOid = index->indrelid;
- rtc->indexOid = index->indexrelid;
- rtcs = lappend(rtcs, rtc);
+ /* Use a permanent memory context for the result list */
+ old_context = MemoryContextSwitchTo(permcxt);
- MemoryContextSwitchTo(old_context);
+ rtc = (RelToCluster *) palloc(sizeof(RelToCluster));
+ rtc->tableOid = index->indrelid;
+ rtc->indexOid = index->indexrelid;
+ rtcs = lappend(rtcs, rtc);
+
+ MemoryContextSwitchTo(old_context);
+ }
}
+ else
+ {
+ catalog = table_open(RelationRelationId, AccessShareLock);
+ scan = table_beginscan_catalog(catalog, 0, NULL);
+
+ while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
+ {
+ RelToCluster *rtc;
+ Form_pg_class class;
+
+ class = (Form_pg_class) GETSTRUCT(tuple);
+
+ /*
+ * Try to obtain a light lock on the table, to ensure it doesn't
+ * go away while we collect the list. If we cannot, just
+ * disregard the table. XXX we could release at the bottom of the
+ * loop, but for now just hold it until this transaction is
+ * finished.
+ */
+ if (!ConditionalLockRelationOid(class->oid, AccessShareLock))
+ continue;
+
+ /* Verify that the table still exists. */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(class->oid)))
+ {
+ /* Release useless lock */
+ UnlockRelationOid(class->oid, AccessShareLock);
+ continue;
+ }
+
+ /* Can only process plain tables and matviews */
+ if (class->relkind != RELKIND_RELATION &&
+ class->relkind != RELKIND_MATVIEW)
+ continue;
+
+ if (!cluster_is_permitted_for_relation(command, class->oid,
+ GetUserId()))
+ continue;
+
+ /* Use a permanent memory context for the result list */
+ old_context = MemoryContextSwitchTo(permcxt);
+
+ rtc = (RelToCluster *) palloc(sizeof(RelToCluster));
+ rtc->tableOid = class->oid;
+ rtc->indexOid = InvalidOid;
+ rtcs = lappend(rtcs, rtc);
+
+ MemoryContextSwitchTo(old_context);
+ }
+ }
+
table_endscan(scan);
-
- relation_close(indRelation, AccessShareLock);
+ relation_close(catalog, AccessShareLock);
return rtcs;
}
/*
- * Given an index on a partitioned table, return a list of RelToCluster for
+ * Given a partitioned table or its index, return a list of RelToCluster for
* all the children leaves tables/indexes.
*
* Like expand_vacuum_rel, but here caller must hold AccessExclusiveLock
* on the table containing the index.
+ *
+ * 'rel_is_index' tells whether 'relid' is that of an index (true) or of the
+ * owning relation.
*/
static List *
-get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid)
+get_tables_to_repack_partitioned(RepackCommand cmd, MemoryContext cluster_context,
+ Oid relid, bool rel_is_index)
{
List *inhoids;
ListCell *lc;
@@ -1702,17 +1797,33 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid)
MemoryContext old_context;
/* Do not lock the children until they're processed */
- inhoids = find_all_inheritors(indexOid, NoLock, NULL);
+ inhoids = find_all_inheritors(relid, NoLock, NULL);
foreach(lc, inhoids)
{
- Oid indexrelid = lfirst_oid(lc);
- Oid relid = IndexGetRelation(indexrelid, false);
+ Oid inhoid = lfirst_oid(lc);
+ Oid inhrelid,
+ inhindid;
RelToCluster *rtc;
- /* consider only leaf indexes */
- if (get_rel_relkind(indexrelid) != RELKIND_INDEX)
- continue;
+ if (rel_is_index)
+ {
+ /* consider only leaf indexes */
+ if (get_rel_relkind(inhoid) != RELKIND_INDEX)
+ continue;
+
+ inhrelid = IndexGetRelation(inhoid, false);
+ inhindid = inhoid;
+ }
+ else
+ {
+ /* consider only leaf relations */
+ if (get_rel_relkind(inhoid) != RELKIND_RELATION)
+ continue;
+
+ inhrelid = inhoid;
+ inhindid = InvalidOid;
+ }
/*
* It's possible that the user does not have privileges to CLUSTER the
@@ -1720,15 +1831,15 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid)
* table. We skip any partitions which the user is not permitted to
* CLUSTER.
*/
- if (!cluster_is_permitted_for_relation(relid, GetUserId()))
+ if (!cluster_is_permitted_for_relation(cmd, inhrelid, GetUserId()))
continue;
/* Use a permanent memory context for the result list */
old_context = MemoryContextSwitchTo(cluster_context);
rtc = (RelToCluster *) palloc(sizeof(RelToCluster));
- rtc->tableOid = relid;
- rtc->indexOid = indexrelid;
+ rtc->tableOid = inhrelid;
+ rtc->indexOid = inhindid;
rtcs = lappend(rtcs, rtc);
MemoryContextSwitchTo(old_context);
@@ -1742,13 +1853,148 @@ get_tables_to_cluster_partitioned(MemoryContext cluster_context, Oid indexOid)
* function emits a WARNING.
*/
static bool
-cluster_is_permitted_for_relation(Oid relid, Oid userid)
+cluster_is_permitted_for_relation(RepackCommand cmd, Oid relid, Oid userid)
{
if (pg_class_aclcheck(relid, userid, ACL_MAINTAIN) == ACLCHECK_OK)
return true;
+ Assert(cmd == REPACK_COMMAND_CLUSTER || cmd == REPACK_COMMAND_REPACK);
ereport(WARNING,
- (errmsg("permission denied to cluster \"%s\", skipping it",
- get_rel_name(relid))));
+ errmsg("permission denied to execute %s on \"%s\", skipping it",
+ cmd == REPACK_COMMAND_CLUSTER ? "CLUSTER" : "REPACK",
+ get_rel_name(relid)));
+
return false;
}
+
+
+/*
+ * Given a RepackStmt with an indicated relation name, resolve the relation
+ * name, obtain lock on it, then determine what to do based on the relation
+ * type: if it's not a partitioned table, repack it as indicated (using an
+ * existing clustered index, or following the indicated index), and return
+ * NULL.
+ *
+ * On the other hand, if the table is partitioned, do nothing further and
+ * instead return the opened relcache entry, so that caller can process the
+ * partitions using the multiple-table handling code. The index name is not
+ * resolve in this case.
+ */
+static Relation
+process_single_relation(RepackStmt *stmt, ClusterParams *params)
+{
+ Relation rel;
+ Oid tableOid;
+
+ Assert(stmt->relation != NULL);
+ Assert(stmt->command == REPACK_COMMAND_CLUSTER ||
+ stmt->command == REPACK_COMMAND_REPACK);
+
+ /*
+ * Find, lock, and check permissions on the table. We obtain
+ * AccessExclusiveLock right away to avoid lock-upgrade hazard in the
+ * single-transaction case.
+ */
+ tableOid = RangeVarGetRelidExtended(stmt->relation,
+ AccessExclusiveLock,
+ 0,
+ RangeVarCallbackMaintainsTable,
+ NULL);
+ rel = table_open(tableOid, NoLock);
+
+ /*
+ * Reject clustering a remote temp table ... their local buffer manager is
+ * not going to cope.
+ */
+ if (RELATION_IS_OTHER_TEMP(rel))
+ {
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot execute %s on temporary tables of other sessions",
+ RepackCommandAsString(stmt->command)));
+ }
+
+ /*
+ * For partitioned tables, let caller handle this. Otherwise, process it
+ * here and we're done.
+ */
+ if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ return rel;
+ else
+ {
+ Oid indexOid;
+
+ indexOid = determine_clustered_index(rel, stmt->usingindex,
+ stmt->indexname);
+ if (OidIsValid(indexOid))
+ check_index_is_clusterable(rel, indexOid, AccessExclusiveLock);
+ cluster_rel(stmt->command, stmt->usingindex, rel, indexOid, params);
+
+ /* Do an analyze, if requested */
+ if (params->options & CLUOPT_ANALYZE)
+ {
+ VacuumParams vac_params = {0};
+
+ vac_params.options |= VACOPT_ANALYZE;
+ if (params->options & CLUOPT_VERBOSE)
+ vac_params.options |= VACOPT_VERBOSE;
+ analyze_rel(RelationGetRelid(rel), NULL, vac_params, NIL, true,
+ NULL);
+ }
+
+ return NULL;
+ }
+}
+
+/*
+ * Given a relation and the usingindex/indexname options in a
+ * REPACK USING INDEX or CLUSTER command, return the OID of the index to use
+ * for clustering the table.
+ *
+ * Caller must hold lock on the relation so that the set of indexes doesn't
+ * change, and must call check_index_is_clusterable.
+ */
+static Oid
+determine_clustered_index(Relation rel, bool usingindex, const char *indexname)
+{
+ Oid indexOid;
+
+ if (indexname == NULL && usingindex)
+ {
+ ListCell *lc;
+
+ /* Find an index with indisclustered set, or report error */
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ indexOid = lfirst_oid(lc);
+
+ if (get_index_isclustered(indexOid))
+ break;
+ indexOid = InvalidOid;
+ }
+
+ if (!OidIsValid(indexOid))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("there is no previously clustered index for table \"%s\"",
+ RelationGetRelationName(rel)));
+ }
+ else if (indexname != NULL)
+ {
+ /*
+ * An index was specified; figure out its OID. It must be in the same
+ * namespace as the relation.
+ */
+ indexOid = get_relname_relid(indexname,
+ rel->rd_rel->relnamespace);
+ if (!OidIsValid(indexOid))
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("index \"%s\" for table \"%s\" does not exist",
+ indexname, RelationGetRelationName(rel)));
+ }
+ else
+ indexOid = InvalidOid;
+
+ return indexOid;
+}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 733ef40ae7c..8863ad0e8bd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -2287,7 +2287,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
cluster_params.options |= CLUOPT_VERBOSE;
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
- cluster_rel(rel, InvalidOid, &cluster_params);
+ cluster_rel(REPACK_COMMAND_VACUUMFULL, false, rel, InvalidOid,
+ &cluster_params);
/* cluster_rel closes the relation, but keeps lock */
rel = NULL;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index db43034b9db..f9152728021 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -280,7 +280,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
AlterCompositeTypeStmt AlterUserMappingStmt
AlterRoleStmt AlterRoleSetStmt AlterPolicyStmt AlterStatsStmt
AlterDefaultPrivilegesStmt DefACLAction
- AnalyzeStmt CallStmt ClosePortalStmt ClusterStmt CommentStmt
+ AnalyzeStmt CallStmt ClosePortalStmt CommentStmt
ConstraintsSetStmt CopyStmt CreateAsStmt CreateCastStmt
CreateDomainStmt CreateExtensionStmt CreateGroupStmt CreateOpClassStmt
CreateOpFamilyStmt AlterOpFamilyStmt CreatePLangStmt
@@ -297,7 +297,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
GrantStmt GrantRoleStmt ImportForeignSchemaStmt IndexStmt InsertStmt
ListenStmt LoadStmt LockStmt MergeStmt NotifyStmt ExplainableStmt PreparableStmt
CreateFunctionStmt AlterFunctionStmt ReindexStmt RemoveAggrStmt
- RemoveFuncStmt RemoveOperStmt RenameStmt ReturnStmt RevokeStmt RevokeRoleStmt
+ RemoveFuncStmt RemoveOperStmt RenameStmt RepackStmt ReturnStmt RevokeStmt RevokeRoleStmt
RuleActionStmt RuleActionStmtOrEmpty RuleStmt
SecLabelStmt SelectStmt TransactionStmt TransactionStmtLegacy TruncateStmt
UnlistenStmt UpdateStmt VacuumStmt
@@ -316,7 +316,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <str> opt_single_name
%type <list> opt_qualified_name
-%type <boolean> opt_concurrently
+%type <boolean> opt_concurrently opt_usingindex
%type <dbehavior> opt_drop_behavior
%type <list> opt_utility_option_list
%type <list> utility_option_list
@@ -763,7 +763,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
QUOTE QUOTES
RANGE READ REAL REASSIGN RECURSIVE REF_P REFERENCES REFERENCING
- REFRESH REINDEX RELATIVE_P RELEASE RENAME REPEATABLE REPLACE REPLICA
+ REFRESH REINDEX RELATIVE_P RELEASE RENAME REPACK REPEATABLE REPLACE REPLICA
RESET RESTART RESTRICT RETURN RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROLLUP
ROUTINE ROUTINES ROW ROWS RULE
@@ -1025,7 +1025,6 @@ stmt:
| CallStmt
| CheckPointStmt
| ClosePortalStmt
- | ClusterStmt
| CommentStmt
| ConstraintsSetStmt
| CopyStmt
@@ -1099,6 +1098,7 @@ stmt:
| RemoveFuncStmt
| RemoveOperStmt
| RenameStmt
+ | RepackStmt
| RevokeStmt
| RevokeRoleStmt
| RuleStmt
@@ -1135,6 +1135,11 @@ opt_concurrently:
| /*EMPTY*/ { $$ = false; }
;
+opt_usingindex:
+ USING INDEX { $$ = true; }
+ | /* EMPTY */ { $$ = false; }
+ ;
+
opt_drop_behavior:
CASCADE { $$ = DROP_CASCADE; }
| RESTRICT { $$ = DROP_RESTRICT; }
@@ -11912,38 +11917,91 @@ CreateConversionStmt:
/*****************************************************************************
*
* QUERY:
+ * REPACK [ (options) ] [ <qualified_name> [ USING INDEX <index_name> ] ]
+ *
+ * obsolete variants:
* CLUSTER (options) [ <qualified_name> [ USING <index_name> ] ]
* CLUSTER [VERBOSE] [ <qualified_name> [ USING <index_name> ] ]
* CLUSTER [VERBOSE] <index_name> ON <qualified_name> (for pre-8.3)
*
*****************************************************************************/
-ClusterStmt:
- CLUSTER '(' utility_option_list ')' qualified_name cluster_index_specification
+RepackStmt:
+ REPACK opt_utility_option_list qualified_name USING INDEX name
{
- ClusterStmt *n = makeNode(ClusterStmt);
+ RepackStmt *n = makeNode(RepackStmt);
+ n->command = REPACK_COMMAND_REPACK;
+ n->relation = $3;
+ n->indexname = $6;
+ n->usingindex = true;
+ n->params = $2;
+ $$ = (Node *) n;
+ }
+ | REPACK opt_utility_option_list qualified_name opt_usingindex
+ {
+ RepackStmt *n = makeNode(RepackStmt);
+
+ n->command = REPACK_COMMAND_REPACK;
+ n->relation = $3;
+ n->indexname = NULL;
+ n->usingindex = $4;
+ n->params = $2;
+ $$ = (Node *) n;
+ }
+ | REPACK '(' utility_option_list ')'
+ {
+ RepackStmt *n = makeNode(RepackStmt);
+
+ n->command = REPACK_COMMAND_REPACK;
+ n->relation = NULL;
+ n->indexname = NULL;
+ n->usingindex = false;
+ n->params = $3;
+ $$ = (Node *) n;
+ }
+ | REPACK opt_usingindex
+ {
+ RepackStmt *n = makeNode(RepackStmt);
+
+ n->command = REPACK_COMMAND_REPACK;
+ n->relation = NULL;
+ n->indexname = NULL;
+ n->usingindex = $2;
+ n->params = NIL;
+ $$ = (Node *) n;
+ }
+ | CLUSTER '(' utility_option_list ')' qualified_name cluster_index_specification
+ {
+ RepackStmt *n = makeNode(RepackStmt);
+
+ n->command = REPACK_COMMAND_CLUSTER;
n->relation = $5;
n->indexname = $6;
+ n->usingindex = true;
n->params = $3;
$$ = (Node *) n;
}
| CLUSTER opt_utility_option_list
{
- ClusterStmt *n = makeNode(ClusterStmt);
+ RepackStmt *n = makeNode(RepackStmt);
+ n->command = REPACK_COMMAND_CLUSTER;
n->relation = NULL;
n->indexname = NULL;
+ n->usingindex = true;
n->params = $2;
$$ = (Node *) n;
}
/* unparenthesized VERBOSE kept for pre-14 compatibility */
| CLUSTER opt_verbose qualified_name cluster_index_specification
{
- ClusterStmt *n = makeNode(ClusterStmt);
+ RepackStmt *n = makeNode(RepackStmt);
+ n->command = REPACK_COMMAND_CLUSTER;
n->relation = $3;
n->indexname = $4;
+ n->usingindex = true;
if ($2)
n->params = list_make1(makeDefElem("verbose", NULL, @2));
$$ = (Node *) n;
@@ -11951,20 +12009,24 @@ ClusterStmt:
/* unparenthesized VERBOSE kept for pre-17 compatibility */
| CLUSTER VERBOSE
{
- ClusterStmt *n = makeNode(ClusterStmt);
+ RepackStmt *n = makeNode(RepackStmt);
+ n->command = REPACK_COMMAND_CLUSTER;
n->relation = NULL;
n->indexname = NULL;
+ n->usingindex = true;
n->params = list_make1(makeDefElem("verbose", NULL, @2));
$$ = (Node *) n;
}
/* kept for pre-8.3 compatibility */
| CLUSTER opt_verbose name ON qualified_name
{
- ClusterStmt *n = makeNode(ClusterStmt);
+ RepackStmt *n = makeNode(RepackStmt);
+ n->command = REPACK_COMMAND_CLUSTER;
n->relation = $5;
n->indexname = $3;
+ n->usingindex = true;
if ($2)
n->params = list_make1(makeDefElem("verbose", NULL, @2));
$$ = (Node *) n;
@@ -17960,6 +18022,7 @@ unreserved_keyword:
| RELATIVE_P
| RELEASE
| RENAME
+ | REPACK
| REPEATABLE
| REPLACE
| REPLICA
@@ -18592,6 +18655,7 @@ bare_label_keyword:
| RELATIVE_P
| RELEASE
| RENAME
+ | REPACK
| REPEATABLE
| REPLACE
| REPLICA
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 5f442bc3bd4..cf6db581007 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -277,9 +277,9 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
return COMMAND_OK_IN_RECOVERY | COMMAND_OK_IN_READ_ONLY_TXN;
}
- case T_ClusterStmt:
case T_ReindexStmt:
case T_VacuumStmt:
+ case T_RepackStmt:
{
/*
* These commands write WAL, so they're not strictly
@@ -854,14 +854,14 @@ standard_ProcessUtility(PlannedStmt *pstmt,
ExecuteCallStmt(castNode(CallStmt, parsetree), params, isAtomicContext, dest);
break;
- case T_ClusterStmt:
- cluster(pstate, (ClusterStmt *) parsetree, isTopLevel);
- break;
-
case T_VacuumStmt:
ExecVacuum(pstate, (VacuumStmt *) parsetree, isTopLevel);
break;
+ case T_RepackStmt:
+ ExecRepack(pstate, (RepackStmt *) parsetree, isTopLevel);
+ break;
+
case T_ExplainStmt:
ExplainQuery(pstate, (ExplainStmt *) parsetree, params, dest);
break;
@@ -2851,10 +2851,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CALL;
break;
- case T_ClusterStmt:
- tag = CMDTAG_CLUSTER;
- break;
-
case T_VacuumStmt:
if (((VacuumStmt *) parsetree)->is_vacuumcmd)
tag = CMDTAG_VACUUM;
@@ -2862,6 +2858,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_ANALYZE;
break;
+ case T_RepackStmt:
+ tag = CMDTAG_REPACK;
+ break;
+
case T_ExplainStmt:
tag = CMDTAG_EXPLAIN;
break;
@@ -3499,7 +3499,7 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_ALL;
break;
- case T_ClusterStmt:
+ case T_RepackStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..a1e10e8c2f6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -268,6 +268,8 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
cmdtype = PROGRESS_COMMAND_ANALYZE;
else if (pg_strcasecmp(cmd, "CLUSTER") == 0)
cmdtype = PROGRESS_COMMAND_CLUSTER;
+ else if (pg_strcasecmp(cmd, "REPACK") == 0)
+ cmdtype = PROGRESS_COMMAND_REPACK;
else if (pg_strcasecmp(cmd, "CREATE INDEX") == 0)
cmdtype = PROGRESS_COMMAND_CREATE_INDEX;
else if (pg_strcasecmp(cmd, "BASEBACKUP") == 0)
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 8b10f2313f3..59ff6e0923b 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -1247,7 +1247,7 @@ static const char *const sql_commands[] = {
"DELETE FROM", "DISCARD", "DO", "DROP", "END", "EXECUTE", "EXPLAIN",
"FETCH", "GRANT", "IMPORT FOREIGN SCHEMA", "INSERT INTO", "LISTEN", "LOAD", "LOCK",
"MERGE INTO", "MOVE", "NOTIFY", "PREPARE",
- "REASSIGN", "REFRESH MATERIALIZED VIEW", "REINDEX", "RELEASE",
+ "REASSIGN", "REFRESH MATERIALIZED VIEW", "REINDEX", "RELEASE", "REPACK",
"RESET", "REVOKE", "ROLLBACK",
"SAVEPOINT", "SECURITY LABEL", "SELECT", "SET", "SHOW", "START",
"TABLE", "TRUNCATE", "UNLISTEN", "UPDATE", "VACUUM", "VALUES", "WITH",
@@ -4997,6 +4997,37 @@ match_previous_words(int pattern_id,
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
+/* REPACK */
+ else if (Matches("REPACK"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_clusterables);
+ else if (Matches("REPACK", "(*)"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_clusterables);
+ /* If we have REPACK <sth>, then add "USING INDEX" */
+ else if (Matches("REPACK", MatchAnyExcept("(")))
+ COMPLETE_WITH("USING INDEX");
+ /* If we have REPACK (*) <sth>, then add "USING INDEX" */
+ else if (Matches("REPACK", "(*)", MatchAny))
+ COMPLETE_WITH("USING INDEX");
+ /* If we have REPACK <sth> USING, then add the index as well */
+ else if (Matches("REPACK", MatchAny, "USING", "INDEX"))
+ {
+ set_completion_reference(prev3_wd);
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_index_of_table);
+ }
+ else if (HeadMatches("REPACK", "(*") &&
+ !HeadMatches("REPACK", "(*)"))
+ {
+ /*
+ * This fires if we're in an unfinished parenthesized option list.
+ * get_previous_words treats a completed parenthesized option list as
+ * one word, so the above test is correct.
+ */
+ if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
+ COMPLETE_WITH("VERBOSE");
+ else if (TailMatches("VERBOSE"))
+ COMPLETE_WITH("ON", "OFF");
+ }
+
/* SECURITY LABEL */
else if (Matches("SECURITY"))
COMPLETE_WITH("LABEL");
diff --git a/src/bin/scripts/Makefile b/src/bin/scripts/Makefile
index 019ca06455d..f0c1bd4175c 100644
--- a/src/bin/scripts/Makefile
+++ b/src/bin/scripts/Makefile
@@ -16,7 +16,7 @@ subdir = src/bin/scripts
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-PROGRAMS = createdb createuser dropdb dropuser clusterdb vacuumdb reindexdb pg_isready
+PROGRAMS = createdb createuser dropdb dropuser clusterdb vacuumdb reindexdb pg_isready pg_repackdb
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
@@ -31,6 +31,7 @@ clusterdb: clusterdb.o common.o $(WIN32RES) | submake-libpq submake-libpgport su
vacuumdb: vacuumdb.o vacuuming.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
reindexdb: reindexdb.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
pg_isready: pg_isready.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+pg_repackdb: pg_repackdb.o vacuuming.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
install: all installdirs
$(INSTALL_PROGRAM) createdb$(X) '$(DESTDIR)$(bindir)'/createdb$(X)
@@ -41,6 +42,7 @@ install: all installdirs
$(INSTALL_PROGRAM) vacuumdb$(X) '$(DESTDIR)$(bindir)'/vacuumdb$(X)
$(INSTALL_PROGRAM) reindexdb$(X) '$(DESTDIR)$(bindir)'/reindexdb$(X)
$(INSTALL_PROGRAM) pg_isready$(X) '$(DESTDIR)$(bindir)'/pg_isready$(X)
+ $(INSTALL_PROGRAM) pg_repackdb$(X) '$(DESTDIR)$(bindir)'/pg_repackdb$(X)
installdirs:
$(MKDIR_P) '$(DESTDIR)$(bindir)'
diff --git a/src/bin/scripts/meson.build b/src/bin/scripts/meson.build
index a4fed59d1c9..18410fb80dd 100644
--- a/src/bin/scripts/meson.build
+++ b/src/bin/scripts/meson.build
@@ -42,6 +42,7 @@ vacuuming_common = static_library('libvacuuming_common',
binaries = [
'vacuumdb',
+ 'pg_repackdb'
]
foreach binary : binaries
binary_sources = files('@0@.c'.format(binary))
@@ -80,6 +81,7 @@ tests += {
't/100_vacuumdb.pl',
't/101_vacuumdb_all.pl',
't/102_vacuumdb_stages.pl',
+ 't/103_repackdb.pl',
't/200_connstr.pl',
],
},
diff --git a/src/bin/scripts/pg_repackdb.c b/src/bin/scripts/pg_repackdb.c
new file mode 100644
index 00000000000..23326372a77
--- /dev/null
+++ b/src/bin/scripts/pg_repackdb.c
@@ -0,0 +1,226 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_repackdb
+ * An utility to run REPACK
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * FIXME: this is missing a way to specify the index to use to repack one
+ * table, or whether to pass a WITH INDEX clause when multiple tables are
+ * used. Something like --index[=indexname]. Adding that bleeds into
+ * vacuuming.c as well.
+ *
+ * src/bin/scripts/pg_repackdb.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+
+#include "common.h"
+#include "common/logging.h"
+#include "fe_utils/option_utils.h"
+#include "vacuuming.h"
+
+static void help(const char *progname);
+void check_objfilter(void);
+
+int
+main(int argc, char *argv[])
+{
+ static struct option long_options[] = {
+ {"host", required_argument, NULL, 'h'},
+ {"port", required_argument, NULL, 'p'},
+ {"username", required_argument, NULL, 'U'},
+ {"no-password", no_argument, NULL, 'w'},
+ {"password", no_argument, NULL, 'W'},
+ {"echo", no_argument, NULL, 'e'},
+ {"quiet", no_argument, NULL, 'q'},
+ {"dbname", required_argument, NULL, 'd'},
+ {"all", no_argument, NULL, 'a'},
+ {"table", required_argument, NULL, 't'},
+ {"verbose", no_argument, NULL, 'v'},
+ {"jobs", required_argument, NULL, 'j'},
+ {"schema", required_argument, NULL, 'n'},
+ {"exclude-schema", required_argument, NULL, 'N'},
+ {"maintenance-db", required_argument, NULL, 2},
+ {NULL, 0, NULL, 0}
+ };
+
+ const char *progname;
+ int optindex;
+ int c;
+ const char *dbname = NULL;
+ const char *maintenance_db = NULL;
+ ConnParams cparams;
+ bool echo = false;
+ bool quiet = false;
+ vacuumingOptions vacopts;
+ SimpleStringList objects = {NULL, NULL};
+ int concurrentCons = 1;
+ int tbl_count = 0;
+
+ /* initialize options */
+ memset(&vacopts, 0, sizeof(vacopts));
+ vacopts.mode = MODE_REPACK;
+
+ /* the same for connection parameters */
+ memset(&cparams, 0, sizeof(cparams));
+ cparams.prompt_password = TRI_DEFAULT;
+
+ pg_logging_init(argv[0]);
+ progname = get_progname(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
+
+ handle_help_version_opts(argc, argv, progname, help);
+
+ while ((c = getopt_long(argc, argv, "ad:eh:j:n:N:p:qt:U:vwW",
+ long_options, &optindex)) != -1)
+ {
+ switch (c)
+ {
+ case 'a':
+ objfilter |= OBJFILTER_ALL_DBS;
+ break;
+ case 'd':
+ objfilter |= OBJFILTER_DATABASE;
+ dbname = pg_strdup(optarg);
+ break;
+ case 'e':
+ echo = true;
+ break;
+ case 'h':
+ cparams.pghost = pg_strdup(optarg);
+ break;
+ case 'j':
+ if (!option_parse_int(optarg, "-j/--jobs", 1, INT_MAX,
+ &concurrentCons))
+ exit(1);
+ break;
+ case 'n':
+ objfilter |= OBJFILTER_SCHEMA;
+ simple_string_list_append(&objects, optarg);
+ break;
+ case 'N':
+ objfilter |= OBJFILTER_SCHEMA_EXCLUDE;
+ simple_string_list_append(&objects, optarg);
+ break;
+ case 'p':
+ cparams.pgport = pg_strdup(optarg);
+ break;
+ case 'q':
+ quiet = true;
+ break;
+ case 't':
+ objfilter |= OBJFILTER_TABLE;
+ simple_string_list_append(&objects, optarg);
+ tbl_count++;
+ break;
+ case 'U':
+ cparams.pguser = pg_strdup(optarg);
+ break;
+ case 'v':
+ vacopts.verbose = true;
+ break;
+ case 'w':
+ cparams.prompt_password = TRI_NO;
+ break;
+ case 'W':
+ cparams.prompt_password = TRI_YES;
+ break;
+ case 2:
+ maintenance_db = pg_strdup(optarg);
+ break;
+ default:
+ /* getopt_long already emitted a complaint */
+ pg_log_error_hint("Try \"%s --help\" for more information.", progname);
+ exit(1);
+ }
+ }
+
+ /*
+ * Non-option argument specifies database name as long as it wasn't
+ * already specified with -d / --dbname
+ */
+ if (optind < argc && dbname == NULL)
+ {
+ objfilter |= OBJFILTER_DATABASE;
+ dbname = argv[optind];
+ optind++;
+ }
+
+ if (optind < argc)
+ {
+ pg_log_error("too many command-line arguments (first is \"%s\")",
+ argv[optind]);
+ pg_log_error_hint("Try \"%s --help\" for more information.", progname);
+ exit(1);
+ }
+
+ /*
+ * Validate the combination of filters specified in the command-line
+ * options.
+ */
+ check_objfilter();
+
+ vacuuming_main(&cparams, dbname, maintenance_db, &vacopts, &objects,
+ false, tbl_count, concurrentCons,
+ progname, echo, quiet);
+ exit(0);
+}
+
+/*
+ * Verify that the filters used at command line are compatible.
+ */
+void
+check_objfilter(void)
+{
+ if ((objfilter & OBJFILTER_ALL_DBS) &&
+ (objfilter & OBJFILTER_DATABASE))
+ pg_fatal("cannot repack all databases and a specific one at the same time");
+
+ if ((objfilter & OBJFILTER_TABLE) &&
+ (objfilter & OBJFILTER_SCHEMA))
+ pg_fatal("cannot repack all tables in schema(s) and specific table(s) at the same time");
+
+ if ((objfilter & OBJFILTER_TABLE) &&
+ (objfilter & OBJFILTER_SCHEMA_EXCLUDE))
+ pg_fatal("cannot repack specific table(s) and exclude schema(s) at the same time");
+
+ if ((objfilter & OBJFILTER_SCHEMA) &&
+ (objfilter & OBJFILTER_SCHEMA_EXCLUDE))
+ pg_fatal("cannot repack all tables in schema(s) and exclude schema(s) at the same time");
+}
+
+static void
+help(const char *progname)
+{
+ printf(_("%s repacks a PostgreSQL database.\n\n"), progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION]... [DBNAME]\n"), progname);
+ printf(_("\nOptions:\n"));
+ printf(_(" -a, --all repack all databases\n"));
+ printf(_(" -d, --dbname=DBNAME database to repack\n"));
+ printf(_(" -e, --echo show the commands being sent to the server\n"));
+ printf(_(" -j, --jobs=NUM use this many concurrent connections to repack\n"));
+ printf(_(" -n, --schema=SCHEMA repack tables in the specified schema(s) only\n"));
+ printf(_(" -N, --exclude-schema=SCHEMA do not repack tables in the specified schema(s)\n"));
+ printf(_(" -q, --quiet don't write any messages\n"));
+ printf(_(" -t, --table='TABLE' repack specific table(s) only\n"));
+ printf(_(" -v, --verbose write a lot of output\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_("\nConnection options:\n"));
+ printf(_(" -h, --host=HOSTNAME database server host or socket directory\n"));
+ printf(_(" -p, --port=PORT database server port\n"));
+ printf(_(" -U, --username=USERNAME user name to connect as\n"));
+ printf(_(" -w, --no-password never prompt for password\n"));
+ printf(_(" -W, --password force password prompt\n"));
+ printf(_(" --maintenance-db=DBNAME alternate maintenance database\n"));
+ printf(_("\nRead the description of the SQL command REPACK for details.\n"));
+ printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
+ printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
+}
diff --git a/src/bin/scripts/t/103_repackdb.pl b/src/bin/scripts/t/103_repackdb.pl
new file mode 100644
index 00000000000..51de4d7ab34
--- /dev/null
+++ b/src/bin/scripts/t/103_repackdb.pl
@@ -0,0 +1,24 @@
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+program_help_ok('pg_repackdb');
+program_version_ok('pg_repackdb');
+program_options_handling_ok('pg_repackdb');
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+$node->init;
+$node->start;
+
+$node->issues_sql_like(
+ [ 'pg_repackdb', 'postgres' ],
+ qr/statement: REPACK.*;/,
+ 'SQL REPACK run');
+
+
+done_testing();
diff --git a/src/bin/scripts/vacuuming.c b/src/bin/scripts/vacuuming.c
index 9be37fcc45a..e07071c38ee 100644
--- a/src/bin/scripts/vacuuming.c
+++ b/src/bin/scripts/vacuuming.c
@@ -1,6 +1,6 @@
/*-------------------------------------------------------------------------
* vacuuming.c
- * Common routines for vacuumdb
+ * Common routines for vacuumdb and pg_repackdb
*
* Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -166,6 +166,14 @@ vacuum_one_database(ConnParams *cparams,
conn = connectDatabase(cparams, progname, echo, false, true);
+ if (vacopts->mode == MODE_REPACK && PQserverVersion(conn) < 190000)
+ {
+ /* XXX arguably, here we should use VACUUM FULL instead of failing */
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" command on server versions older than PostgreSQL %s",
+ "REPACK", "19");
+ }
+
if (vacopts->disable_page_skipping && PQserverVersion(conn) < 90600)
{
PQfinish(conn);
@@ -258,9 +266,15 @@ vacuum_one_database(ConnParams *cparams,
if (stage != ANALYZE_NO_STAGE)
printf(_("%s: processing database \"%s\": %s\n"),
progname, PQdb(conn), _(stage_messages[stage]));
- else
+ else if (vacopts->mode == MODE_VACUUM)
printf(_("%s: vacuuming database \"%s\"\n"),
progname, PQdb(conn));
+ else
+ {
+ Assert(vacopts->mode == MODE_REPACK);
+ printf(_("%s: repacking database \"%s\"\n"),
+ progname, PQdb(conn));
+ }
fflush(stdout);
}
@@ -350,7 +364,7 @@ vacuum_one_database(ConnParams *cparams,
* through ParallelSlotsGetIdle.
*/
ParallelSlotSetHandler(free_slot, TableCommandResultHandler, NULL);
- run_vacuum_command(free_slot->connection, sql.data,
+ run_vacuum_command(free_slot->connection, vacopts, sql.data,
echo, tabname);
cell = cell->next;
@@ -363,7 +377,7 @@ vacuum_one_database(ConnParams *cparams,
}
/* If we used SKIP_DATABASE_STATS, mop up with ONLY_DATABASE_STATS */
- if (vacopts->skip_database_stats &&
+ if (vacopts->mode == MODE_VACUUM && vacopts->skip_database_stats &&
stage == ANALYZE_NO_STAGE)
{
const char *cmd = "VACUUM (ONLY_DATABASE_STATS);";
@@ -376,7 +390,7 @@ vacuum_one_database(ConnParams *cparams,
}
ParallelSlotSetHandler(free_slot, TableCommandResultHandler, NULL);
- run_vacuum_command(free_slot->connection, cmd, echo, NULL);
+ run_vacuum_command(free_slot->connection, vacopts, cmd, echo, NULL);
if (!ParallelSlotsWaitCompletion(sa))
failed = true;
@@ -708,6 +722,12 @@ vacuum_all_databases(ConnParams *cparams,
int i;
conn = connectMaintenanceDatabase(cparams, progname, echo);
+ if (vacopts->mode == MODE_REPACK && PQserverVersion(conn) < 190000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" command on server versions older than PostgreSQL %s",
+ "REPACK", "19");
+ }
result = executeQuery(conn,
"SELECT datname FROM pg_database WHERE datallowconn AND datconnlimit <> -2 ORDER BY 1;",
echo);
@@ -761,7 +781,7 @@ vacuum_all_databases(ConnParams *cparams,
}
/*
- * Construct a vacuum/analyze command to run based on the given
+ * Construct a vacuum/analyze/repack command to run based on the given
* options, in the given string buffer, which may contain previous garbage.
*
* The table name used must be already properly quoted. The command generated
@@ -777,7 +797,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
resetPQExpBuffer(sql);
- if (vacopts->analyze_only)
+ if (vacopts->mode == MODE_REPACK)
+ {
+ appendPQExpBufferStr(sql, "REPACK");
+ if (vacopts->verbose)
+ appendPQExpBufferStr(sql, " (VERBOSE)");
+ }
+ else if (vacopts->analyze_only)
{
appendPQExpBufferStr(sql, "ANALYZE");
@@ -938,8 +964,8 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
* Any errors during command execution are reported to stderr.
*/
void
-run_vacuum_command(PGconn *conn, const char *sql, bool echo,
- const char *table)
+run_vacuum_command(PGconn *conn, vacuumingOptions *vacopts,
+ const char *sql, bool echo, const char *table)
{
bool status;
@@ -952,13 +978,21 @@ run_vacuum_command(PGconn *conn, const char *sql, bool echo,
{
if (table)
{
- pg_log_error("vacuuming of table \"%s\" in database \"%s\" failed: %s",
- table, PQdb(conn), PQerrorMessage(conn));
+ if (vacopts->mode == MODE_VACUUM)
+ pg_log_error("vacuuming of table \"%s\" in database \"%s\" failed: %s",
+ table, PQdb(conn), PQerrorMessage(conn));
+ else
+ pg_log_error("repacking of table \"%s\" in database \"%s\" failed: %s",
+ table, PQdb(conn), PQerrorMessage(conn));
}
else
{
- pg_log_error("vacuuming of database \"%s\" failed: %s",
- PQdb(conn), PQerrorMessage(conn));
+ if (vacopts->mode == MODE_VACUUM)
+ pg_log_error("vacuuming of database \"%s\" failed: %s",
+ PQdb(conn), PQerrorMessage(conn));
+ else
+ pg_log_error("repacking of database \"%s\" failed: %s",
+ PQdb(conn), PQerrorMessage(conn));
}
}
}
diff --git a/src/bin/scripts/vacuuming.h b/src/bin/scripts/vacuuming.h
index d3f000840fa..154bc9925c0 100644
--- a/src/bin/scripts/vacuuming.h
+++ b/src/bin/scripts/vacuuming.h
@@ -17,6 +17,12 @@
#include "fe_utils/connect_utils.h"
#include "fe_utils/simple_list.h"
+typedef enum
+{
+ MODE_VACUUM,
+ MODE_REPACK
+} RunMode;
+
/* For analyze-in-stages mode */
#define ANALYZE_NO_STAGE -1
#define ANALYZE_NUM_STAGES 3
@@ -24,6 +30,7 @@
/* vacuum options controlled by user flags */
typedef struct vacuumingOptions
{
+ RunMode mode;
bool analyze_only;
bool verbose;
bool and_analyze;
@@ -87,8 +94,8 @@ extern void vacuum_all_databases(ConnParams *cparams,
extern void prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
vacuumingOptions *vacopts, const char *table);
-extern void run_vacuum_command(PGconn *conn, const char *sql, bool echo,
- const char *table);
+extern void run_vacuum_command(PGconn *conn, vacuumingOptions *vacopts,
+ const char *sql, bool echo, const char *table);
extern char *escape_quotes(const char *src);
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
index 60088a64cbb..890998d84bb 100644
--- a/src/include/commands/cluster.h
+++ b/src/include/commands/cluster.h
@@ -24,6 +24,7 @@
#define CLUOPT_RECHECK 0x02 /* recheck relation state */
#define CLUOPT_RECHECK_ISCLUSTERED 0x04 /* recheck relation state for
* indisclustered */
+#define CLUOPT_ANALYZE 0x08 /* do an ANALYZE */
/* options for CLUSTER */
typedef struct ClusterParams
@@ -31,8 +32,11 @@ typedef struct ClusterParams
bits32 options; /* bitmask of CLUOPT_* */
} ClusterParams;
-extern void cluster(ParseState *pstate, ClusterStmt *stmt, bool isTopLevel);
-extern void cluster_rel(Relation OldHeap, Oid indexOid, ClusterParams *params);
+
+extern void ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel);
+
+extern void cluster_rel(RepackCommand command, bool usingindex,
+ Relation OldHeap, Oid indexOid, ClusterParams *params);
extern void check_index_is_clusterable(Relation OldHeap, Oid indexOid,
LOCKMODE lockmode);
extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal);
diff --git a/src/include/commands/progress.h b/src/include/commands/progress.h
index 1cde4bd9bcf..5b6639c114c 100644
--- a/src/include/commands/progress.h
+++ b/src/include/commands/progress.h
@@ -56,24 +56,51 @@
#define PROGRESS_ANALYZE_PHASE_COMPUTE_EXT_STATS 4
#define PROGRESS_ANALYZE_PHASE_FINALIZE_ANALYZE 5
-/* Progress parameters for cluster */
-#define PROGRESS_CLUSTER_COMMAND 0
-#define PROGRESS_CLUSTER_PHASE 1
-#define PROGRESS_CLUSTER_INDEX_RELID 2
-#define PROGRESS_CLUSTER_HEAP_TUPLES_SCANNED 3
-#define PROGRESS_CLUSTER_HEAP_TUPLES_WRITTEN 4
-#define PROGRESS_CLUSTER_TOTAL_HEAP_BLKS 5
-#define PROGRESS_CLUSTER_HEAP_BLKS_SCANNED 6
-#define PROGRESS_CLUSTER_INDEX_REBUILD_COUNT 7
+/*
+ * Progress parameters for REPACK.
+ *
+ * Note: Since REPACK shares some code with CLUSTER, these values are also
+ * used by CLUSTER. (CLUSTER is now deprecated, so it makes little sense to
+ * introduce a separate set of constants.)
+ */
+#define PROGRESS_REPACK_COMMAND 0
+#define PROGRESS_REPACK_PHASE 1
+#define PROGRESS_REPACK_INDEX_RELID 2
+#define PROGRESS_REPACK_HEAP_TUPLES_SCANNED 3
+#define PROGRESS_REPACK_HEAP_TUPLES_WRITTEN 4
+#define PROGRESS_REPACK_TOTAL_HEAP_BLKS 5
+#define PROGRESS_REPACK_HEAP_BLKS_SCANNED 6
+#define PROGRESS_REPACK_INDEX_REBUILD_COUNT 7
-/* Phases of cluster (as advertised via PROGRESS_CLUSTER_PHASE) */
-#define PROGRESS_CLUSTER_PHASE_SEQ_SCAN_HEAP 1
-#define PROGRESS_CLUSTER_PHASE_INDEX_SCAN_HEAP 2
-#define PROGRESS_CLUSTER_PHASE_SORT_TUPLES 3
-#define PROGRESS_CLUSTER_PHASE_WRITE_NEW_HEAP 4
-#define PROGRESS_CLUSTER_PHASE_SWAP_REL_FILES 5
-#define PROGRESS_CLUSTER_PHASE_REBUILD_INDEX 6
-#define PROGRESS_CLUSTER_PHASE_FINAL_CLEANUP 7
+/*
+ * Phases of repack (as advertised via PROGRESS_REPACK_PHASE).
+ */
+#define PROGRESS_REPACK_PHASE_SEQ_SCAN_HEAP 1
+#define PROGRESS_REPACK_PHASE_INDEX_SCAN_HEAP 2
+#define PROGRESS_REPACK_PHASE_SORT_TUPLES 3
+#define PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP 4
+#define PROGRESS_REPACK_PHASE_SWAP_REL_FILES 5
+#define PROGRESS_REPACK_PHASE_REBUILD_INDEX 6
+#define PROGRESS_REPACK_PHASE_FINAL_CLEANUP 7
+
+/*
+ * Commands of PROGRESS_REPACK
+ *
+ * Currently we only have one command, so the PROGRESS_REPACK_COMMAND
+ * parameter is not necessary. However it makes cluster.c simpler if we have
+ * the same set of parameters for CLUSTER and REPACK - see the note on REPACK
+ * parameters above.
+ */
+#define PROGRESS_REPACK_COMMAND_REPACK 1
+
+/*
+ * Progress parameters for cluster.
+ *
+ * Although we need to report REPACK and CLUSTER in separate views, the
+ * parameters and phases of CLUSTER are a subset of those of REPACK. Therefore
+ * we just use the appropriate values defined for REPACK above instead of
+ * defining a separate set of constants here.
+ */
/* Commands of PROGRESS_CLUSTER */
#define PROGRESS_CLUSTER_COMMAND_CLUSTER 1
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 86a236bd58b..fcc25a0c592 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3949,16 +3949,26 @@ typedef struct AlterSystemStmt
} AlterSystemStmt;
/* ----------------------
- * Cluster Statement (support pbrown's cluster index implementation)
+ * Repack Statement
* ----------------------
*/
-typedef struct ClusterStmt
+typedef enum RepackCommand
+{
+ REPACK_COMMAND_CLUSTER,
+ REPACK_COMMAND_REPACK,
+ REPACK_COMMAND_VACUUMFULL,
+} RepackCommand;
+
+typedef struct RepackStmt
{
NodeTag type;
- RangeVar *relation; /* relation being indexed, or NULL if all */
- char *indexname; /* original index defined */
+ RepackCommand command; /* type of command being run */
+ RangeVar *relation; /* relation being repacked */
+ char *indexname; /* order tuples by this index */
+ bool usingindex; /* whether USING INDEX is specified */
List *params; /* list of DefElem nodes */
-} ClusterStmt;
+} RepackStmt;
+
/* ----------------------
* Vacuum and Analyze Statements
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index a4af3f717a1..22559369e2c 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -374,6 +374,7 @@ PG_KEYWORD("reindex", REINDEX, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("relative", RELATIVE_P, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("release", RELEASE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("rename", RENAME, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("repack", REPACK, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("repeatable", REPEATABLE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replace", REPLACE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("replica", REPLICA, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index d250a714d59..cceb312f2b3 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -196,6 +196,7 @@ PG_CMDTAG(CMDTAG_REASSIGN_OWNED, "REASSIGN OWNED", false, false, false)
PG_CMDTAG(CMDTAG_REFRESH_MATERIALIZED_VIEW, "REFRESH MATERIALIZED VIEW", true, false, false)
PG_CMDTAG(CMDTAG_REINDEX, "REINDEX", true, false, false)
PG_CMDTAG(CMDTAG_RELEASE, "RELEASE", false, false, false)
+PG_CMDTAG(CMDTAG_REPACK, "REPACK", false, false, false)
PG_CMDTAG(CMDTAG_RESET, "RESET", false, false, false)
PG_CMDTAG(CMDTAG_REVOKE, "REVOKE", true, false, false)
PG_CMDTAG(CMDTAG_REVOKE_ROLE, "REVOKE ROLE", false, false, false)
diff --git a/src/include/utils/backend_progress.h b/src/include/utils/backend_progress.h
index dda813ab407..e69e366dcdc 100644
--- a/src/include/utils/backend_progress.h
+++ b/src/include/utils/backend_progress.h
@@ -28,6 +28,7 @@ typedef enum ProgressCommandType
PROGRESS_COMMAND_CREATE_INDEX,
PROGRESS_COMMAND_BASEBACKUP,
PROGRESS_COMMAND_COPY,
+ PROGRESS_COMMAND_REPACK,
} ProgressCommandType;
#define PGSTAT_NUM_PROGRESS_PARAM 20
diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out
index 4d40a6809ab..5256628b51d 100644
--- a/src/test/regress/expected/cluster.out
+++ b/src/test/regress/expected/cluster.out
@@ -254,6 +254,63 @@ ORDER BY 1;
clstr_tst_pkey
(3 rows)
+-- REPACK handles individual tables identically to CLUSTER, but it's worth
+-- checking if it handles table hierarchies identically as well.
+REPACK clstr_tst USING INDEX clstr_tst_c;
+-- Verify that inheritance link still works
+INSERT INTO clstr_tst_inh VALUES (0, 100, 'in child table 2');
+SELECT a,b,c,substring(d for 30), length(d) from clstr_tst;
+ a | b | c | substring | length
+----+-----+------------------+--------------------------------+--------
+ 10 | 14 | catorce | |
+ 18 | 5 | cinco | |
+ 9 | 4 | cuatro | |
+ 26 | 19 | diecinueve | |
+ 12 | 18 | dieciocho | |
+ 30 | 16 | dieciseis | |
+ 24 | 17 | diecisiete | |
+ 2 | 10 | diez | |
+ 23 | 12 | doce | |
+ 11 | 2 | dos | |
+ 25 | 9 | nueve | |
+ 31 | 8 | ocho | |
+ 1 | 11 | once | |
+ 28 | 15 | quince | |
+ 32 | 6 | seis | xyzzyxyzzyxyzzyxyzzyxyzzyxyzzy | 500000
+ 29 | 7 | siete | |
+ 15 | 13 | trece | |
+ 22 | 30 | treinta | |
+ 17 | 32 | treinta y dos | |
+ 3 | 31 | treinta y uno | |
+ 5 | 3 | tres | |
+ 20 | 1 | uno | |
+ 6 | 20 | veinte | |
+ 14 | 25 | veinticinco | |
+ 21 | 24 | veinticuatro | |
+ 4 | 22 | veintidos | |
+ 19 | 29 | veintinueve | |
+ 16 | 28 | veintiocho | |
+ 27 | 26 | veintiseis | |
+ 13 | 27 | veintisiete | |
+ 7 | 23 | veintitres | |
+ 8 | 21 | veintiuno | |
+ 0 | 100 | in child table | |
+ 0 | 100 | in child table 2 | |
+(34 rows)
+
+-- Verify that foreign key link still works
+INSERT INTO clstr_tst (b, c) VALUES (1111, 'this should fail');
+ERROR: insert or update on table "clstr_tst" violates foreign key constraint "clstr_tst_con"
+DETAIL: Key (b)=(1111) is not present in table "clstr_tst_s".
+SELECT conname FROM pg_constraint WHERE conrelid = 'clstr_tst'::regclass
+ORDER BY 1;
+ conname
+----------------------
+ clstr_tst_a_not_null
+ clstr_tst_con
+ clstr_tst_pkey
+(3 rows)
+
SELECT relname, relkind,
EXISTS(SELECT 1 FROM pg_class WHERE oid = c.reltoastrelid) AS hastoast
FROM pg_class c WHERE relname LIKE 'clstr_tst%' ORDER BY relname;
@@ -381,6 +438,35 @@ SELECT * FROM clstr_1;
2
(2 rows)
+-- REPACK w/o argument performs no ordering, so we can only check which tables
+-- have the relfilenode changed.
+RESET SESSION AUTHORIZATION;
+CREATE TEMP TABLE relnodes_old AS
+(SELECT relname, relfilenode
+FROM pg_class
+WHERE relname IN ('clstr_1', 'clstr_2', 'clstr_3'));
+SET SESSION AUTHORIZATION regress_clstr_user;
+SET client_min_messages = ERROR; -- order of "skipping" warnings may vary
+REPACK;
+RESET client_min_messages;
+RESET SESSION AUTHORIZATION;
+CREATE TEMP TABLE relnodes_new AS
+(SELECT relname, relfilenode
+FROM pg_class
+WHERE relname IN ('clstr_1', 'clstr_2', 'clstr_3'));
+-- Do the actual comparison. Unlike CLUSTER, clstr_3 should have been
+-- processed because there is nothing like clustering index here.
+SELECT o.relname FROM relnodes_old o
+JOIN relnodes_new n ON o.relname = n.relname
+WHERE o.relfilenode <> n.relfilenode
+ORDER BY o.relname;
+ relname
+---------
+ clstr_1
+ clstr_3
+(2 rows)
+
+SET SESSION AUTHORIZATION regress_clstr_user;
-- Test MVCC-safety of cluster. There isn't much we can do to verify the
-- results with a single backend...
CREATE TABLE clustertest (key int PRIMARY KEY);
@@ -495,6 +581,43 @@ ALTER TABLE clstrpart SET WITHOUT CLUSTER;
ERROR: cannot mark index clustered in partitioned table
ALTER TABLE clstrpart CLUSTER ON clstrpart_idx;
ERROR: cannot mark index clustered in partitioned table
+-- Check that REPACK sets new relfilenodes: it should process exactly the same
+-- tables as CLUSTER did.
+DROP TABLE old_cluster_info;
+DROP TABLE new_cluster_info;
+CREATE TEMP TABLE old_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+REPACK clstrpart USING INDEX clstrpart_idx;
+CREATE TEMP TABLE new_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+SELECT relname, old.level, old.relkind, old.relfilenode = new.relfilenode FROM old_cluster_info AS old JOIN new_cluster_info AS new USING (relname) ORDER BY relname COLLATE "C";
+ relname | level | relkind | ?column?
+-------------+-------+---------+----------
+ clstrpart | 0 | p | t
+ clstrpart1 | 1 | p | t
+ clstrpart11 | 2 | r | f
+ clstrpart12 | 2 | p | t
+ clstrpart2 | 1 | r | f
+ clstrpart3 | 1 | p | t
+ clstrpart33 | 2 | r | f
+(7 rows)
+
+-- And finally the same for REPACK w/o index.
+DROP TABLE old_cluster_info;
+DROP TABLE new_cluster_info;
+CREATE TEMP TABLE old_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+REPACK clstrpart;
+CREATE TEMP TABLE new_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+SELECT relname, old.level, old.relkind, old.relfilenode = new.relfilenode FROM old_cluster_info AS old JOIN new_cluster_info AS new USING (relname) ORDER BY relname COLLATE "C";
+ relname | level | relkind | ?column?
+-------------+-------+---------+----------
+ clstrpart | 0 | p | t
+ clstrpart1 | 1 | p | t
+ clstrpart11 | 2 | r | f
+ clstrpart12 | 2 | p | t
+ clstrpart2 | 1 | r | f
+ clstrpart3 | 1 | p | t
+ clstrpart33 | 2 | r | f
+(7 rows)
+
DROP TABLE clstrpart;
-- Ownership of partitions is checked
CREATE TABLE ptnowner(i int unique) PARTITION BY LIST (i);
@@ -513,7 +636,7 @@ CREATE TEMP TABLE ptnowner_oldnodes AS
JOIN pg_class AS c ON c.oid=tree.relid;
SET SESSION AUTHORIZATION regress_ptnowner;
CLUSTER ptnowner USING ptnowner_i_idx;
-WARNING: permission denied to cluster "ptnowner2", skipping it
+WARNING: permission denied to execute CLUSTER on "ptnowner2", skipping it
RESET SESSION AUTHORIZATION;
SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a
JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C";
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..3a1d1d28282 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2071,6 +2071,29 @@ pg_stat_progress_create_index| SELECT s.pid,
s.param15 AS partitions_done
FROM (pg_stat_get_progress_info('CREATE INDEX'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20)
LEFT JOIN pg_database d ON ((s.datid = d.oid)));
+pg_stat_progress_repack| SELECT s.pid,
+ s.datid,
+ d.datname,
+ s.relid,
+ CASE s.param2
+ WHEN 0 THEN 'initializing'::text
+ WHEN 1 THEN 'seq scanning heap'::text
+ WHEN 2 THEN 'index scanning heap'::text
+ WHEN 3 THEN 'sorting tuples'::text
+ WHEN 4 THEN 'writing new heap'::text
+ WHEN 5 THEN 'swapping relation files'::text
+ WHEN 6 THEN 'rebuilding index'::text
+ WHEN 7 THEN 'performing final cleanup'::text
+ ELSE NULL::text
+ END AS phase,
+ (s.param3)::oid AS repack_index_relid,
+ s.param4 AS heap_tuples_scanned,
+ s.param5 AS heap_tuples_written,
+ s.param6 AS heap_blks_total,
+ s.param7 AS heap_blks_scanned,
+ s.param8 AS index_rebuild_count
+ FROM (pg_stat_get_progress_info('REPACK'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20)
+ LEFT JOIN pg_database d ON ((s.datid = d.oid)));
pg_stat_progress_vacuum| SELECT s.pid,
s.datid,
d.datname,
diff --git a/src/test/regress/sql/cluster.sql b/src/test/regress/sql/cluster.sql
index b7115f86104..cfcc3dc9761 100644
--- a/src/test/regress/sql/cluster.sql
+++ b/src/test/regress/sql/cluster.sql
@@ -76,6 +76,19 @@ INSERT INTO clstr_tst (b, c) VALUES (1111, 'this should fail');
SELECT conname FROM pg_constraint WHERE conrelid = 'clstr_tst'::regclass
ORDER BY 1;
+-- REPACK handles individual tables identically to CLUSTER, but it's worth
+-- checking if it handles table hierarchies identically as well.
+REPACK clstr_tst USING INDEX clstr_tst_c;
+
+-- Verify that inheritance link still works
+INSERT INTO clstr_tst_inh VALUES (0, 100, 'in child table 2');
+SELECT a,b,c,substring(d for 30), length(d) from clstr_tst;
+
+-- Verify that foreign key link still works
+INSERT INTO clstr_tst (b, c) VALUES (1111, 'this should fail');
+
+SELECT conname FROM pg_constraint WHERE conrelid = 'clstr_tst'::regclass
+ORDER BY 1;
SELECT relname, relkind,
EXISTS(SELECT 1 FROM pg_class WHERE oid = c.reltoastrelid) AS hastoast
@@ -159,6 +172,34 @@ INSERT INTO clstr_1 VALUES (1);
CLUSTER clstr_1;
SELECT * FROM clstr_1;
+-- REPACK w/o argument performs no ordering, so we can only check which tables
+-- have the relfilenode changed.
+RESET SESSION AUTHORIZATION;
+CREATE TEMP TABLE relnodes_old AS
+(SELECT relname, relfilenode
+FROM pg_class
+WHERE relname IN ('clstr_1', 'clstr_2', 'clstr_3'));
+
+SET SESSION AUTHORIZATION regress_clstr_user;
+SET client_min_messages = ERROR; -- order of "skipping" warnings may vary
+REPACK;
+RESET client_min_messages;
+
+RESET SESSION AUTHORIZATION;
+CREATE TEMP TABLE relnodes_new AS
+(SELECT relname, relfilenode
+FROM pg_class
+WHERE relname IN ('clstr_1', 'clstr_2', 'clstr_3'));
+
+-- Do the actual comparison. Unlike CLUSTER, clstr_3 should have been
+-- processed because there is nothing like clustering index here.
+SELECT o.relname FROM relnodes_old o
+JOIN relnodes_new n ON o.relname = n.relname
+WHERE o.relfilenode <> n.relfilenode
+ORDER BY o.relname;
+
+SET SESSION AUTHORIZATION regress_clstr_user;
+
-- Test MVCC-safety of cluster. There isn't much we can do to verify the
-- results with a single backend...
@@ -229,6 +270,24 @@ SELECT relname, old.level, old.relkind, old.relfilenode = new.relfilenode FROM o
CLUSTER clstrpart;
ALTER TABLE clstrpart SET WITHOUT CLUSTER;
ALTER TABLE clstrpart CLUSTER ON clstrpart_idx;
+
+-- Check that REPACK sets new relfilenodes: it should process exactly the same
+-- tables as CLUSTER did.
+DROP TABLE old_cluster_info;
+DROP TABLE new_cluster_info;
+CREATE TEMP TABLE old_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+REPACK clstrpart USING INDEX clstrpart_idx;
+CREATE TEMP TABLE new_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+SELECT relname, old.level, old.relkind, old.relfilenode = new.relfilenode FROM old_cluster_info AS old JOIN new_cluster_info AS new USING (relname) ORDER BY relname COLLATE "C";
+
+-- And finally the same for REPACK w/o index.
+DROP TABLE old_cluster_info;
+DROP TABLE new_cluster_info;
+CREATE TEMP TABLE old_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+REPACK clstrpart;
+CREATE TEMP TABLE new_cluster_info AS SELECT relname, level, relfilenode, relkind FROM pg_partition_tree('clstrpart'::regclass) AS tree JOIN pg_class c ON c.oid=tree.relid ;
+SELECT relname, old.level, old.relkind, old.relfilenode = new.relfilenode FROM old_cluster_info AS old JOIN new_cluster_info AS new USING (relname) ORDER BY relname COLLATE "C";
+
DROP TABLE clstrpart;
-- Ownership of partitions is checked
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a13e8162890..98242e25432 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2537,6 +2537,8 @@ ReorderBufferTupleCidKey
ReorderBufferUpdateProgressTxnCB
ReorderTuple
RepOriginId
+RepackCommand
+RepackStmt
ReparameterizeForeignPathByChild_function
ReplaceVarsFromTargetList_context
ReplaceVarsNoMatchOption
@@ -2603,6 +2605,7 @@ RtlNtStatusToDosError_t
RuleInfo
RuleLock
RuleStmt
+RunMode
RunningTransactions
RunningTransactionsData
SASLStatus
--
2.43.0
[application/octet-stream] v21-0004-Move-conversion-of-a-historic-to-MVCC-snapshot-t.patch (5.4K, ../../CADzfLwUgPMLiFkXRnk97ugPqkDfsNJ3TRdw9gjJM=8WB4_nXwQ@mail.gmail.com/6-v21-0004-Move-conversion-of-a-historic-to-MVCC-snapshot-t.patch)
download | inline diff:
From b9384aa62c96c94d45bb7e97a56acda5590f0c5f Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Mon, 11 Aug 2025 15:23:05 +0200
Subject: [PATCH v21 4/6] Move conversion of a "historic" to MVCC snapshot to a
separate function.
The conversion is now handled by SnapBuildMVCCFromHistoric(). REPACK
CONCURRENTLY will also need it.
---
src/backend/replication/logical/snapbuild.c | 51 +++++++++++++++++----
src/backend/utils/time/snapmgr.c | 3 +-
src/include/replication/snapbuild.h | 1 +
src/include/utils/snapmgr.h | 1 +
4 files changed, 45 insertions(+), 11 deletions(-)
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 98ddee20929..a2f1803622c 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -440,10 +440,7 @@ Snapshot
SnapBuildInitialSnapshot(SnapBuild *builder)
{
Snapshot snap;
- TransactionId xid;
TransactionId safeXid;
- TransactionId *newxip;
- int newxcnt = 0;
Assert(XactIsoLevel == XACT_REPEATABLE_READ);
Assert(builder->building_full_snapshot);
@@ -485,6 +482,31 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
MyProc->xmin = snap->xmin;
+ /* Convert the historic snapshot to MVCC snapshot. */
+ return SnapBuildMVCCFromHistoric(snap, true);
+}
+
+/*
+ * Turn a historic MVCC snapshot into an ordinary MVCC snapshot.
+ *
+ * Unlike a regular (non-historic) MVCC snapshot, the xip array of this
+ * snapshot contains not only running main transactions, but also their
+ * subtransactions. This difference does has no impact on XidInMVCCSnapshot().
+ *
+ * Pass true for 'in_place' if you don't care about modifying the source
+ * snapshot. If you need a new instance, and one that was allocated as a
+ * single chunk of memory, pass false.
+ */
+Snapshot
+SnapBuildMVCCFromHistoric(Snapshot snapshot, bool in_place)
+{
+ TransactionId xid;
+ TransactionId *oldxip = snapshot->xip;
+ uint32 oldxcnt = snapshot->xcnt;
+ TransactionId *newxip;
+ int newxcnt = 0;
+ Snapshot result;
+
/* allocate in transaction context */
newxip = (TransactionId *)
palloc(sizeof(TransactionId) * GetMaxSnapshotXidCount());
@@ -495,7 +517,7 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
* classical snapshot by marking all non-committed transactions as
* in-progress. This can be expensive.
*/
- for (xid = snap->xmin; NormalTransactionIdPrecedes(xid, snap->xmax);)
+ for (xid = snapshot->xmin; NormalTransactionIdPrecedes(xid, snapshot->xmax);)
{
void *test;
@@ -503,7 +525,7 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
* Check whether transaction committed using the decoding snapshot
* meaning of ->xip.
*/
- test = bsearch(&xid, snap->xip, snap->xcnt,
+ test = bsearch(&xid, snapshot->xip, snapshot->xcnt,
sizeof(TransactionId), xidComparator);
if (test == NULL)
@@ -520,11 +542,22 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
}
/* adjust remaining snapshot fields as needed */
- snap->snapshot_type = SNAPSHOT_MVCC;
- snap->xcnt = newxcnt;
- snap->xip = newxip;
+ snapshot->xcnt = newxcnt;
+ snapshot->xip = newxip;
- return snap;
+ if (in_place)
+ result = snapshot;
+ else
+ {
+ result = CopySnapshot(snapshot);
+
+ /* Restore the original values so the source is intact. */
+ snapshot->xip = oldxip;
+ snapshot->xcnt = oldxcnt;
+ }
+ result->snapshot_type = SNAPSHOT_MVCC;
+
+ return result;
}
/*
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 65561cc6bc3..bc7840052fe 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -212,7 +212,6 @@ typedef struct ExportedSnapshot
static List *exportedSnapshots = NIL;
/* Prototypes for local functions */
-static Snapshot CopySnapshot(Snapshot snapshot);
static void UnregisterSnapshotNoOwner(Snapshot snapshot);
static void FreeSnapshot(Snapshot snapshot);
static void SnapshotResetXmin(void);
@@ -602,7 +601,7 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid,
* The copy is palloc'd in TopTransactionContext and has initial refcounts set
* to 0. The returned snapshot has the copied flag set.
*/
-static Snapshot
+Snapshot
CopySnapshot(Snapshot snapshot)
{
Snapshot newsnap;
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index 44031dcf6e3..6d4d2d1814c 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -73,6 +73,7 @@ extern void FreeSnapshotBuilder(SnapBuild *builder);
extern void SnapBuildSnapDecRefcount(Snapshot snap);
extern Snapshot SnapBuildInitialSnapshot(SnapBuild *builder);
+extern Snapshot SnapBuildMVCCFromHistoric(Snapshot snapshot, bool in_place);
extern const char *SnapBuildExportSnapshot(SnapBuild *builder);
extern void SnapBuildClearExportedSnapshot(void);
extern void SnapBuildResetExportedSnapshotState(void);
diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h
index 604c1f90216..f65f83c85cd 100644
--- a/src/include/utils/snapmgr.h
+++ b/src/include/utils/snapmgr.h
@@ -63,6 +63,7 @@ extern Snapshot GetTransactionSnapshot(void);
extern Snapshot GetLatestSnapshot(void);
extern void SnapshotSetCommandId(CommandId curcid);
+extern Snapshot CopySnapshot(Snapshot snapshot);
extern Snapshot GetCatalogSnapshot(Oid relid);
extern Snapshot GetNonHistoricCatalogSnapshot(Oid relid);
extern void InvalidateCatalogSnapshot(void);
--
2.43.0
[application/octet-stream] v21-0001-Split-vacuumdb-to-create-vacuuming.c-h.patch (69.3K, ../../CADzfLwUgPMLiFkXRnk97ugPqkDfsNJ3TRdw9gjJM=8WB4_nXwQ@mail.gmail.com/7-v21-0001-Split-vacuumdb-to-create-vacuuming.c-h.patch)
download | inline diff:
From 2206b215a8855cf8a9c29889f5feab4a0bd8a7e0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
Date: Sat, 30 Aug 2025 14:39:49 +0200
Subject: [PATCH v21 1/6] Split vacuumdb to create vacuuming.c/h
---
src/bin/scripts/Makefile | 4 +-
src/bin/scripts/meson.build | 28 +-
src/bin/scripts/vacuumdb.c | 1048 +----------------------------------
src/bin/scripts/vacuuming.c | 978 ++++++++++++++++++++++++++++++++
src/bin/scripts/vacuuming.h | 95 ++++
5 files changed, 1119 insertions(+), 1034 deletions(-)
create mode 100644 src/bin/scripts/vacuuming.c
create mode 100644 src/bin/scripts/vacuuming.h
diff --git a/src/bin/scripts/Makefile b/src/bin/scripts/Makefile
index f6b4d40810b..019ca06455d 100644
--- a/src/bin/scripts/Makefile
+++ b/src/bin/scripts/Makefile
@@ -28,7 +28,7 @@ createuser: createuser.o common.o $(WIN32RES) | submake-libpq submake-libpgport
dropdb: dropdb.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
dropuser: dropuser.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
clusterdb: clusterdb.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
-vacuumdb: vacuumdb.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
+vacuumdb: vacuumdb.o vacuuming.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
reindexdb: reindexdb.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
pg_isready: pg_isready.o common.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
@@ -50,7 +50,7 @@ uninstall:
clean distclean:
rm -f $(addsuffix $(X), $(PROGRAMS)) $(addsuffix .o, $(PROGRAMS))
- rm -f common.o $(WIN32RES)
+ rm -f common.o vacuuming.o $(WIN32RES)
rm -rf tmp_check
export with_icu
diff --git a/src/bin/scripts/meson.build b/src/bin/scripts/meson.build
index 80df7c33257..a4fed59d1c9 100644
--- a/src/bin/scripts/meson.build
+++ b/src/bin/scripts/meson.build
@@ -12,7 +12,6 @@ binaries = [
'createuser',
'dropuser',
'clusterdb',
- 'vacuumdb',
'reindexdb',
'pg_isready',
]
@@ -35,6 +34,33 @@ foreach binary : binaries
bin_targets += binary
endforeach
+vacuuming_common = static_library('libvacuuming_common',
+ files('common.c', 'vacuuming.c'),
+ dependencies: [frontend_code, libpq],
+ kwargs: internal_lib_args,
+)
+
+binaries = [
+ 'vacuumdb',
+]
+foreach binary : binaries
+ binary_sources = files('@0@.c'.format(binary))
+
+ if host_system == 'windows'
+ binary_sources += rc_bin_gen.process(win32ver_rc, extra_args: [
+ '--NAME', binary,
+ '--FILEDESC', '@0@ - PostgreSQL utility'.format(binary),])
+ endif
+
+ binary = executable(binary,
+ binary_sources,
+ link_with: [vacuuming_common],
+ dependencies: [frontend_code, libpq],
+ kwargs: default_bin_args,
+ )
+ bin_targets += binary
+endforeach
+
tests += {
'name': 'scripts',
'sd': meson.current_source_dir(),
diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c
index fd236087e90..b1be61ddf25 100644
--- a/src/bin/scripts/vacuumdb.c
+++ b/src/bin/scripts/vacuumdb.c
@@ -14,92 +14,13 @@
#include <limits.h>
-#include "catalog/pg_attribute_d.h"
-#include "catalog/pg_class_d.h"
#include "common.h"
-#include "common/connect.h"
#include "common/logging.h"
-#include "fe_utils/cancel.h"
#include "fe_utils/option_utils.h"
-#include "fe_utils/parallel_slot.h"
-#include "fe_utils/query_utils.h"
-#include "fe_utils/simple_list.h"
-#include "fe_utils/string_utils.h"
-
-
-/* vacuum options controlled by user flags */
-typedef struct vacuumingOptions
-{
- bool analyze_only;
- bool verbose;
- bool and_analyze;
- bool full;
- bool freeze;
- bool disable_page_skipping;
- bool skip_locked;
- int min_xid_age;
- int min_mxid_age;
- int parallel_workers; /* >= 0 indicates user specified the
- * parallel degree, otherwise -1 */
- bool no_index_cleanup;
- bool force_index_cleanup;
- bool do_truncate;
- bool process_main;
- bool process_toast;
- bool skip_database_stats;
- char *buffer_usage_limit;
- bool missing_stats_only;
-} vacuumingOptions;
-
-/* object filter options */
-typedef enum
-{
- OBJFILTER_NONE = 0, /* no filter used */
- OBJFILTER_ALL_DBS = (1 << 0), /* -a | --all */
- OBJFILTER_DATABASE = (1 << 1), /* -d | --dbname */
- OBJFILTER_TABLE = (1 << 2), /* -t | --table */
- OBJFILTER_SCHEMA = (1 << 3), /* -n | --schema */
- OBJFILTER_SCHEMA_EXCLUDE = (1 << 4), /* -N | --exclude-schema */
-} VacObjFilter;
-
-static VacObjFilter objfilter = OBJFILTER_NONE;
-
-static SimpleStringList *retrieve_objects(PGconn *conn,
- vacuumingOptions *vacopts,
- SimpleStringList *objects,
- bool echo);
-
-static void vacuum_one_database(ConnParams *cparams,
- vacuumingOptions *vacopts,
- int stage,
- SimpleStringList *objects,
- SimpleStringList **found_objs,
- int concurrentCons,
- const char *progname, bool echo, bool quiet);
-
-static void vacuum_all_databases(ConnParams *cparams,
- vacuumingOptions *vacopts,
- bool analyze_in_stages,
- SimpleStringList *objects,
- int concurrentCons,
- const char *progname, bool echo, bool quiet);
-
-static void prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
- vacuumingOptions *vacopts, const char *table);
-
-static void run_vacuum_command(PGconn *conn, const char *sql, bool echo,
- const char *table);
+#include "vacuuming.h"
static void help(const char *progname);
-
-void check_objfilter(void);
-
-static char *escape_quotes(const char *src);
-
-/* For analyze-in-stages mode */
-#define ANALYZE_NO_STAGE -1
-#define ANALYZE_NUM_STAGES 3
-
+static void check_objfilter(void);
int
main(int argc, char *argv[])
@@ -145,10 +66,6 @@ main(int argc, char *argv[])
int c;
const char *dbname = NULL;
const char *maintenance_db = NULL;
- char *host = NULL;
- char *port = NULL;
- char *username = NULL;
- enum trivalue prompt_password = TRI_DEFAULT;
ConnParams cparams;
bool echo = false;
bool quiet = false;
@@ -168,13 +85,18 @@ main(int argc, char *argv[])
vacopts.process_main = true;
vacopts.process_toast = true;
+ /* the same for connection parameters */
+ memset(&cparams, 0, sizeof(cparams));
+ cparams.prompt_password = TRI_DEFAULT;
+
pg_logging_init(argv[0]);
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
- handle_help_version_opts(argc, argv, "vacuumdb", help);
+ handle_help_version_opts(argc, argv, progname, help);
- while ((c = getopt_long(argc, argv, "ad:efFh:j:n:N:p:P:qt:U:vwWzZ", long_options, &optindex)) != -1)
+ while ((c = getopt_long(argc, argv, "ad:efFh:j:n:N:p:P:qt:U:vwWzZ",
+ long_options, &optindex)) != -1)
{
switch (c)
{
@@ -195,7 +117,7 @@ main(int argc, char *argv[])
vacopts.freeze = true;
break;
case 'h':
- host = pg_strdup(optarg);
+ cparams.pghost = pg_strdup(optarg);
break;
case 'j':
if (!option_parse_int(optarg, "-j/--jobs", 1, INT_MAX,
@@ -211,7 +133,7 @@ main(int argc, char *argv[])
simple_string_list_append(&objects, optarg);
break;
case 'p':
- port = pg_strdup(optarg);
+ cparams.pgport = pg_strdup(optarg);
break;
case 'P':
if (!option_parse_int(optarg, "-P/--parallel", 0, INT_MAX,
@@ -227,16 +149,16 @@ main(int argc, char *argv[])
tbl_count++;
break;
case 'U':
- username = pg_strdup(optarg);
+ cparams.pguser = pg_strdup(optarg);
break;
case 'v':
vacopts.verbose = true;
break;
case 'w':
- prompt_password = TRI_NO;
+ cparams.prompt_password = TRI_NO;
break;
case 'W':
- prompt_password = TRI_YES;
+ cparams.prompt_password = TRI_YES;
break;
case 'z':
vacopts.and_analyze = true;
@@ -380,66 +302,9 @@ main(int argc, char *argv[])
pg_fatal("cannot use the \"%s\" option without \"%s\" or \"%s\"",
"missing-stats-only", "analyze-only", "analyze-in-stages");
- /* fill cparams except for dbname, which is set below */
- cparams.pghost = host;
- cparams.pgport = port;
- cparams.pguser = username;
- cparams.prompt_password = prompt_password;
- cparams.override_dbname = NULL;
-
- setup_cancel_handler(NULL);
-
- /* Avoid opening extra connections. */
- if (tbl_count && (concurrentCons > tbl_count))
- concurrentCons = tbl_count;
-
- if (objfilter & OBJFILTER_ALL_DBS)
- {
- cparams.dbname = maintenance_db;
-
- vacuum_all_databases(&cparams, &vacopts,
- analyze_in_stages,
- &objects,
- concurrentCons,
- progname, echo, quiet);
- }
- else
- {
- if (dbname == NULL)
- {
- if (getenv("PGDATABASE"))
- dbname = getenv("PGDATABASE");
- else if (getenv("PGUSER"))
- dbname = getenv("PGUSER");
- else
- dbname = get_user_name_or_exit(progname);
- }
-
- cparams.dbname = dbname;
-
- if (analyze_in_stages)
- {
- int stage;
- SimpleStringList *found_objs = NULL;
-
- for (stage = 0; stage < ANALYZE_NUM_STAGES; stage++)
- {
- vacuum_one_database(&cparams, &vacopts,
- stage,
- &objects,
- vacopts.missing_stats_only ? &found_objs : NULL,
- concurrentCons,
- progname, echo, quiet);
- }
- }
- else
- vacuum_one_database(&cparams, &vacopts,
- ANALYZE_NO_STAGE,
- &objects, NULL,
- concurrentCons,
- progname, echo, quiet);
- }
-
+ vacuuming_main(&cparams, dbname, maintenance_db, &vacopts, &objects,
+ analyze_in_stages, tbl_count, concurrentCons,
+ progname, echo, quiet);
exit(0);
}
@@ -466,885 +331,6 @@ check_objfilter(void)
pg_fatal("cannot vacuum all tables in schema(s) and exclude schema(s) at the same time");
}
-/*
- * Returns a newly malloc'd version of 'src' with escaped single quotes and
- * backslashes.
- */
-static char *
-escape_quotes(const char *src)
-{
- char *result = escape_single_quotes_ascii(src);
-
- if (!result)
- pg_fatal("out of memory");
- return result;
-}
-
-/*
- * vacuum_one_database
- *
- * Process tables in the given database.
- *
- * There are two ways to specify the list of objects to process:
- *
- * 1) The "found_objs" parameter is a double pointer to a fully qualified list
- * of objects to process, as returned by a previous call to
- * vacuum_one_database().
- *
- * a) If both "found_objs" (the double pointer) and "*found_objs" (the
- * once-dereferenced double pointer) are not NULL, this list takes
- * priority, and anything specified in "objects" is ignored.
- *
- * b) If "found_objs" (the double pointer) is not NULL but "*found_objs"
- * (the once-dereferenced double pointer) _is_ NULL, the "objects"
- * parameter takes priority, and the results of the catalog query
- * described in (2) are stored in "found_objs".
- *
- * c) If "found_objs" (the double pointer) is NULL, the "objects"
- * parameter again takes priority, and the results of the catalog query
- * are not saved.
- *
- * 2) The "objects" parameter is a user-specified list of objects to process.
- * When (1b) or (1c) applies, this function performs a catalog query to
- * retrieve a fully qualified list of objects to process, as described
- * below.
- *
- * a) If "objects" is not NULL, the catalog query gathers only the objects
- * listed in "objects".
- *
- * b) If "objects" is NULL, all tables in the database are gathered.
- *
- * Note that this function is only concerned with running exactly one stage
- * when in analyze-in-stages mode; caller must iterate on us if necessary.
- *
- * If concurrentCons is > 1, multiple connections are used to vacuum tables
- * in parallel.
- */
-static void
-vacuum_one_database(ConnParams *cparams,
- vacuumingOptions *vacopts,
- int stage,
- SimpleStringList *objects,
- SimpleStringList **found_objs,
- int concurrentCons,
- const char *progname, bool echo, bool quiet)
-{
- PQExpBufferData sql;
- PGconn *conn;
- SimpleStringListCell *cell;
- ParallelSlotArray *sa;
- int ntups = 0;
- bool failed = false;
- const char *initcmd;
- SimpleStringList *ret = NULL;
- const char *stage_commands[] = {
- "SET default_statistics_target=1; SET vacuum_cost_delay=0;",
- "SET default_statistics_target=10; RESET vacuum_cost_delay;",
- "RESET default_statistics_target;"
- };
- const char *stage_messages[] = {
- gettext_noop("Generating minimal optimizer statistics (1 target)"),
- gettext_noop("Generating medium optimizer statistics (10 targets)"),
- gettext_noop("Generating default (full) optimizer statistics")
- };
-
- Assert(stage == ANALYZE_NO_STAGE ||
- (stage >= 0 && stage < ANALYZE_NUM_STAGES));
-
- conn = connectDatabase(cparams, progname, echo, false, true);
-
- if (vacopts->disable_page_skipping && PQserverVersion(conn) < 90600)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "disable-page-skipping", "9.6");
- }
-
- if (vacopts->no_index_cleanup && PQserverVersion(conn) < 120000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "no-index-cleanup", "12");
- }
-
- if (vacopts->force_index_cleanup && PQserverVersion(conn) < 120000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "force-index-cleanup", "12");
- }
-
- if (!vacopts->do_truncate && PQserverVersion(conn) < 120000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "no-truncate", "12");
- }
-
- if (!vacopts->process_main && PQserverVersion(conn) < 160000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "no-process-main", "16");
- }
-
- if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "no-process-toast", "14");
- }
-
- if (vacopts->skip_locked && PQserverVersion(conn) < 120000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "skip-locked", "12");
- }
-
- if (vacopts->min_xid_age != 0 && PQserverVersion(conn) < 90600)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "--min-xid-age", "9.6");
- }
-
- if (vacopts->min_mxid_age != 0 && PQserverVersion(conn) < 90600)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "--min-mxid-age", "9.6");
- }
-
- if (vacopts->parallel_workers >= 0 && PQserverVersion(conn) < 130000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "--parallel", "13");
- }
-
- if (vacopts->buffer_usage_limit && PQserverVersion(conn) < 160000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "--buffer-usage-limit", "16");
- }
-
- if (vacopts->missing_stats_only && PQserverVersion(conn) < 150000)
- {
- PQfinish(conn);
- pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
- "--missing-stats-only", "15");
- }
-
- /* skip_database_stats is used automatically if server supports it */
- vacopts->skip_database_stats = (PQserverVersion(conn) >= 160000);
-
- if (!quiet)
- {
- if (stage != ANALYZE_NO_STAGE)
- printf(_("%s: processing database \"%s\": %s\n"),
- progname, PQdb(conn), _(stage_messages[stage]));
- else
- printf(_("%s: vacuuming database \"%s\"\n"),
- progname, PQdb(conn));
- fflush(stdout);
- }
-
- /*
- * If the caller provided the results of a previous catalog query, just
- * use that. Otherwise, run the catalog query ourselves and set the
- * return variable if provided.
- */
- if (found_objs && *found_objs)
- ret = *found_objs;
- else
- {
- ret = retrieve_objects(conn, vacopts, objects, echo);
- if (found_objs)
- *found_objs = ret;
- }
-
- /*
- * Count the number of objects in the catalog query result. If there are
- * none, we are done.
- */
- for (cell = ret ? ret->head : NULL; cell; cell = cell->next)
- ntups++;
-
- if (ntups == 0)
- {
- PQfinish(conn);
- return;
- }
-
- /*
- * Ensure concurrentCons is sane. If there are more connections than
- * vacuumable relations, we don't need to use them all.
- */
- if (concurrentCons > ntups)
- concurrentCons = ntups;
- if (concurrentCons <= 0)
- concurrentCons = 1;
-
- /*
- * All slots need to be prepared to run the appropriate analyze stage, if
- * caller requested that mode. We have to prepare the initial connection
- * ourselves before setting up the slots.
- */
- if (stage == ANALYZE_NO_STAGE)
- initcmd = NULL;
- else
- {
- initcmd = stage_commands[stage];
- executeCommand(conn, initcmd, echo);
- }
-
- /*
- * Setup the database connections. We reuse the connection we already have
- * for the first slot. If not in parallel mode, the first slot in the
- * array contains the connection.
- */
- sa = ParallelSlotsSetup(concurrentCons, cparams, progname, echo, initcmd);
- ParallelSlotsAdoptConn(sa, conn);
-
- initPQExpBuffer(&sql);
-
- cell = ret->head;
- do
- {
- const char *tabname = cell->val;
- ParallelSlot *free_slot;
-
- if (CancelRequested)
- {
- failed = true;
- goto finish;
- }
-
- free_slot = ParallelSlotsGetIdle(sa, NULL);
- if (!free_slot)
- {
- failed = true;
- goto finish;
- }
-
- prepare_vacuum_command(&sql, PQserverVersion(free_slot->connection),
- vacopts, tabname);
-
- /*
- * Execute the vacuum. All errors are handled in processQueryResult
- * through ParallelSlotsGetIdle.
- */
- ParallelSlotSetHandler(free_slot, TableCommandResultHandler, NULL);
- run_vacuum_command(free_slot->connection, sql.data,
- echo, tabname);
-
- cell = cell->next;
- } while (cell != NULL);
-
- if (!ParallelSlotsWaitCompletion(sa))
- {
- failed = true;
- goto finish;
- }
-
- /* If we used SKIP_DATABASE_STATS, mop up with ONLY_DATABASE_STATS */
- if (vacopts->skip_database_stats && stage == ANALYZE_NO_STAGE)
- {
- const char *cmd = "VACUUM (ONLY_DATABASE_STATS);";
- ParallelSlot *free_slot = ParallelSlotsGetIdle(sa, NULL);
-
- if (!free_slot)
- {
- failed = true;
- goto finish;
- }
-
- ParallelSlotSetHandler(free_slot, TableCommandResultHandler, NULL);
- run_vacuum_command(free_slot->connection, cmd, echo, NULL);
-
- if (!ParallelSlotsWaitCompletion(sa))
- failed = true;
- }
-
-finish:
- ParallelSlotsTerminate(sa);
- pg_free(sa);
-
- termPQExpBuffer(&sql);
-
- if (failed)
- exit(1);
-}
-
-/*
- * Prepare the list of tables to process by querying the catalogs.
- *
- * Since we execute the constructed query with the default search_path (which
- * could be unsafe), everything in this query MUST be fully qualified.
- *
- * First, build a WITH clause for the catalog query if any tables were
- * specified, with a set of values made of relation names and their optional
- * set of columns. This is used to match any provided column lists with the
- * generated qualified identifiers and to filter for the tables provided via
- * --table. If a listed table does not exist, the catalog query will fail.
- */
-static SimpleStringList *
-retrieve_objects(PGconn *conn, vacuumingOptions *vacopts,
- SimpleStringList *objects, bool echo)
-{
- PQExpBufferData buf;
- PQExpBufferData catalog_query;
- PGresult *res;
- SimpleStringListCell *cell;
- SimpleStringList *found_objs = palloc0(sizeof(SimpleStringList));
- bool objects_listed = false;
-
- initPQExpBuffer(&catalog_query);
- for (cell = objects ? objects->head : NULL; cell; cell = cell->next)
- {
- char *just_table = NULL;
- const char *just_columns = NULL;
-
- if (!objects_listed)
- {
- appendPQExpBufferStr(&catalog_query,
- "WITH listed_objects (object_oid, column_list) "
- "AS (\n VALUES (");
- objects_listed = true;
- }
- else
- appendPQExpBufferStr(&catalog_query, ",\n (");
-
- if (objfilter & (OBJFILTER_SCHEMA | OBJFILTER_SCHEMA_EXCLUDE))
- {
- appendStringLiteralConn(&catalog_query, cell->val, conn);
- appendPQExpBufferStr(&catalog_query, "::pg_catalog.regnamespace, ");
- }
-
- if (objfilter & OBJFILTER_TABLE)
- {
- /*
- * Split relation and column names given by the user, this is used
- * to feed the CTE with values on which are performed pre-run
- * validity checks as well. For now these happen only on the
- * relation name.
- */
- splitTableColumnsSpec(cell->val, PQclientEncoding(conn),
- &just_table, &just_columns);
-
- appendStringLiteralConn(&catalog_query, just_table, conn);
- appendPQExpBufferStr(&catalog_query, "::pg_catalog.regclass, ");
- }
-
- if (just_columns && just_columns[0] != '\0')
- appendStringLiteralConn(&catalog_query, just_columns, conn);
- else
- appendPQExpBufferStr(&catalog_query, "NULL");
-
- appendPQExpBufferStr(&catalog_query, "::pg_catalog.text)");
-
- pg_free(just_table);
- }
-
- /* Finish formatting the CTE */
- if (objects_listed)
- appendPQExpBufferStr(&catalog_query, "\n)\n");
-
- appendPQExpBufferStr(&catalog_query, "SELECT c.relname, ns.nspname");
-
- if (objects_listed)
- appendPQExpBufferStr(&catalog_query, ", listed_objects.column_list");
-
- appendPQExpBufferStr(&catalog_query,
- " FROM pg_catalog.pg_class c\n"
- " JOIN pg_catalog.pg_namespace ns"
- " ON c.relnamespace OPERATOR(pg_catalog.=) ns.oid\n"
- " CROSS JOIN LATERAL (SELECT c.relkind IN ("
- CppAsString2(RELKIND_PARTITIONED_TABLE) ", "
- CppAsString2(RELKIND_PARTITIONED_INDEX) ")) as p (inherited)\n"
- " LEFT JOIN pg_catalog.pg_class t"
- " ON c.reltoastrelid OPERATOR(pg_catalog.=) t.oid\n");
-
- /*
- * Used to match the tables or schemas listed by the user, completing the
- * JOIN clause.
- */
- if (objects_listed)
- {
- appendPQExpBufferStr(&catalog_query, " LEFT JOIN listed_objects"
- " ON listed_objects.object_oid"
- " OPERATOR(pg_catalog.=) ");
-
- if (objfilter & OBJFILTER_TABLE)
- appendPQExpBufferStr(&catalog_query, "c.oid\n");
- else
- appendPQExpBufferStr(&catalog_query, "ns.oid\n");
- }
-
- /*
- * Exclude temporary tables, beginning the WHERE clause.
- */
- appendPQExpBufferStr(&catalog_query,
- " WHERE c.relpersistence OPERATOR(pg_catalog.!=) "
- CppAsString2(RELPERSISTENCE_TEMP) "\n");
-
- /*
- * Used to match the tables or schemas listed by the user, for the WHERE
- * clause.
- */
- if (objects_listed)
- {
- if (objfilter & OBJFILTER_SCHEMA_EXCLUDE)
- appendPQExpBufferStr(&catalog_query,
- " AND listed_objects.object_oid IS NULL\n");
- else
- appendPQExpBufferStr(&catalog_query,
- " AND listed_objects.object_oid IS NOT NULL\n");
- }
-
- /*
- * If no tables were listed, filter for the relevant relation types. If
- * tables were given via --table, don't bother filtering by relation type.
- * Instead, let the server decide whether a given relation can be
- * processed in which case the user will know about it.
- */
- if ((objfilter & OBJFILTER_TABLE) == 0)
- {
- /*
- * vacuumdb should generally follow the behavior of the underlying
- * VACUUM and ANALYZE commands. If analyze_only is true, process
- * regular tables, materialized views, and partitioned tables, just
- * like ANALYZE (with no specific target tables) does. Otherwise,
- * process only regular tables and materialized views, since VACUUM
- * skips partitioned tables when no target tables are specified.
- */
- if (vacopts->analyze_only)
- appendPQExpBufferStr(&catalog_query,
- " AND c.relkind OPERATOR(pg_catalog.=) ANY (array["
- CppAsString2(RELKIND_RELATION) ", "
- CppAsString2(RELKIND_MATVIEW) ", "
- CppAsString2(RELKIND_PARTITIONED_TABLE) "])\n");
- else
- appendPQExpBufferStr(&catalog_query,
- " AND c.relkind OPERATOR(pg_catalog.=) ANY (array["
- CppAsString2(RELKIND_RELATION) ", "
- CppAsString2(RELKIND_MATVIEW) "])\n");
-
- }
-
- /*
- * For --min-xid-age and --min-mxid-age, the age of the relation is the
- * greatest of the ages of the main relation and its associated TOAST
- * table. The commands generated by vacuumdb will also process the TOAST
- * table for the relation if necessary, so it does not need to be
- * considered separately.
- */
- if (vacopts->min_xid_age != 0)
- {
- appendPQExpBuffer(&catalog_query,
- " AND GREATEST(pg_catalog.age(c.relfrozenxid),"
- " pg_catalog.age(t.relfrozenxid)) "
- " OPERATOR(pg_catalog.>=) '%d'::pg_catalog.int4\n"
- " AND c.relfrozenxid OPERATOR(pg_catalog.!=)"
- " '0'::pg_catalog.xid\n",
- vacopts->min_xid_age);
- }
-
- if (vacopts->min_mxid_age != 0)
- {
- appendPQExpBuffer(&catalog_query,
- " AND GREATEST(pg_catalog.mxid_age(c.relminmxid),"
- " pg_catalog.mxid_age(t.relminmxid)) OPERATOR(pg_catalog.>=)"
- " '%d'::pg_catalog.int4\n"
- " AND c.relminmxid OPERATOR(pg_catalog.!=)"
- " '0'::pg_catalog.xid\n",
- vacopts->min_mxid_age);
- }
-
- if (vacopts->missing_stats_only)
- {
- appendPQExpBufferStr(&catalog_query, " AND (\n");
-
- /* regular stats */
- appendPQExpBufferStr(&catalog_query,
- " EXISTS (SELECT NULL FROM pg_catalog.pg_attribute a\n"
- " WHERE a.attrelid OPERATOR(pg_catalog.=) c.oid\n"
- " AND a.attnum OPERATOR(pg_catalog.>) 0::pg_catalog.int2\n"
- " AND NOT a.attisdropped\n"
- " AND a.attstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
- " AND a.attgenerated OPERATOR(pg_catalog.<>) "
- CppAsString2(ATTRIBUTE_GENERATED_VIRTUAL) "\n"
- " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic s\n"
- " WHERE s.starelid OPERATOR(pg_catalog.=) a.attrelid\n"
- " AND s.staattnum OPERATOR(pg_catalog.=) a.attnum\n"
- " AND s.stainherit OPERATOR(pg_catalog.=) p.inherited))\n");
-
- /* extended stats */
- appendPQExpBufferStr(&catalog_query,
- " OR EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext e\n"
- " WHERE e.stxrelid OPERATOR(pg_catalog.=) c.oid\n"
- " AND e.stxstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
- " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext_data d\n"
- " WHERE d.stxoid OPERATOR(pg_catalog.=) e.oid\n"
- " AND d.stxdinherit OPERATOR(pg_catalog.=) p.inherited))\n");
-
- /* expression indexes */
- appendPQExpBufferStr(&catalog_query,
- " OR EXISTS (SELECT NULL FROM pg_catalog.pg_attribute a\n"
- " JOIN pg_catalog.pg_index i"
- " ON i.indexrelid OPERATOR(pg_catalog.=) a.attrelid\n"
- " WHERE i.indrelid OPERATOR(pg_catalog.=) c.oid\n"
- " AND i.indkey[a.attnum OPERATOR(pg_catalog.-) 1::pg_catalog.int2]"
- " OPERATOR(pg_catalog.=) 0::pg_catalog.int2\n"
- " AND a.attnum OPERATOR(pg_catalog.>) 0::pg_catalog.int2\n"
- " AND NOT a.attisdropped\n"
- " AND a.attstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
- " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic s\n"
- " WHERE s.starelid OPERATOR(pg_catalog.=) a.attrelid\n"
- " AND s.staattnum OPERATOR(pg_catalog.=) a.attnum\n"
- " AND s.stainherit OPERATOR(pg_catalog.=) p.inherited))\n");
-
- /* inheritance and regular stats */
- appendPQExpBufferStr(&catalog_query,
- " OR EXISTS (SELECT NULL FROM pg_catalog.pg_attribute a\n"
- " WHERE a.attrelid OPERATOR(pg_catalog.=) c.oid\n"
- " AND a.attnum OPERATOR(pg_catalog.>) 0::pg_catalog.int2\n"
- " AND NOT a.attisdropped\n"
- " AND a.attstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
- " AND a.attgenerated OPERATOR(pg_catalog.<>) "
- CppAsString2(ATTRIBUTE_GENERATED_VIRTUAL) "\n"
- " AND c.relhassubclass\n"
- " AND NOT p.inherited\n"
- " AND EXISTS (SELECT NULL FROM pg_catalog.pg_inherits h\n"
- " WHERE h.inhparent OPERATOR(pg_catalog.=) c.oid)\n"
- " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic s\n"
- " WHERE s.starelid OPERATOR(pg_catalog.=) a.attrelid\n"
- " AND s.staattnum OPERATOR(pg_catalog.=) a.attnum\n"
- " AND s.stainherit))\n");
-
- /* inheritance and extended stats */
- appendPQExpBufferStr(&catalog_query,
- " OR EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext e\n"
- " WHERE e.stxrelid OPERATOR(pg_catalog.=) c.oid\n"
- " AND e.stxstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
- " AND c.relhassubclass\n"
- " AND NOT p.inherited\n"
- " AND EXISTS (SELECT NULL FROM pg_catalog.pg_inherits h\n"
- " WHERE h.inhparent OPERATOR(pg_catalog.=) c.oid)\n"
- " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext_data d\n"
- " WHERE d.stxoid OPERATOR(pg_catalog.=) e.oid\n"
- " AND d.stxdinherit))\n");
-
- appendPQExpBufferStr(&catalog_query, " )\n");
- }
-
- /*
- * Execute the catalog query. We use the default search_path for this
- * query for consistency with table lookups done elsewhere by the user.
- */
- appendPQExpBufferStr(&catalog_query, " ORDER BY c.relpages DESC;");
- executeCommand(conn, "RESET search_path;", echo);
- res = executeQuery(conn, catalog_query.data, echo);
- termPQExpBuffer(&catalog_query);
- PQclear(executeQuery(conn, ALWAYS_SECURE_SEARCH_PATH_SQL, echo));
-
- /*
- * Build qualified identifiers for each table, including the column list
- * if given.
- */
- initPQExpBuffer(&buf);
- for (int i = 0; i < PQntuples(res); i++)
- {
- appendPQExpBufferStr(&buf,
- fmtQualifiedIdEnc(PQgetvalue(res, i, 1),
- PQgetvalue(res, i, 0),
- PQclientEncoding(conn)));
-
- if (objects_listed && !PQgetisnull(res, i, 2))
- appendPQExpBufferStr(&buf, PQgetvalue(res, i, 2));
-
- simple_string_list_append(found_objs, buf.data);
- resetPQExpBuffer(&buf);
- }
- termPQExpBuffer(&buf);
- PQclear(res);
-
- return found_objs;
-}
-
-/*
- * Vacuum/analyze all connectable databases.
- *
- * In analyze-in-stages mode, we process all databases in one stage before
- * moving on to the next stage. That ensure minimal stats are available
- * quickly everywhere before generating more detailed ones.
- */
-static void
-vacuum_all_databases(ConnParams *cparams,
- vacuumingOptions *vacopts,
- bool analyze_in_stages,
- SimpleStringList *objects,
- int concurrentCons,
- const char *progname, bool echo, bool quiet)
-{
- PGconn *conn;
- PGresult *result;
- int stage;
- int i;
-
- conn = connectMaintenanceDatabase(cparams, progname, echo);
- result = executeQuery(conn,
- "SELECT datname FROM pg_database WHERE datallowconn AND datconnlimit <> -2 ORDER BY 1;",
- echo);
- PQfinish(conn);
-
- if (analyze_in_stages)
- {
- SimpleStringList **found_objs = NULL;
-
- if (vacopts->missing_stats_only)
- found_objs = palloc0(PQntuples(result) * sizeof(SimpleStringList *));
-
- /*
- * When analyzing all databases in stages, we analyze them all in the
- * fastest stage first, so that initial statistics become available
- * for all of them as soon as possible.
- *
- * This means we establish several times as many connections, but
- * that's a secondary consideration.
- */
- for (stage = 0; stage < ANALYZE_NUM_STAGES; stage++)
- {
- for (i = 0; i < PQntuples(result); i++)
- {
- cparams->override_dbname = PQgetvalue(result, i, 0);
-
- vacuum_one_database(cparams, vacopts,
- stage,
- objects,
- vacopts->missing_stats_only ? &found_objs[i] : NULL,
- concurrentCons,
- progname, echo, quiet);
- }
- }
- }
- else
- {
- for (i = 0; i < PQntuples(result); i++)
- {
- cparams->override_dbname = PQgetvalue(result, i, 0);
-
- vacuum_one_database(cparams, vacopts,
- ANALYZE_NO_STAGE,
- objects, NULL,
- concurrentCons,
- progname, echo, quiet);
- }
- }
-
- PQclear(result);
-}
-
-/*
- * Construct a vacuum/analyze command to run based on the given options, in the
- * given string buffer, which may contain previous garbage.
- *
- * The table name used must be already properly quoted. The command generated
- * depends on the server version involved and it is semicolon-terminated.
- */
-static void
-prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
- vacuumingOptions *vacopts, const char *table)
-{
- const char *paren = " (";
- const char *comma = ", ";
- const char *sep = paren;
-
- resetPQExpBuffer(sql);
-
- if (vacopts->analyze_only)
- {
- appendPQExpBufferStr(sql, "ANALYZE");
-
- /* parenthesized grammar of ANALYZE is supported since v11 */
- if (serverVersion >= 110000)
- {
- if (vacopts->skip_locked)
- {
- /* SKIP_LOCKED is supported since v12 */
- Assert(serverVersion >= 120000);
- appendPQExpBuffer(sql, "%sSKIP_LOCKED", sep);
- sep = comma;
- }
- if (vacopts->verbose)
- {
- appendPQExpBuffer(sql, "%sVERBOSE", sep);
- sep = comma;
- }
- if (vacopts->buffer_usage_limit)
- {
- Assert(serverVersion >= 160000);
- appendPQExpBuffer(sql, "%sBUFFER_USAGE_LIMIT '%s'", sep,
- vacopts->buffer_usage_limit);
- sep = comma;
- }
- if (sep != paren)
- appendPQExpBufferChar(sql, ')');
- }
- else
- {
- if (vacopts->verbose)
- appendPQExpBufferStr(sql, " VERBOSE");
- }
- }
- else
- {
- appendPQExpBufferStr(sql, "VACUUM");
-
- /* parenthesized grammar of VACUUM is supported since v9.0 */
- if (serverVersion >= 90000)
- {
- if (vacopts->disable_page_skipping)
- {
- /* DISABLE_PAGE_SKIPPING is supported since v9.6 */
- Assert(serverVersion >= 90600);
- appendPQExpBuffer(sql, "%sDISABLE_PAGE_SKIPPING", sep);
- sep = comma;
- }
- if (vacopts->no_index_cleanup)
- {
- /* "INDEX_CLEANUP FALSE" has been supported since v12 */
- Assert(serverVersion >= 120000);
- Assert(!vacopts->force_index_cleanup);
- appendPQExpBuffer(sql, "%sINDEX_CLEANUP FALSE", sep);
- sep = comma;
- }
- if (vacopts->force_index_cleanup)
- {
- /* "INDEX_CLEANUP TRUE" has been supported since v12 */
- Assert(serverVersion >= 120000);
- Assert(!vacopts->no_index_cleanup);
- appendPQExpBuffer(sql, "%sINDEX_CLEANUP TRUE", sep);
- sep = comma;
- }
- if (!vacopts->do_truncate)
- {
- /* TRUNCATE is supported since v12 */
- Assert(serverVersion >= 120000);
- appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
- sep = comma;
- }
- if (!vacopts->process_main)
- {
- /* PROCESS_MAIN is supported since v16 */
- Assert(serverVersion >= 160000);
- appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
- sep = comma;
- }
- if (!vacopts->process_toast)
- {
- /* PROCESS_TOAST is supported since v14 */
- Assert(serverVersion >= 140000);
- appendPQExpBuffer(sql, "%sPROCESS_TOAST FALSE", sep);
- sep = comma;
- }
- if (vacopts->skip_database_stats)
- {
- /* SKIP_DATABASE_STATS is supported since v16 */
- Assert(serverVersion >= 160000);
- appendPQExpBuffer(sql, "%sSKIP_DATABASE_STATS", sep);
- sep = comma;
- }
- if (vacopts->skip_locked)
- {
- /* SKIP_LOCKED is supported since v12 */
- Assert(serverVersion >= 120000);
- appendPQExpBuffer(sql, "%sSKIP_LOCKED", sep);
- sep = comma;
- }
- if (vacopts->full)
- {
- appendPQExpBuffer(sql, "%sFULL", sep);
- sep = comma;
- }
- if (vacopts->freeze)
- {
- appendPQExpBuffer(sql, "%sFREEZE", sep);
- sep = comma;
- }
- if (vacopts->verbose)
- {
- appendPQExpBuffer(sql, "%sVERBOSE", sep);
- sep = comma;
- }
- if (vacopts->and_analyze)
- {
- appendPQExpBuffer(sql, "%sANALYZE", sep);
- sep = comma;
- }
- if (vacopts->parallel_workers >= 0)
- {
- /* PARALLEL is supported since v13 */
- Assert(serverVersion >= 130000);
- appendPQExpBuffer(sql, "%sPARALLEL %d", sep,
- vacopts->parallel_workers);
- sep = comma;
- }
- if (vacopts->buffer_usage_limit)
- {
- Assert(serverVersion >= 160000);
- appendPQExpBuffer(sql, "%sBUFFER_USAGE_LIMIT '%s'", sep,
- vacopts->buffer_usage_limit);
- sep = comma;
- }
- if (sep != paren)
- appendPQExpBufferChar(sql, ')');
- }
- else
- {
- if (vacopts->full)
- appendPQExpBufferStr(sql, " FULL");
- if (vacopts->freeze)
- appendPQExpBufferStr(sql, " FREEZE");
- if (vacopts->verbose)
- appendPQExpBufferStr(sql, " VERBOSE");
- if (vacopts->and_analyze)
- appendPQExpBufferStr(sql, " ANALYZE");
- }
- }
-
- appendPQExpBuffer(sql, " %s;", table);
-}
-
-/*
- * Send a vacuum/analyze command to the server, returning after sending the
- * command.
- *
- * Any errors during command execution are reported to stderr.
- */
-static void
-run_vacuum_command(PGconn *conn, const char *sql, bool echo,
- const char *table)
-{
- bool status;
-
- if (echo)
- printf("%s\n", sql);
-
- status = PQsendQuery(conn, sql) == 1;
-
- if (!status)
- {
- if (table)
- pg_log_error("vacuuming of table \"%s\" in database \"%s\" failed: %s",
- table, PQdb(conn), PQerrorMessage(conn));
- else
- pg_log_error("vacuuming of database \"%s\" failed: %s",
- PQdb(conn), PQerrorMessage(conn));
- }
-}
static void
help(const char *progname)
diff --git a/src/bin/scripts/vacuuming.c b/src/bin/scripts/vacuuming.c
new file mode 100644
index 00000000000..9be37fcc45a
--- /dev/null
+++ b/src/bin/scripts/vacuuming.c
@@ -0,0 +1,978 @@
+/*-------------------------------------------------------------------------
+ * vacuuming.c
+ * Common routines for vacuumdb
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/scripts/vacuuming.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <limits.h>
+
+#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_class_d.h"
+#include "common/connect.h"
+#include "common/logging.h"
+#include "fe_utils/cancel.h"
+#include "fe_utils/option_utils.h"
+#include "fe_utils/parallel_slot.h"
+#include "fe_utils/query_utils.h"
+#include "fe_utils/string_utils.h"
+#include "vacuuming.h"
+
+VacObjFilter objfilter = OBJFILTER_NONE;
+
+
+/*
+ * Executes vacuum/analyze as indicated, or dies in case of failure.
+ */
+void
+vacuuming_main(ConnParams *cparams, const char *dbname,
+ const char *maintenance_db, vacuumingOptions *vacopts,
+ SimpleStringList *objects, bool analyze_in_stages,
+ int tbl_count, int concurrentCons,
+ const char *progname, bool echo, bool quiet)
+{
+ setup_cancel_handler(NULL);
+
+ /* Avoid opening extra connections. */
+ if (tbl_count && (concurrentCons > tbl_count))
+ concurrentCons = tbl_count;
+
+ if (objfilter & OBJFILTER_ALL_DBS)
+ {
+ cparams->dbname = maintenance_db;
+
+ vacuum_all_databases(cparams, vacopts,
+ analyze_in_stages,
+ objects,
+ concurrentCons,
+ progname, echo, quiet);
+ }
+ else
+ {
+ if (dbname == NULL)
+ {
+ if (getenv("PGDATABASE"))
+ dbname = getenv("PGDATABASE");
+ else if (getenv("PGUSER"))
+ dbname = getenv("PGUSER");
+ else
+ dbname = get_user_name_or_exit(progname);
+ }
+
+ cparams->dbname = dbname;
+
+ if (analyze_in_stages)
+ {
+ int stage;
+ SimpleStringList *found_objs = NULL;
+
+ for (stage = 0; stage < ANALYZE_NUM_STAGES; stage++)
+ {
+ vacuum_one_database(cparams, vacopts,
+ stage,
+ objects,
+ vacopts->missing_stats_only ? &found_objs : NULL,
+ concurrentCons,
+ progname, echo, quiet);
+ }
+ }
+ else
+ vacuum_one_database(cparams, vacopts,
+ ANALYZE_NO_STAGE,
+ objects, NULL,
+ concurrentCons,
+ progname, echo, quiet);
+ }
+}
+
+
+/*
+ * vacuum_one_database
+ *
+ * Process tables in the given database.
+ *
+ * There are two ways to specify the list of objects to process:
+ *
+ * 1) The "found_objs" parameter is a double pointer to a fully qualified list
+ * of objects to process, as returned by a previous call to
+ * vacuum_one_database().
+ *
+ * a) If both "found_objs" (the double pointer) and "*found_objs" (the
+ * once-dereferenced double pointer) are not NULL, this list takes
+ * priority, and anything specified in "objects" is ignored.
+ *
+ * b) If "found_objs" (the double pointer) is not NULL but "*found_objs"
+ * (the once-dereferenced double pointer) _is_ NULL, the "objects"
+ * parameter takes priority, and the results of the catalog query
+ * described in (2) are stored in "found_objs".
+ *
+ * c) If "found_objs" (the double pointer) is NULL, the "objects"
+ * parameter again takes priority, and the results of the catalog query
+ * are not saved.
+ *
+ * 2) The "objects" parameter is a user-specified list of objects to process.
+ * When (1b) or (1c) applies, this function performs a catalog query to
+ * retrieve a fully qualified list of objects to process, as described
+ * below.
+ *
+ * a) If "objects" is not NULL, the catalog query gathers only the objects
+ * listed in "objects".
+ *
+ * b) If "objects" is NULL, all tables in the database are gathered.
+ *
+ * Note that this function is only concerned with running exactly one stage
+ * when in analyze-in-stages mode; caller must iterate on us if necessary.
+ *
+ * If concurrentCons is > 1, multiple connections are used to vacuum tables
+ * in parallel.
+ */
+void
+vacuum_one_database(ConnParams *cparams,
+ vacuumingOptions *vacopts,
+ int stage,
+ SimpleStringList *objects,
+ SimpleStringList **found_objs,
+ int concurrentCons,
+ const char *progname, bool echo, bool quiet)
+{
+ PQExpBufferData sql;
+ PGconn *conn;
+ SimpleStringListCell *cell;
+ ParallelSlotArray *sa;
+ int ntups = 0;
+ bool failed = false;
+ const char *initcmd;
+ SimpleStringList *ret = NULL;
+ const char *stage_commands[] = {
+ "SET default_statistics_target=1; SET vacuum_cost_delay=0;",
+ "SET default_statistics_target=10; RESET vacuum_cost_delay;",
+ "RESET default_statistics_target;"
+ };
+ const char *stage_messages[] = {
+ gettext_noop("Generating minimal optimizer statistics (1 target)"),
+ gettext_noop("Generating medium optimizer statistics (10 targets)"),
+ gettext_noop("Generating default (full) optimizer statistics")
+ };
+
+ Assert(stage == ANALYZE_NO_STAGE ||
+ (stage >= 0 && stage < ANALYZE_NUM_STAGES));
+
+ conn = connectDatabase(cparams, progname, echo, false, true);
+
+ if (vacopts->disable_page_skipping && PQserverVersion(conn) < 90600)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "disable-page-skipping", "9.6");
+ }
+
+ if (vacopts->no_index_cleanup && PQserverVersion(conn) < 120000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-index-cleanup", "12");
+ }
+
+ if (vacopts->force_index_cleanup && PQserverVersion(conn) < 120000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "force-index-cleanup", "12");
+ }
+
+ if (!vacopts->do_truncate && PQserverVersion(conn) < 120000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-truncate", "12");
+ }
+
+ if (!vacopts->process_main && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-main", "16");
+ }
+
+ if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-toast", "14");
+ }
+
+ if (vacopts->skip_locked && PQserverVersion(conn) < 120000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "skip-locked", "12");
+ }
+
+ if (vacopts->min_xid_age != 0 && PQserverVersion(conn) < 90600)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "--min-xid-age", "9.6");
+ }
+
+ if (vacopts->min_mxid_age != 0 && PQserverVersion(conn) < 90600)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "--min-mxid-age", "9.6");
+ }
+
+ if (vacopts->parallel_workers >= 0 && PQserverVersion(conn) < 130000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "--parallel", "13");
+ }
+
+ if (vacopts->buffer_usage_limit && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "--buffer-usage-limit", "16");
+ }
+
+ if (vacopts->missing_stats_only && PQserverVersion(conn) < 150000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "--missing-stats-only", "15");
+ }
+
+ /* skip_database_stats is used automatically if server supports it */
+ vacopts->skip_database_stats = (PQserverVersion(conn) >= 160000);
+
+ if (!quiet)
+ {
+ if (stage != ANALYZE_NO_STAGE)
+ printf(_("%s: processing database \"%s\": %s\n"),
+ progname, PQdb(conn), _(stage_messages[stage]));
+ else
+ printf(_("%s: vacuuming database \"%s\"\n"),
+ progname, PQdb(conn));
+ fflush(stdout);
+ }
+
+ /*
+ * If the caller provided the results of a previous catalog query, just
+ * use that. Otherwise, run the catalog query ourselves and set the
+ * return variable if provided.
+ */
+ if (found_objs && *found_objs)
+ ret = *found_objs;
+ else
+ {
+ ret = retrieve_objects(conn, vacopts, objects, echo);
+ if (found_objs)
+ *found_objs = ret;
+ }
+
+ /*
+ * Count the number of objects in the catalog query result. If there are
+ * none, we are done.
+ */
+ for (cell = ret ? ret->head : NULL; cell; cell = cell->next)
+ ntups++;
+
+ if (ntups == 0)
+ {
+ PQfinish(conn);
+ return;
+ }
+
+ /*
+ * Ensure concurrentCons is sane. If there are more connections than
+ * vacuumable relations, we don't need to use them all.
+ */
+ if (concurrentCons > ntups)
+ concurrentCons = ntups;
+ if (concurrentCons <= 0)
+ concurrentCons = 1;
+
+ /*
+ * All slots need to be prepared to run the appropriate analyze stage, if
+ * caller requested that mode. We have to prepare the initial connection
+ * ourselves before setting up the slots.
+ */
+ if (stage == ANALYZE_NO_STAGE)
+ initcmd = NULL;
+ else
+ {
+ initcmd = stage_commands[stage];
+ executeCommand(conn, initcmd, echo);
+ }
+
+ /*
+ * Setup the database connections. We reuse the connection we already have
+ * for the first slot. If not in parallel mode, the first slot in the
+ * array contains the connection.
+ */
+ sa = ParallelSlotsSetup(concurrentCons, cparams, progname, echo, initcmd);
+ ParallelSlotsAdoptConn(sa, conn);
+
+ initPQExpBuffer(&sql);
+
+ cell = ret->head;
+ do
+ {
+ const char *tabname = cell->val;
+ ParallelSlot *free_slot;
+
+ if (CancelRequested)
+ {
+ failed = true;
+ goto finish;
+ }
+
+ free_slot = ParallelSlotsGetIdle(sa, NULL);
+ if (!free_slot)
+ {
+ failed = true;
+ goto finish;
+ }
+
+ prepare_vacuum_command(&sql, PQserverVersion(free_slot->connection),
+ vacopts, tabname);
+
+ /*
+ * Execute the vacuum. All errors are handled in processQueryResult
+ * through ParallelSlotsGetIdle.
+ */
+ ParallelSlotSetHandler(free_slot, TableCommandResultHandler, NULL);
+ run_vacuum_command(free_slot->connection, sql.data,
+ echo, tabname);
+
+ cell = cell->next;
+ } while (cell != NULL);
+
+ if (!ParallelSlotsWaitCompletion(sa))
+ {
+ failed = true;
+ goto finish;
+ }
+
+ /* If we used SKIP_DATABASE_STATS, mop up with ONLY_DATABASE_STATS */
+ if (vacopts->skip_database_stats &&
+ stage == ANALYZE_NO_STAGE)
+ {
+ const char *cmd = "VACUUM (ONLY_DATABASE_STATS);";
+ ParallelSlot *free_slot = ParallelSlotsGetIdle(sa, NULL);
+
+ if (!free_slot)
+ {
+ failed = true;
+ goto finish;
+ }
+
+ ParallelSlotSetHandler(free_slot, TableCommandResultHandler, NULL);
+ run_vacuum_command(free_slot->connection, cmd, echo, NULL);
+
+ if (!ParallelSlotsWaitCompletion(sa))
+ failed = true;
+ }
+
+finish:
+ ParallelSlotsTerminate(sa);
+ pg_free(sa);
+
+ termPQExpBuffer(&sql);
+
+ if (failed)
+ exit(1);
+}
+
+/*
+ * Prepare the list of tables to process by querying the catalogs.
+ *
+ * Since we execute the constructed query with the default search_path (which
+ * could be unsafe), everything in this query MUST be fully qualified.
+ *
+ * First, build a WITH clause for the catalog query if any tables were
+ * specified, with a set of values made of relation names and their optional
+ * set of columns. This is used to match any provided column lists with the
+ * generated qualified identifiers and to filter for the tables provided via
+ * --table. If a listed table does not exist, the catalog query will fail.
+ */
+SimpleStringList *
+retrieve_objects(PGconn *conn, vacuumingOptions *vacopts,
+ SimpleStringList *objects, bool echo)
+{
+ PQExpBufferData buf;
+ PQExpBufferData catalog_query;
+ PGresult *res;
+ SimpleStringListCell *cell;
+ SimpleStringList *found_objs = palloc0(sizeof(SimpleStringList));
+ bool objects_listed = false;
+
+ initPQExpBuffer(&catalog_query);
+ for (cell = objects ? objects->head : NULL; cell; cell = cell->next)
+ {
+ char *just_table = NULL;
+ const char *just_columns = NULL;
+
+ if (!objects_listed)
+ {
+ appendPQExpBufferStr(&catalog_query,
+ "WITH listed_objects (object_oid, column_list) AS (\n"
+ " VALUES (");
+ objects_listed = true;
+ }
+ else
+ appendPQExpBufferStr(&catalog_query, ",\n (");
+
+ if (objfilter & (OBJFILTER_SCHEMA | OBJFILTER_SCHEMA_EXCLUDE))
+ {
+ appendStringLiteralConn(&catalog_query, cell->val, conn);
+ appendPQExpBufferStr(&catalog_query, "::pg_catalog.regnamespace, ");
+ }
+
+ if (objfilter & OBJFILTER_TABLE)
+ {
+ /*
+ * Split relation and column names given by the user, this is used
+ * to feed the CTE with values on which are performed pre-run
+ * validity checks as well. For now these happen only on the
+ * relation name.
+ */
+ splitTableColumnsSpec(cell->val, PQclientEncoding(conn),
+ &just_table, &just_columns);
+
+ appendStringLiteralConn(&catalog_query, just_table, conn);
+ appendPQExpBufferStr(&catalog_query, "::pg_catalog.regclass, ");
+ }
+
+ if (just_columns && just_columns[0] != '\0')
+ appendStringLiteralConn(&catalog_query, just_columns, conn);
+ else
+ appendPQExpBufferStr(&catalog_query, "NULL");
+
+ appendPQExpBufferStr(&catalog_query, "::pg_catalog.text)");
+
+ pg_free(just_table);
+ }
+
+ /* Finish formatting the CTE */
+ if (objects_listed)
+ appendPQExpBufferStr(&catalog_query, "\n)\n");
+
+ appendPQExpBufferStr(&catalog_query, "SELECT c.relname, ns.nspname");
+
+ if (objects_listed)
+ appendPQExpBufferStr(&catalog_query, ", listed_objects.column_list");
+
+ appendPQExpBufferStr(&catalog_query,
+ " FROM pg_catalog.pg_class c\n"
+ " JOIN pg_catalog.pg_namespace ns"
+ " ON c.relnamespace OPERATOR(pg_catalog.=) ns.oid\n"
+ " CROSS JOIN LATERAL (SELECT c.relkind IN ("
+ CppAsString2(RELKIND_PARTITIONED_TABLE) ", "
+ CppAsString2(RELKIND_PARTITIONED_INDEX) ")) as p (inherited)\n"
+ " LEFT JOIN pg_catalog.pg_class t"
+ " ON c.reltoastrelid OPERATOR(pg_catalog.=) t.oid\n");
+
+ /*
+ * Used to match the tables or schemas listed by the user, completing the
+ * JOIN clause.
+ */
+ if (objects_listed)
+ {
+ appendPQExpBufferStr(&catalog_query, " LEFT JOIN listed_objects"
+ " ON listed_objects.object_oid"
+ " OPERATOR(pg_catalog.=) ");
+
+ if (objfilter & OBJFILTER_TABLE)
+ appendPQExpBufferStr(&catalog_query, "c.oid\n");
+ else
+ appendPQExpBufferStr(&catalog_query, "ns.oid\n");
+ }
+
+ /*
+ * Exclude temporary tables, beginning the WHERE clause.
+ */
+ appendPQExpBufferStr(&catalog_query,
+ " WHERE c.relpersistence OPERATOR(pg_catalog.!=) "
+ CppAsString2(RELPERSISTENCE_TEMP) "\n");
+
+ /*
+ * Used to match the tables or schemas listed by the user, for the WHERE
+ * clause.
+ */
+ if (objects_listed)
+ {
+ if (objfilter & OBJFILTER_SCHEMA_EXCLUDE)
+ appendPQExpBufferStr(&catalog_query,
+ " AND listed_objects.object_oid IS NULL\n");
+ else
+ appendPQExpBufferStr(&catalog_query,
+ " AND listed_objects.object_oid IS NOT NULL\n");
+ }
+
+ /*
+ * If no tables were listed, filter for the relevant relation types. If
+ * tables were given via --table, don't bother filtering by relation type.
+ * Instead, let the server decide whether a given relation can be
+ * processed in which case the user will know about it.
+ */
+ if ((objfilter & OBJFILTER_TABLE) == 0)
+ {
+ /*
+ * vacuumdb should generally follow the behavior of the underlying
+ * VACUUM and ANALYZE commands. If analyze_only is true, process
+ * regular tables, materialized views, and partitioned tables, just
+ * like ANALYZE (with no specific target tables) does. Otherwise,
+ * process only regular tables and materialized views, since VACUUM
+ * skips partitioned tables when no target tables are specified.
+ */
+ if (vacopts->analyze_only)
+ appendPQExpBufferStr(&catalog_query,
+ " AND c.relkind OPERATOR(pg_catalog.=) ANY (array["
+ CppAsString2(RELKIND_RELATION) ", "
+ CppAsString2(RELKIND_MATVIEW) ", "
+ CppAsString2(RELKIND_PARTITIONED_TABLE) "])\n");
+ else
+ appendPQExpBufferStr(&catalog_query,
+ " AND c.relkind OPERATOR(pg_catalog.=) ANY (array["
+ CppAsString2(RELKIND_RELATION) ", "
+ CppAsString2(RELKIND_MATVIEW) "])\n");
+ }
+
+ /*
+ * For --min-xid-age and --min-mxid-age, the age of the relation is the
+ * greatest of the ages of the main relation and its associated TOAST
+ * table. The commands generated by vacuumdb will also process the TOAST
+ * table for the relation if necessary, so it does not need to be
+ * considered separately.
+ */
+ if (vacopts->min_xid_age != 0)
+ {
+ appendPQExpBuffer(&catalog_query,
+ " AND GREATEST(pg_catalog.age(c.relfrozenxid),"
+ " pg_catalog.age(t.relfrozenxid)) "
+ " OPERATOR(pg_catalog.>=) '%d'::pg_catalog.int4\n"
+ " AND c.relfrozenxid OPERATOR(pg_catalog.!=)"
+ " '0'::pg_catalog.xid\n",
+ vacopts->min_xid_age);
+ }
+
+ if (vacopts->min_mxid_age != 0)
+ {
+ appendPQExpBuffer(&catalog_query,
+ " AND GREATEST(pg_catalog.mxid_age(c.relminmxid),"
+ " pg_catalog.mxid_age(t.relminmxid)) OPERATOR(pg_catalog.>=)"
+ " '%d'::pg_catalog.int4\n"
+ " AND c.relminmxid OPERATOR(pg_catalog.!=)"
+ " '0'::pg_catalog.xid\n",
+ vacopts->min_mxid_age);
+ }
+
+ if (vacopts->missing_stats_only)
+ {
+ appendPQExpBufferStr(&catalog_query, " AND (\n");
+
+ /* regular stats */
+ appendPQExpBufferStr(&catalog_query,
+ " EXISTS (SELECT NULL FROM pg_catalog.pg_attribute a\n"
+ " WHERE a.attrelid OPERATOR(pg_catalog.=) c.oid\n"
+ " AND a.attnum OPERATOR(pg_catalog.>) 0::pg_catalog.int2\n"
+ " AND NOT a.attisdropped\n"
+ " AND a.attstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
+ " AND a.attgenerated OPERATOR(pg_catalog.<>) "
+ CppAsString2(ATTRIBUTE_GENERATED_VIRTUAL) "\n"
+ " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic s\n"
+ " WHERE s.starelid OPERATOR(pg_catalog.=) a.attrelid\n"
+ " AND s.staattnum OPERATOR(pg_catalog.=) a.attnum\n"
+ " AND s.stainherit OPERATOR(pg_catalog.=) p.inherited))\n");
+
+ /* extended stats */
+ appendPQExpBufferStr(&catalog_query,
+ " OR EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext e\n"
+ " WHERE e.stxrelid OPERATOR(pg_catalog.=) c.oid\n"
+ " AND e.stxstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
+ " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext_data d\n"
+ " WHERE d.stxoid OPERATOR(pg_catalog.=) e.oid\n"
+ " AND d.stxdinherit OPERATOR(pg_catalog.=) p.inherited))\n");
+
+ /* expression indexes */
+ appendPQExpBufferStr(&catalog_query,
+ " OR EXISTS (SELECT NULL FROM pg_catalog.pg_attribute a\n"
+ " JOIN pg_catalog.pg_index i"
+ " ON i.indexrelid OPERATOR(pg_catalog.=) a.attrelid\n"
+ " WHERE i.indrelid OPERATOR(pg_catalog.=) c.oid\n"
+ " AND i.indkey[a.attnum OPERATOR(pg_catalog.-) 1::pg_catalog.int2]"
+ " OPERATOR(pg_catalog.=) 0::pg_catalog.int2\n"
+ " AND a.attnum OPERATOR(pg_catalog.>) 0::pg_catalog.int2\n"
+ " AND NOT a.attisdropped\n"
+ " AND a.attstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
+ " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic s\n"
+ " WHERE s.starelid OPERATOR(pg_catalog.=) a.attrelid\n"
+ " AND s.staattnum OPERATOR(pg_catalog.=) a.attnum\n"
+ " AND s.stainherit OPERATOR(pg_catalog.=) p.inherited))\n");
+
+ /* inheritance and regular stats */
+ appendPQExpBufferStr(&catalog_query,
+ " OR EXISTS (SELECT NULL FROM pg_catalog.pg_attribute a\n"
+ " WHERE a.attrelid OPERATOR(pg_catalog.=) c.oid\n"
+ " AND a.attnum OPERATOR(pg_catalog.>) 0::pg_catalog.int2\n"
+ " AND NOT a.attisdropped\n"
+ " AND a.attstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
+ " AND a.attgenerated OPERATOR(pg_catalog.<>) "
+ CppAsString2(ATTRIBUTE_GENERATED_VIRTUAL) "\n"
+ " AND c.relhassubclass\n"
+ " AND NOT p.inherited\n"
+ " AND EXISTS (SELECT NULL FROM pg_catalog.pg_inherits h\n"
+ " WHERE h.inhparent OPERATOR(pg_catalog.=) c.oid)\n"
+ " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic s\n"
+ " WHERE s.starelid OPERATOR(pg_catalog.=) a.attrelid\n"
+ " AND s.staattnum OPERATOR(pg_catalog.=) a.attnum\n"
+ " AND s.stainherit))\n");
+
+ /* inheritance and extended stats */
+ appendPQExpBufferStr(&catalog_query,
+ " OR EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext e\n"
+ " WHERE e.stxrelid OPERATOR(pg_catalog.=) c.oid\n"
+ " AND e.stxstattarget IS DISTINCT FROM 0::pg_catalog.int2\n"
+ " AND c.relhassubclass\n"
+ " AND NOT p.inherited\n"
+ " AND EXISTS (SELECT NULL FROM pg_catalog.pg_inherits h\n"
+ " WHERE h.inhparent OPERATOR(pg_catalog.=) c.oid)\n"
+ " AND NOT EXISTS (SELECT NULL FROM pg_catalog.pg_statistic_ext_data d\n"
+ " WHERE d.stxoid OPERATOR(pg_catalog.=) e.oid\n"
+ " AND d.stxdinherit))\n");
+
+ appendPQExpBufferStr(&catalog_query, " )\n");
+ }
+
+ /*
+ * Execute the catalog query. We use the default search_path for this
+ * query for consistency with table lookups done elsewhere by the user.
+ */
+ appendPQExpBufferStr(&catalog_query, " ORDER BY c.relpages DESC;");
+ executeCommand(conn, "RESET search_path;", echo);
+ res = executeQuery(conn, catalog_query.data, echo);
+ termPQExpBuffer(&catalog_query);
+ PQclear(executeQuery(conn, ALWAYS_SECURE_SEARCH_PATH_SQL, echo));
+
+ /*
+ * Build qualified identifiers for each table, including the column list
+ * if given.
+ */
+ initPQExpBuffer(&buf);
+ for (int i = 0; i < PQntuples(res); i++)
+ {
+ appendPQExpBufferStr(&buf,
+ fmtQualifiedIdEnc(PQgetvalue(res, i, 1),
+ PQgetvalue(res, i, 0),
+ PQclientEncoding(conn)));
+
+ if (objects_listed && !PQgetisnull(res, i, 2))
+ appendPQExpBufferStr(&buf, PQgetvalue(res, i, 2));
+
+ simple_string_list_append(found_objs, buf.data);
+ resetPQExpBuffer(&buf);
+ }
+ termPQExpBuffer(&buf);
+ PQclear(res);
+
+ return found_objs;
+}
+
+/*
+ * Vacuum/analyze all connectable databases.
+ *
+ * In analyze-in-stages mode, we process all databases in one stage before
+ * moving on to the next stage. That ensure minimal stats are available
+ * quickly everywhere before generating more detailed ones.
+ */
+void
+vacuum_all_databases(ConnParams *cparams,
+ vacuumingOptions *vacopts,
+ bool analyze_in_stages,
+ SimpleStringList *objects,
+ int concurrentCons,
+ const char *progname, bool echo, bool quiet)
+{
+ PGconn *conn;
+ PGresult *result;
+ int stage;
+ int i;
+
+ conn = connectMaintenanceDatabase(cparams, progname, echo);
+ result = executeQuery(conn,
+ "SELECT datname FROM pg_database WHERE datallowconn AND datconnlimit <> -2 ORDER BY 1;",
+ echo);
+ PQfinish(conn);
+
+ if (analyze_in_stages)
+ {
+ SimpleStringList **found_objs = NULL;
+
+ if (vacopts->missing_stats_only)
+ found_objs = palloc0(PQntuples(result) * sizeof(SimpleStringList *));
+
+ /*
+ * When analyzing all databases in stages, we analyze them all in the
+ * fastest stage first, so that initial statistics become available
+ * for all of them as soon as possible.
+ *
+ * This means we establish several times as many connections, but
+ * that's a secondary consideration.
+ */
+ for (stage = 0; stage < ANALYZE_NUM_STAGES; stage++)
+ {
+ for (i = 0; i < PQntuples(result); i++)
+ {
+ cparams->override_dbname = PQgetvalue(result, i, 0);
+
+ vacuum_one_database(cparams, vacopts,
+ stage,
+ objects,
+ vacopts->missing_stats_only ? &found_objs[i] : NULL,
+ concurrentCons,
+ progname, echo, quiet);
+ }
+ }
+ }
+ else
+ {
+ for (i = 0; i < PQntuples(result); i++)
+ {
+ cparams->override_dbname = PQgetvalue(result, i, 0);
+
+ vacuum_one_database(cparams, vacopts,
+ ANALYZE_NO_STAGE,
+ objects, NULL,
+ concurrentCons,
+ progname, echo, quiet);
+ }
+ }
+
+ PQclear(result);
+}
+
+/*
+ * Construct a vacuum/analyze command to run based on the given
+ * options, in the given string buffer, which may contain previous garbage.
+ *
+ * The table name used must be already properly quoted. The command generated
+ * depends on the server version involved and it is semicolon-terminated.
+ */
+void
+prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
+ vacuumingOptions *vacopts, const char *table)
+{
+ const char *paren = " (";
+ const char *comma = ", ";
+ const char *sep = paren;
+
+ resetPQExpBuffer(sql);
+
+ if (vacopts->analyze_only)
+ {
+ appendPQExpBufferStr(sql, "ANALYZE");
+
+ /* parenthesized grammar of ANALYZE is supported since v11 */
+ if (serverVersion >= 110000)
+ {
+ if (vacopts->skip_locked)
+ {
+ /* SKIP_LOCKED is supported since v12 */
+ Assert(serverVersion >= 120000);
+ appendPQExpBuffer(sql, "%sSKIP_LOCKED", sep);
+ sep = comma;
+ }
+ if (vacopts->verbose)
+ {
+ appendPQExpBuffer(sql, "%sVERBOSE", sep);
+ sep = comma;
+ }
+ if (vacopts->buffer_usage_limit)
+ {
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sBUFFER_USAGE_LIMIT '%s'", sep,
+ vacopts->buffer_usage_limit);
+ sep = comma;
+ }
+ if (sep != paren)
+ appendPQExpBufferChar(sql, ')');
+ }
+ else
+ {
+ if (vacopts->verbose)
+ appendPQExpBufferStr(sql, " VERBOSE");
+ }
+ }
+ else
+ {
+ appendPQExpBufferStr(sql, "VACUUM");
+
+ /* parenthesized grammar of VACUUM is supported since v9.0 */
+ if (serverVersion >= 90000)
+ {
+ if (vacopts->disable_page_skipping)
+ {
+ /* DISABLE_PAGE_SKIPPING is supported since v9.6 */
+ Assert(serverVersion >= 90600);
+ appendPQExpBuffer(sql, "%sDISABLE_PAGE_SKIPPING", sep);
+ sep = comma;
+ }
+ if (vacopts->no_index_cleanup)
+ {
+ /* "INDEX_CLEANUP FALSE" has been supported since v12 */
+ Assert(serverVersion >= 120000);
+ Assert(!vacopts->force_index_cleanup);
+ appendPQExpBuffer(sql, "%sINDEX_CLEANUP FALSE", sep);
+ sep = comma;
+ }
+ if (vacopts->force_index_cleanup)
+ {
+ /* "INDEX_CLEANUP TRUE" has been supported since v12 */
+ Assert(serverVersion >= 120000);
+ Assert(!vacopts->no_index_cleanup);
+ appendPQExpBuffer(sql, "%sINDEX_CLEANUP TRUE", sep);
+ sep = comma;
+ }
+ if (!vacopts->do_truncate)
+ {
+ /* TRUNCATE is supported since v12 */
+ Assert(serverVersion >= 120000);
+ appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
+ sep = comma;
+ }
+ if (!vacopts->process_main)
+ {
+ /* PROCESS_MAIN is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
+ sep = comma;
+ }
+ if (!vacopts->process_toast)
+ {
+ /* PROCESS_TOAST is supported since v14 */
+ Assert(serverVersion >= 140000);
+ appendPQExpBuffer(sql, "%sPROCESS_TOAST FALSE", sep);
+ sep = comma;
+ }
+ if (vacopts->skip_database_stats)
+ {
+ /* SKIP_DATABASE_STATS is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sSKIP_DATABASE_STATS", sep);
+ sep = comma;
+ }
+ if (vacopts->skip_locked)
+ {
+ /* SKIP_LOCKED is supported since v12 */
+ Assert(serverVersion >= 120000);
+ appendPQExpBuffer(sql, "%sSKIP_LOCKED", sep);
+ sep = comma;
+ }
+ if (vacopts->full)
+ {
+ appendPQExpBuffer(sql, "%sFULL", sep);
+ sep = comma;
+ }
+ if (vacopts->freeze)
+ {
+ appendPQExpBuffer(sql, "%sFREEZE", sep);
+ sep = comma;
+ }
+ if (vacopts->verbose)
+ {
+ appendPQExpBuffer(sql, "%sVERBOSE", sep);
+ sep = comma;
+ }
+ if (vacopts->and_analyze)
+ {
+ appendPQExpBuffer(sql, "%sANALYZE", sep);
+ sep = comma;
+ }
+ if (vacopts->parallel_workers >= 0)
+ {
+ /* PARALLEL is supported since v13 */
+ Assert(serverVersion >= 130000);
+ appendPQExpBuffer(sql, "%sPARALLEL %d", sep,
+ vacopts->parallel_workers);
+ sep = comma;
+ }
+ if (vacopts->buffer_usage_limit)
+ {
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sBUFFER_USAGE_LIMIT '%s'", sep,
+ vacopts->buffer_usage_limit);
+ sep = comma;
+ }
+ if (sep != paren)
+ appendPQExpBufferChar(sql, ')');
+ }
+ else
+ {
+ if (vacopts->full)
+ appendPQExpBufferStr(sql, " FULL");
+ if (vacopts->freeze)
+ appendPQExpBufferStr(sql, " FREEZE");
+ if (vacopts->verbose)
+ appendPQExpBufferStr(sql, " VERBOSE");
+ if (vacopts->and_analyze)
+ appendPQExpBufferStr(sql, " ANALYZE");
+ }
+ }
+
+ appendPQExpBuffer(sql, " %s;", table);
+}
+
+/*
+ * Send a vacuum/analyze command to the server, returning after sending the
+ * command.
+ *
+ * Any errors during command execution are reported to stderr.
+ */
+void
+run_vacuum_command(PGconn *conn, const char *sql, bool echo,
+ const char *table)
+{
+ bool status;
+
+ if (echo)
+ printf("%s\n", sql);
+
+ status = PQsendQuery(conn, sql) == 1;
+
+ if (!status)
+ {
+ if (table)
+ {
+ pg_log_error("vacuuming of table \"%s\" in database \"%s\" failed: %s",
+ table, PQdb(conn), PQerrorMessage(conn));
+ }
+ else
+ {
+ pg_log_error("vacuuming of database \"%s\" failed: %s",
+ PQdb(conn), PQerrorMessage(conn));
+ }
+ }
+}
+
+/*
+ * Returns a newly malloc'd version of 'src' with escaped single quotes and
+ * backslashes.
+ */
+char *
+escape_quotes(const char *src)
+{
+ char *result = escape_single_quotes_ascii(src);
+
+ if (!result)
+ pg_fatal("out of memory");
+ return result;
+}
diff --git a/src/bin/scripts/vacuuming.h b/src/bin/scripts/vacuuming.h
new file mode 100644
index 00000000000..d3f000840fa
--- /dev/null
+++ b/src/bin/scripts/vacuuming.h
@@ -0,0 +1,95 @@
+/*-------------------------------------------------------------------------
+ *
+ * vacuuming.h
+ * Common declarations for vacuuming.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/bin/scripts/vacuuming.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef VACUUMING_H
+#define VACUUMING_H
+
+#include "common.h"
+#include "fe_utils/connect_utils.h"
+#include "fe_utils/simple_list.h"
+
+/* For analyze-in-stages mode */
+#define ANALYZE_NO_STAGE -1
+#define ANALYZE_NUM_STAGES 3
+
+/* vacuum options controlled by user flags */
+typedef struct vacuumingOptions
+{
+ bool analyze_only;
+ bool verbose;
+ bool and_analyze;
+ bool full;
+ bool freeze;
+ bool disable_page_skipping;
+ bool skip_locked;
+ int min_xid_age;
+ int min_mxid_age;
+ int parallel_workers; /* >= 0 indicates user specified the
+ * parallel degree, otherwise -1 */
+ bool no_index_cleanup;
+ bool force_index_cleanup;
+ bool do_truncate;
+ bool process_main;
+ bool process_toast;
+ bool skip_database_stats;
+ char *buffer_usage_limit;
+ bool missing_stats_only;
+} vacuumingOptions;
+
+/* object filter options */
+typedef enum
+{
+ OBJFILTER_NONE = 0, /* no filter used */
+ OBJFILTER_ALL_DBS = (1 << 0), /* -a | --all */
+ OBJFILTER_DATABASE = (1 << 1), /* -d | --dbname */
+ OBJFILTER_TABLE = (1 << 2), /* -t | --table */
+ OBJFILTER_SCHEMA = (1 << 3), /* -n | --schema */
+ OBJFILTER_SCHEMA_EXCLUDE = (1 << 4), /* -N | --exclude-schema */
+} VacObjFilter;
+
+extern VacObjFilter objfilter;
+
+extern void vacuuming_main(ConnParams *cparams, const char *dbname,
+ const char *maintenance_db, vacuumingOptions *vacopts,
+ SimpleStringList *objects, bool analyze_in_stages,
+ int tbl_count, int concurrentCons,
+ const char *progname, bool echo, bool quiet);
+
+extern SimpleStringList *retrieve_objects(PGconn *conn,
+ vacuumingOptions *vacopts,
+ SimpleStringList *objects,
+ bool echo);
+
+extern void vacuum_one_database(ConnParams *cparams,
+ vacuumingOptions *vacopts,
+ int stage,
+ SimpleStringList *objects,
+ SimpleStringList **found_objs,
+ int concurrentCons,
+ const char *progname, bool echo, bool quiet);
+
+extern void vacuum_all_databases(ConnParams *cparams,
+ vacuumingOptions *vacopts,
+ bool analyze_in_stages,
+ SimpleStringList *objects,
+ int concurrentCons,
+ const char *progname, bool echo, bool quiet);
+
+extern void prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
+ vacuumingOptions *vacopts, const char *table);
+
+extern void run_vacuum_command(PGconn *conn, const char *sql, bool echo,
+ const char *table);
+
+extern char *escape_quotes(const char *src);
+
+#endif /* VACUUMING_H */
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 05:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-01 09:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 15:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-02 10:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-09-03 09:55 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-09-03 09:55 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> While testing MVCC-safe version with stress-tests
> 007_repack_concurrently_mvcc.pl I encountered some random crashes with
> such logs:
>
> 25-09-02 12:24:40.039 CEST client backend[261907]
> 007_repack_concurrently_mvcc.pl ERROR: relcache reference
> 0x7715b9f394a8 is not owned by resource owner TopTransaction
> ...
> This time I was clever and tried to attempt to reproduce the issue on
> a non-MVCC safe version at first - and it is reproducible.
Thanks again for a thorough testing!
I think this should be fixed separately [1].
[1] https://www.postgresql.org/message-id/119497.1756892972%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-09-23 15:51 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-09-23 15:51 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; +Cc: Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Hello,
Barring further commentary, I intend to get 0001 committed tomorrow, and
0002 some time later -- perhaps by end of this week, or sometime next
week.
Regards
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-09-25 18:12 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-09-25 20:20 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
2025-09-26 14:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-26 17:30 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
1 sibling, 3 replies; 416+ messages in thread
From: Álvaro Herrera @ 2025-09-25 18:12 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; +Cc: Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
After looking at this some more, I realized that 0001 had been written a
bit too hastily and that it could use with some more cleanup -- in
particular, we don't need to export most of the function prototypes
other than vacuuming_main() (and the trivial escape_quotes helper). I
made the other functions static. Also, prepare_vacuum_command() also
needs the encoding in order to do fmtIdEnc() on a given index name (for
`pg_repackdb -t table --index=foobar`), so I changed it to take the
PGconn instead of just the serverVersion. I realized that it makes no
sense that objfilter is a global variable instead of living inside
`main` and be passed as argument where needed. (Heck, maybe it should
be inside vacuumingOpts). Lastly, it seemed weird coding that the
functions would sometimes exit(1) instead of returning a result code, so
I made them do that and have the callers react appropriately. These are
all fairly straightforward changes.
So here's v22 with those and rebased to current sources. Only the first
two patches this time, which are the ones I would be glad to receive
input on.
I also wonder if analyze_only and analyze_in_stages should be new values
in RunMode rather than separate booleans ... I think that might make the
code simpler. I didn't try though.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Los dioses no protegen a los insensatos. Éstos reciben protección de
otros insensatos mejor dotados" (Luis Wu, Mundo Anillo)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-09-25 20:20 ` Marcos Pegoraro <marcos@f10.com.br>
2025-09-25 21:31 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2 siblings, 1 reply; 416+ messages in thread
From: Marcos Pegoraro @ 2025-09-25 20:20 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Em qui., 25 de set. de 2025 às 15:12, Álvaro Herrera <alvherre@kurilemu.de>
escreveu:
Some typos I've found on usage of pg_repackdb.
+ printf(_(" -n, --schema=SCHEMA repack tables in the
specified schema(s) only\n"));
+ printf(_(" -N, --exclude-schema=SCHEMA do not repack tables in the
specified schema(s)\n"));
both options can point to a single schema, so "(s)" should be removed.
"in the specified schema(s)" should be "in the specified schema"
Same occurs on this one, which should be table, not table(s)
+ printf(_(" -t, --table='TABLE' repack specific table(s)
only\n"));
regards
Marcos
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-09-25 20:20 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
@ 2025-09-25 21:31 ` Robert Treat <rob@xzilla.net>
2025-09-25 21:46 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
0 siblings, 1 reply; 416+ messages in thread
From: Robert Treat @ 2025-09-25 21:31 UTC (permalink / raw)
To: Marcos Pegoraro <marcos@f10.com.br>; +Cc: Álvaro Herrera <alvherre@kurilemu.de>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Thu, Sep 25, 2025 at 4:21 PM Marcos Pegoraro <marcos@f10.com.br> wrote:
>
> Em qui., 25 de set. de 2025 às 15:12, Álvaro Herrera <alvherre@kurilemu.de> escreveu:
>
> Some typos I've found on usage of pg_repackdb.
>
> + printf(_(" -n, --schema=SCHEMA repack tables in the specified schema(s) only\n"));
> + printf(_(" -N, --exclude-schema=SCHEMA do not repack tables in the specified schema(s)\n"));
> both options can point to a single schema, so "(s)" should be removed.
> "in the specified schema(s)" should be "in the specified schema"
>
> Same occurs on this one, which should be table, not table(s)
> + printf(_(" -t, --table='TABLE' repack specific table(s) only\n"));
>
This pattern is used because you can pass more than one argument, for
example, something like
pg_repackdb -d pagila -v -n public -n legacy
While I agree that the wording is a little awkward; I'd prefer "repack
tables only in the specified schema(s)"; but this follows the same
pattern as pg_dump and friends.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-09-25 20:20 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
2025-09-25 21:31 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-09-25 21:46 ` Marcos Pegoraro <marcos@f10.com.br>
0 siblings, 0 replies; 416+ messages in thread
From: Marcos Pegoraro @ 2025-09-25 21:46 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Álvaro Herrera <alvherre@kurilemu.de>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Em qui., 25 de set. de 2025 às 18:31, Robert Treat <rob@xzilla.net>
escreveu:
> This pattern is used because you can pass more than one argument, for
> example, something like
I know that
>
> While I agree that the wording is a little awkward, this follows the same
> pattern as pg_dump and friends.
>
well, I think pg_dump looks wrong too. Because if you explain that it's a
single table or single schema on docs, why you write on plural on usage ?
+ Repack or analyze all tables in
+ <replaceable class="parameter">schema</replaceable> only. Multiple
+ schemas can be repacked by writing multiple <option>-n</option>
+ switches.
instead of
+ printf(_(" -n, --schema=SCHEMA repack tables in the
specified schema(s) only\n"));
maybe this ?
+ printf(_(" -n, --schema=SCHEMA repack tables in the
specified schema, can be used several times\n"));
regards
Marcos
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-09-26 14:27 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-10-07 14:05 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-09-26 14:27 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Robert Treat <rob@xzilla.net>; Fujii Masao <masao.fujii@gmail.com>
Hello!
Álvaro Herrera <alvherre@kurilemu.de>:
> So here's v22 with those and rebased to current sources. Only the first
> two patches this time, which are the ones I would be glad to receive
> input on.
> get_tables_to_repack_partitioned(RepackCommand cmd, MemoryContext cluster_context,
> Oid relid, bool rel_is_index)
Should we rename it to repack_context to be aligned with the calling side?
---------
'cmd' in
> static List *get_tables_to_repack(RepackCommand cmd, bool usingindex,
> MemoryContext permcxt);
but 'command' in
> get_tables_to_repack(RepackCommand command, bool usingindex,
> MemoryContext permcxt)
---------
> cmd == REPACK_COMMAND_CLUSTER ? "CLUSTER" : "REPACK",
May be changed to RepackCommandAsString
-----------
if (cmd == REPACK_COMMAND_REPACK)
pgstat_progress_update_param(PROGRESS_REPACK_COMMAND,
PROGRESS_REPACK_COMMAND_REPACK);
else if (cmd == REPACK_COMMAND_CLUSTER)
{
pgstat_progress_update_param(PROGRESS_REPACK_COMMAND,
PROGRESS_CLUSTER_COMMAND_CLUSTER);
} else ....
'{' and '}' looks a little bit weird.
--------
Documentation of pg_repackdb contains a lot of "analyze" and even
"--analyze" parameter - but I can't see anything related in the code.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-09-26 14:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-10-07 14:05 ` Álvaro Herrera <alvherre@kurilemu.de>
2025-10-09 06:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-10-13 00:03 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
0 siblings, 2 replies; 416+ messages in thread
From: Álvaro Herrera @ 2025-10-07 14:05 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>
On 2025-Sep-26, Mihail Nikalayeu wrote:
> Should we rename it to repack_context to be aligned with the calling side?
Sure, done.
> > cmd == REPACK_COMMAND_CLUSTER ? "CLUSTER" : "REPACK",
>
> May be changed to RepackCommandAsString
Oh, of course.
> Documentation of pg_repackdb contains a lot of "analyze" and even
> "--analyze" parameter - but I can't see anything related in the code.
Hmm, yeah, that was missing. I added it. In doing so I noticed that
because vacuumdb allows a column list to be given, then we should do
likewise here, both in pg_repackdb and in the REPACK command, so I added
support for that. This changed the grammar a little bit. Note that we
still don't allow multiple tables to be given to the SQL command REPACK,
so if you want to repack multiple tables, you need to call it without
giving a name or give the name of a partitioned table. The pg_repackdb
utility allows you to give multiple -t switches, and in that case it
calls REPACK once for each name.
Also, if you give a column list to pg_repackdb, then you must pass -z.
This is consistent with vacuumdb via VACUUM ANALYZE.
On 2025-Sep-26, Robert Treat wrote:
> #1
> "pg_repackdb --help" does not mention the --index option, although the
> flag is accepted. I'm not sure if this is meant to match clusterdb,
> but since we need the index option to invoke the clustering behavior,
> I think it needs to be there.
Oops, yes, added.
> #2
> [xzilla@zebes] pgsql/bin/pg_repackdb -d pagila -v -t customer
> --index=idx_last_name
> pg_repackdb: repacking database "pagila"
> INFO: clustering "public.customer" using sequential scan and sort
>
> [xzilla@zebes] pgsql/bin/pg_repackdb -d pagila -v -t customer
> pg_repackdb: repacking database "pagila"
> INFO: vacuuming "public.customer"
>
> This was less confusing once I figured out we could pass the --index
> option, but even with that it is a little confusing, I think mostly
> because it looks like we are "vacuuming" the table, which in a world
> of repack and vacuum (ie. no vacuum full) doesn't make sense. I think
> the right thing to do here would be to modify it to be "repacking %s"
> in both cases, with the "using sequential scan and sort" as the means
> to understand which version of repack is being executed.
I changed these messages to always say "repacking", but it will say
"using sequential scan and sort", or "using index", or "following
physical order", respectively.
That said, on this topic, I've always been bothered by our usage of
command names as verbs, because they are (IMO) horrible for translation.
For instance, in this version of the patch I am making this change:
if (OidIsValid(indexOid) && OldHeap->rd_rel->relisshared)
ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot cluster a shared catalog")));
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot run %s on a shared catalog",
+ RepackCommandAsString(cmd)));
In the old version, the message is not very translatable because you
have to find a native word to say "to cluster" or "to vacuum", and that
doesn't always work very well in a direct translation. For instance, in
the Spanish message catalog you find this sort of thing:
msgid "vacuuming \"%s.%s.%s\""
msgstr "haciendo vacuum a «%s.%s.%s»"
which is pretty clear ... but the reason it works, is that I have turned
the phrase around before translating it. I would struggle if I had to
find a Spanish verb that means "to repack" without contorting the
message or saying something absurd and/or against Spanish language
rules, such as "ejecutando repack en table XYZ" or "repaqueando tabl
XYZ" (that's not a word!) or "reempaquetando tabla XYZ" (this is
correct, but far enough from "repack" that it's annoying and potentially
confusing). So I would rather the original used "running REPACK on
table using method XYZ", which is very very easy to translate, and then
the translator doesn't have to editorialize.
> #3
> pg_repackdb does not offer an --analyze option, which istm it should
> to match the REPACK command
Added, as mentioned above.
> #4
Fixed.
> #5
> [xzilla@zebes] pgsql/bin/pg_repackdb -d pagila -v -t film --index
> pg_repackdb: repacking database "pagila"
>
> In the above scenario, I am repacking without having previously
> specified an index. At the SQL level this would throw an error, at the
> command line it gives me a heart attack. :-)
> It's actually not that bad, because we don't actually do anything, but
> maybe we should throw an error?
Yeah, I think this is confusing. I think we should make pg_repackdb
explicitly indicate what has been done, in all cases, without requiring
-v. Otherwise it's too confusing, particularly for the using-index mode
which determines which tables to process based on the existance of an
index marked indiscluster.
> #6
> On the individual command pages (like sql-repack.html), I think there
> should be more cross-linking, ie. repack should probably say "see also
> cluster" and vice versa. Likely similarly with vacuum and repack.
Hmm, I don't necessarily agree -- I think the sql-cluster page should be
mostly empty and reference the sql-repack page. We don't need any
incoming links to sql-cluster, I think. All the useful info should be
in the sql-repack page only. The same applies for VACUUM FULL: an
outgoing link in sql-vacuum to sql-repack is good to have, but we don't
need links from sql-repack to sql-vacuum.
> #7
> Is there some reason you chose to intermingle the repack regression
> tests with the existing tests? I feel like it'd be easier to
> differentiate potential regressions and new functionality if these
> were separated.
I admit I haven't paid too much attention to these tests. I think I
would rather create a separate src/test/regress/sql/repack.sql file with
the tests for this command. Let's consider this part a WIP for now --
clearly more tests are needed both for the SQL command CLUSTER and for
pg_repackdb.
In the meantime, this version has been rebased to current sources.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-09-26 14:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-10-07 14:05 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-10-09 06:38 ` Antonin Houska <ah@cybertec.at>
2025-10-09 11:49 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-10-09 06:38 UTC (permalink / raw)
To: alvherre@kurilemu.de; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Fujii Masao <masao.fujii@gmail.com>
Álvaro Herrera <alvherre@kurilemu.de> wrote:
> On 2025-Sep-26, Mihail Nikalayeu wrote:
>
> > Should we rename it to repack_context to be aligned with the calling side?
>
> Sure, done.
>
> > > cmd == REPACK_COMMAND_CLUSTER ? "CLUSTER" : "REPACK",
> >
> > May be changed to RepackCommandAsString
>
> Oh, of course.
>
> > Documentation of pg_repackdb contains a lot of "analyze" and even
> > "--analyze" parameter - but I can't see anything related in the code.
>
> Hmm, yeah, that was missing. I added it. In doing so I noticed that
> because vacuumdb allows a column list to be given, then we should do
> likewise here, both in pg_repackdb and in the REPACK command, so I added
> support for that.
+ /*
+ * Make sure ANALYZE is specified if a column list is present.
+ */
+ if ((params->options & CLUOPT_ANALYZE) == 0 && stmt->relation->va_cols != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ANALYZE option must be specified when a column list is provided")));
Shouldn't the user documentation mention this restriction?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-09-26 14:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-10-07 14:05 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-10-09 06:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-10-09 11:49 ` Álvaro Herrera <alvherre@kurilemu.de>
0 siblings, 0 replies; 416+ messages in thread
From: Álvaro Herrera @ 2025-10-09 11:49 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Fujii Masao <masao.fujii@gmail.com>
On 2025-Oct-09, Antonin Houska wrote:
> + /*
> + * Make sure ANALYZE is specified if a column list is present.
> + */
> + if ((params->options & CLUOPT_ANALYZE) == 0 && stmt->relation->va_cols != NIL)
> + ereport(ERROR,
> + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> + errmsg("ANALYZE option must be specified when a column list is provided")));
>
> Shouldn't the user documentation mention this restriction?
Hmm, yeah, I guess it should. Will add.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"¿Cómo puedes confiar en algo que pagas y que no ves,
y no confiar en algo que te dan y te lo muestran?" (Germán Poo)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-09-26 14:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-10-07 14:05 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-10-13 00:03 ` Robert Treat <rob@xzilla.net>
1 sibling, 0 replies; 416+ messages in thread
From: Robert Treat @ 2025-10-13 00:03 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>
On Tue, Oct 7, 2025 at 10:05 AM Álvaro Herrera <alvherre@kurilemu.de> wrote:
> On 2025-Sep-26, Robert Treat wrote:
<snip>
> That said, on this topic, I've always been bothered by our usage of
> command names as verbs, because they are (IMO) horrible for translation.
> For instance, in this version of the patch I am making this change:
>
> if (OidIsValid(indexOid) && OldHeap->rd_rel->relisshared)
> ereport(ERROR,
> - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> - errmsg("cannot cluster a shared catalog")));
> + errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> + errmsg("cannot run %s on a shared catalog",
> + RepackCommandAsString(cmd)));
>
> In the old version, the message is not very translatable because you
> have to find a native word to say "to cluster" or "to vacuum", and that
> doesn't always work very well in a direct translation. For instance, in
> the Spanish message catalog you find this sort of thing:
>
> msgid "vacuuming \"%s.%s.%s\""
> msgstr "haciendo vacuum a «%s.%s.%s»"
>
> which is pretty clear ... but the reason it works, is that I have turned
> the phrase around before translating it. I would struggle if I had to
> find a Spanish verb that means "to repack" without contorting the
> message or saying something absurd and/or against Spanish language
> rules, such as "ejecutando repack en table XYZ" or "repaqueando tabl
> XYZ" (that's not a word!) or "reempaquetando tabla XYZ" (this is
> correct, but far enough from "repack" that it's annoying and potentially
> confusing). So I would rather the original used "running REPACK on
> table using method XYZ", which is very very easy to translate, and then
> the translator doesn't have to editorialize.
>
I see you didn't do this in the current patch, but +1 for this idea
from me. And if you think it'd help, I'm also +1 on the idea for the
main docs as well, for example doing something like
+ <para>
- <application>pg_repackdb</application> is a utility for repacking a
+ <application>pg_repackdb</application> is a utility for running REPACK on a
+ <productname>PostgreSQL</productname> database.
I'd be inclined to leave the internal comments alone though, since
they aren't translated.
> > #5
> > [xzilla@zebes] pgsql/bin/pg_repackdb -d pagila -v -t film --index
> > pg_repackdb: repacking database "pagila"
> >
> > In the above scenario, I am repacking without having previously
> > specified an index. At the SQL level this would throw an error, at the
> > command line it gives me a heart attack. :-)
> > It's actually not that bad, because we don't actually do anything, but
> > maybe we should throw an error?
>
> Yeah, I think this is confusing. I think we should make pg_repackdb
> explicitly indicate what has been done, in all cases, without requiring
> -v. Otherwise it's too confusing, particularly for the using-index mode
> which determines which tables to process based on the existance of an
> index marked indiscluster.
>
At the moment, clusterdb runs silently, but vacuumdb emits output, so
there is an argument for either way as default behavior. That said, I
think the current behavior of vacuum, which is what we are currently
following in pg_repackdb, is the worst of the two:
[xzilla@zebes] pgsql/bin/vacuumdb -t actor pagila
vacuumdb: vacuuming database "pagila"
Without any additional information, the information we do give is
misleading; I would rather not say anything. We could of course try to
make this more verbose, but I think clusterdb actually gets this
right...
- say nothing by default (follow the "rule of silence.")
- if we want to see commands, pass -e
- if we want to see the details, pass -v
- if we do something that causes an error, return the error
- if we don't want errors, pass -q
This is also how reindexdb works, and I think most of the other
utilities, and I'd argue this is how vacuumdb should work... to the
extent I almost consider it a bug that it doesn't (I leave a little
room since I am not sure why it doesn't operate like the other
utilities). vacuum is a bit outside the purview of what we are doing
here, but I do think following clusterdb/reindexdb is the behavior we
should follow for pg_repackdb.
> I admit I haven't paid too much attention to these tests. I think I
> would rather create a separate src/test/regress/sql/repack.sql file with
> the tests for this command. Let's consider this part a WIP for now --
> clearly more tests are needed both for the SQL command CLUSTER and for
> pg_repackdb.
Yeah, istm as long as we have all 3 commands (repack, cluster, vacuum
full) we need regression tests for all 3.
> - pg_stat_progress_cluster is no longer a view on top of the low-level
> pg_stat_get_progress_info() function. Instead, it's a view on top of
> pg_stat_progress_repack. The only change it applies on top of that
> one is change the command from REPACK to one of VACUUM FULL or
> CLUSTER, depending on whether an index is being used or not. This
> should keep the behavior identical to previous versions.
> Alternatively we could just hide rows where the command is REPACK, but
> I don't think that would be any better. This way, we maintain
> compatibility with tools reading pg_stat_progress_cluster. Maybe this
> is useless and we should just drop the view, not sure, we can discuss
> separately.
>
I think this mostly depends on how aggressive you want to be in moving
people away from cluster and toward repack. If we remove
_progress_cluster, it will force people to update monitoring which
probably encourages people to switch to pg_repackdb. We probably need
to have at least one "bridge" release though, and I think you've got
the right balance for that.
> - I noticed that you can do "CLUSTER pg_class ON some_index" and it will
> happily modify pg_index.indisclustered, which is a bit weird
> considering that allow_system_table_mods is off -- if you later try
> ALTER TABLE .. SET WITHOUT CLUSTER, it won't let you. I think this is
> bogus and we should change it so that CLUSTER refuses to change the
> clustered index on a system catalog, unless allow_system_table_mods is
> on. However, that would be a change from longstanding behavior which
> is specifically tested for in regression tests, so I didn't do it.
> We can discuss such a change separately. But I did make REPACK refuse
> to do that, because we don't need to propagate bogus historical
> behavior. So REPACK will fail if you try to change the indisclustered
> index, but it will work fine if you repack based on the same index as
> before, or repack with no index.
>
Since cluster will presumably be deprecated with this release, I'd
leave the existing behavior and move forward with repack as you've
laid out.
> - pg_repackdb: if you try with a non-superuser without specifying a
> table name, it will fail as soon as it hits the first catalog table or
> whatever with "ERROR: cannot lock this table". This is sorta fine for
> vacuumdb, but only because VACUUM itself will instead say "WARNING:
> cannot lock table XYZ, skipping", so it's not an error and vacuumdb
> keeps running. IMO this is bogus: vacuumdb should not try to process
> tables that it doesn't have privileges to. However, not wanting to
> change longstanding behavior, I left that alone. For pg_repackdb, I
> added a condition in the WHERE clause there to only fetch tables that
> the current user has MAINTAIN privilege over. Then you can do a
> "pg_repackdb -U foobar" and it will nicely process the tables that
> that user is allowed to process. We can discuss changing the vacuumdb
> behavior separately.
Again, vacuumdb seems to be a good example of what not to do, but I'll
leave that for another thread. In general I like this idea, but it
does make for a weird corner case where if I specify a table with -t
that I don't have permission to repack, repack returns silently whilst
doing nothing. I suppose one way to handle that would be to check if
the table passed in -t is found in the list of tables with MAINTAIN
privileges, and if not to issue a WARNING like "%s not found. Make
sure that the table exists and that you have MAINTAIN privileges".
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
@ 2025-09-26 17:30 ` Robert Treat <rob@xzilla.net>
2 siblings, 0 replies; 416+ messages in thread
From: Robert Treat @ 2025-09-26 17:30 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@kurilemu.de>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Fujii Masao <masao.fujii@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Thu, Sep 25, 2025 at 2:12 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
> So here's v22 with those and rebased to current sources. Only the first
> two patches this time, which are the ones I would be glad to receive
> input on.
>
A number of small issues I noticed. I don't know that they all need
addressing right now, but seems worth asking the questions...
#1
"pg_repackdb --help" does not mention the --index option, although the
flag is accepted. I'm not sure if this is meant to match clusterdb,
but since we need the index option to invoke the clustering behavior,
I think it needs to be there.
#2
[xzilla@zebes] pgsql/bin/pg_repackdb -d pagila -v -t customer
--index=idx_last_name
pg_repackdb: repacking database "pagila"
INFO: clustering "public.customer" using sequential scan and sort
[xzilla@zebes] pgsql/bin/pg_repackdb -d pagila -v -t customer
pg_repackdb: repacking database "pagila"
INFO: vacuuming "public.customer"
This was less confusing once I figured out we could pass the --index
option, but even with that it is a little confusing, I think mostly
because it looks like we are "vacuuming" the table, which in a world
of repack and vacuum (ie. no vacuum full) doesn't make sense. I think
the right thing to do here would be to modify it to be "repacking %s"
in both cases, with the "using sequential scan and sort" as the means
to understand which version of repack is being executed.
#3
pg_repackdb does not offer an --analyze option, which istm it should
to match the REPACK command
#4
SQL level REPACK help shows:
where option can be one of:
VERBOSE [ boolean ]
ANALYSE | ANALYZE
but SQL level VACUUM does
VERBOSE [ boolean ]
ANALYZE [ boolean ]
These operate the same way, so I would expect it to match the language
in vacuum.
#5
[xzilla@zebes] pgsql/bin/pg_repackdb -d pagila -v -t film --index
pg_repackdb: repacking database "pagila"
In the above scenario, I am repacking without having previously
specified an index. At the SQL level this would throw an error, at the
command line it gives me a heart attack. :-)
It's actually not that bad, because we don't actually do anything, but
maybe we should throw an error?
#6
On the individual command pages (like sql-repack.html), I think there
should be more cross-linking, ie. repack should probably say "see also
cluster" and vice versa. Likely similarly with vacuum and repack.
#7
Is there some reason you chose to intermingle the repack regression
tests with the existing tests? I feel like it'd be easier to
differentiate potential regressions and new functionality if these
were separated.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-10-10 14:11 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
6 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-10-10 14:11 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; +Cc: Antonin Houska <ah@cybertec.at>
Hello,
Here's patch v24. I was hoping to push this today, but I think there
were too many changes from v23 for that. Here's what I did:
- pg_stat_progress_cluster is no longer a view on top of the low-level
pg_stat_get_progress_info() function. Instead, it's a view on top of
pg_stat_progress_repack. The only change it applies on top of that
one is change the command from REPACK to one of VACUUM FULL or
CLUSTER, depending on whether an index is being used or not. This
should keep the behavior identical to previous versions.
Alternatively we could just hide rows where the command is REPACK, but
I don't think that would be any better. This way, we maintain
compatibility with tools reading pg_stat_progress_cluster. Maybe this
is useless and we should just drop the view, not sure, we can discuss
separately.
- pg_stat_progress_repack itself now shows the command. Also I got rid
of the separate enum values for the command, and instead used the
values from the parse node (RepackCommand); this removes about a dozen
lines of C code. To forestall potentially bogus usage of value 0, I
made the enum start from 1.
- I noticed that you can do "CLUSTER pg_class ON some_index" and it will
happily modify pg_index.indisclustered, which is a bit weird
considering that allow_system_table_mods is off -- if you later try
ALTER TABLE .. SET WITHOUT CLUSTER, it won't let you. I think this is
bogus and we should change it so that CLUSTER refuses to change the
clustered index on a system catalog, unless allow_system_table_mods is
on. However, that would be a change from longstanding behavior which
is specifically tested for in regression tests, so I didn't do it.
We can discuss such a change separately. But I did make REPACK refuse
to do that, because we don't need to propagate bogus historical
behavior. So REPACK will fail if you try to change the indisclustered
index, but it will work fine if you repack based on the same index as
before, or repack with no index.
- pg_repackdb: if you try with a non-superuser without specifying a
table name, it will fail as soon as it hits the first catalog table or
whatever with "ERROR: cannot lock this table". This is sorta fine for
vacuumdb, but only because VACUUM itself will instead say "WARNING:
cannot lock table XYZ, skipping", so it's not an error and vacuumdb
keeps running. IMO this is bogus: vacuumdb should not try to process
tables that it doesn't have privileges to. However, not wanting to
change longstanding behavior, I left that alone. For pg_repackdb, I
added a condition in the WHERE clause there to only fetch tables that
the current user has MAINTAIN privilege over. Then you can do a
"pg_repackdb -U foobar" and it will nicely process the tables that
that user is allowed to process. We can discuss changing the vacuumdb
behavior separately.
- Added some additional tests for pg_repackdb and REPACK.
- Updated the docs.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-10-30 23:17 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 12:42 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-11-01 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-11-05 02:48 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
6 siblings, 4 replies; 416+ messages in thread
From: Alvaro Herrera @ 2025-10-30 23:17 UTC (permalink / raw)
To: Pg Hackers <pgsql-hackers@lists.postgresql.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>
Hello,
Here's a new installment of this series, v25, including the CONCURRENTLY
part, which required some conflict fixes on top of the much-changed
v24-0001 patch.
After the talk on this subject for PGConf.EU, there were some
reservations about this whole project, and if I understand correctly,
they can be summarized in these three points:
1. Would the spill files for reorderbuffers occupy as much disk space as
it takes to copy the initial contents of the table, for each active
logical decoding replication slot? Antonin claims (I haven't verified
this) that there are some hacks in place to avoid this problem, or that
it is easy to install some -- and if so, then this patch would already
be better than pg_repack. This perhaps merits more testing.
2. Is the concurrent REPACK operation MVCC-safe? At the moment, with
the present implementation, no it is not. There are discussions on
getting this fixed, and Mihail has proposed some patches which at least
are quite short, though their safety is something we need to assess in
more depth.
3. Would the xmin horizon remain stuck at the spot where REPACK started,
thereby preventing VACUUM from cleaning up recently-dead rows in other
tables? As I understand, with the current implementation, yes it would,
and we cannot easily apply hacks such as PROC_IN_VACUUM to prevent it,
because it would introduce the same problems it did for CREATE INDEX
CONCURRENTLY that was fixed in pg14 (commit 042b584c7f7d62). Mihail and
Antonin have discussed possible ways to ease this, but we don't have
code for that yet. This is, again, no worse than VACUUM FULL or
CLUSTER, so lack of this wouldn't be a killer for this project, though
of course it would be much better to do better.
I have not yet addressed Robert Treat's feedback from October 12th.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Officer Krupke, what are we to do?
Gee, officer Krupke, Krup you! (West Side Story, "Gee, Officer Krupke")
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-11-01 12:42 ` jian he <jian.universality@gmail.com>
2025-11-01 12:53 ` Re: Adding REPACK [concurrently] Sergei Kornilov <sk@zsrv.org>
2025-12-04 13:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
3 siblings, 2 replies; 416+ messages in thread
From: jian he @ 2025-11-01 12:42 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>
On Fri, Oct 31, 2025 at 7:17 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Hello,
>
> Here's a new installment of this series, v25, including the CONCURRENTLY
> part, which required some conflict fixes on top of the much-changed
> v24-0001 patch.
>
hi.
if (params.options & CLUOPT_ANALYZE)
ereport(ERROR,
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot %s multiple tables", "REPACK (ANALYZE)"));
for this error case, adding a simple test case would be better?
+ /* Do an analyze, if requested */
+ if (params->options & CLUOPT_ANALYZE)
+ {
+ VacuumParams vac_params = {0};
+
+ vac_params.options |= VACOPT_ANALYZE;
+ if (params->options & CLUOPT_VERBOSE)
+ vac_params.options |= VACOPT_VERBOSE;
+ analyze_rel(RelationGetRelid(rel), NULL, vac_params,
+ stmt->relation->va_cols, true, NULL);
+ }
Looking at the comments in struct VacuumParams, some fields have nonzero default
values — for example, log_vacuum_min_duration.
Do we need to explicitly set these fields to their default values?
(see ExecVacuum)
repack.sgml can also add a
<refsect1> <title>See Also</title>
similar to analyze.sgml, vacuum.sgml
doc/src/sgml/ref/repack.sgml
synopsis section missing syntax:
REPACK USING INDEX
I am wondering, can we also support
REPACK opt_utility_option_list USING INDEX
MATERIALIZED VIEW:
create materialized view a_________ as select * from t2;
repack (verbose);
INFO: repacking "public.a_________" in physical order
INFO: "public.a_________": found 0 removable, 10 nonremovable row
versions in 1 pages
DETAIL: 0 dead row versions cannot be removed yet.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
cluster (verbose);
won't touch materialized view a_________
but materialized views don't have bloat, nothing can be removed.
So here we are waste cycles to repack materialized view?
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 12:42 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
@ 2025-11-01 12:53 ` Sergei Kornilov <sk@zsrv.org>
1 sibling, 0 replies; 416+ messages in thread
From: Sergei Kornilov @ 2025-11-01 12:53 UTC (permalink / raw)
To: jian he <jian.universality@gmail.com>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>; Alvaro Herrera <alvherre@alvh.no-ip.org>
Hello!
> but materialized views don't have bloat, nothing can be removed.
REFRESH MATERIALIZED VIEW CONCURRENTLY does not replace relation completely but updates the relation using insert and delete queries (refresh_by_match_merge in src/backend/commands/matview.c) - so there may be bloat.
regards, Sergei
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 12:42 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
@ 2025-12-04 13:36 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-12-04 13:36 UTC (permalink / raw)
To: jian he <jian.universality@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>
jian he <jian.universality@gmail.com> wrote:
> if (params.options & CLUOPT_ANALYZE)
> ereport(ERROR,
> errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> errmsg("cannot %s multiple tables", "REPACK (ANALYZE)"));
> for this error case, adding a simple test case would be better?
More options should probably be tested, currently we have only very basic
regression test for pg_repackdb. TBD
> + /* Do an analyze, if requested */
> + if (params->options & CLUOPT_ANALYZE)
> + {
> + VacuumParams vac_params = {0};
> +
> + vac_params.options |= VACOPT_ANALYZE;
> + if (params->options & CLUOPT_VERBOSE)
> + vac_params.options |= VACOPT_VERBOSE;
> + analyze_rel(RelationGetRelid(rel), NULL, vac_params,
> + stmt->relation->va_cols, true, NULL);
> + }
>
> Looking at the comments in struct VacuumParams, some fields have nonzero default
> values — for example, log_vacuum_min_duration.
> Do we need to explicitly set these fields to their default values?
> (see ExecVacuum)
Perhaps, TBD.
> repack.sgml can also add a
> <refsect1> <title>See Also</title>
> similar to analyze.sgml, vacuum.sgml
ok, added this in v26 (to be posted today):
<refsect1>
<title>See Also</title>
<simplelist type="inline">
<member><xref linkend="app-pgrepackdb"/></member>
<member><xref linkend="repack-progress-reporting"/></member>
</simplelist>
</refsect1>
(Not added reference to VACUUM FULL and CLUSTER intentionally: whoever uses
REPACK should not need them.
> doc/src/sgml/ref/repack.sgml
> synopsis section missing syntax:
> REPACK USING INDEX
ok, added in v26.
> I am wondering, can we also support
> REPACK opt_utility_option_list USING INDEX
I agree, added that in v26 (Hopefully I haven't broken anything, the syntax is
not trival anymore.)
> MATERIALIZED VIEW:
> create materialized view a_________ as select * from t2;
>
> repack (verbose);
> INFO: repacking "public.a_________" in physical order
> INFO: "public.a_________": found 0 removable, 10 nonremovable row
> versions in 1 pages
> DETAIL: 0 dead row versions cannot be removed yet.
> CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
>
> cluster (verbose);
> won't touch materialized view a_________
>
> but materialized views don't have bloat, nothing can be removed.
> So here we are waste cycles to repack materialized view?
Answered in https://www.postgresql.org/message-id/3436011762001613%40a7af8471-b1b8-48c2-9ff7-631187067407
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-11-01 18:16 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-11-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
3 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-11-01 18:16 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Robert Treat <rob@xzilla.net>
Hello!
On Fri, Oct 31, 2025 at 12:17 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> Here's a new installment of this series, v25, including the CONCURRENTLY
> part, which required some conflict fixes on top of the much-changed
> v24-0001 patch.
> * cluster.c
> * CLUSTER a table on an index. This is now also used for VACUUM FULL.
Should we add something about repack here?
> ii_ExclusinOps
typo here.
> * index is inserted into catalogs and needs to be built later on.
Now it is only in case concurrently == true
> * Build the index information for the new index. Note that rebuild of
> * indexes with exclusion constraints is not supported, hence there is no
> * need to fill all the ii_Exclusion* fields.
Now the function supports its in !concurrently mode. Should we fill
ii_Exclusion? Also, it says
> If !concurrently, ii_ExclusinOps is currently not needed.
But it is not clear - why not?
> newInfo = makeIndexInfo(oldInfo->ii_NumIndexAttrs,
> oldInfo->ii_NumIndexKeyAttrs,
> oldInfo->ii_Am,
> indexExprs,
> indexPreds,
> oldInfo->ii_Unique,
> oldInfo->ii_NullsNotDistinct,
> false, /* not ready for inserts */
> true,
> indexRelation->rd_indam->amsummarizing,
> oldInfo->ii_WithoutOverlaps);
Is it ok we pass isready == false if !concurrent?
Also, we pass concurrent == true even if concurrently == false - feels
strange and probably wrong.
> This difference does has no impact on XidInMVCCSnapshot().
Should it be "This difference has no impact"?
> * pgoutput_cluster.c
> * src/backend/replication/pgoutput_cluster/pgoutput_cluster.c
it is pgoutput_trepack.c :)
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-11-03 07:56 ` Antonin Houska <ah@cybertec.at>
2025-12-02 00:50 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-11-03 07:56 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hello!
>
> On Fri, Oct 31, 2025 at 12:17 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > Here's a new installment of this series, v25, including the CONCURRENTLY
> > part, which required some conflict fixes on top of the much-changed
> > v24-0001 patch.
>
> > * cluster.c
> > * CLUSTER a table on an index. This is now also used for VACUUM FULL.
ok
> Should we add something about repack here?
>
> > ii_ExclusinOps
> typo here.
ok
> > * index is inserted into catalogs and needs to be built later on.
> Now it is only in case concurrently == true
ok
> > * Build the index information for the new index. Note that rebuild of
> > * indexes with exclusion constraints is not supported, hence there is no
> > * need to fill all the ii_Exclusion* fields.
>
> Now the function supports its in !concurrently mode. Should we fill
> ii_Exclusion? Also, it says
>
> > If !concurrently, ii_ExclusinOps is currently not needed.
> But it is not clear - why not?
Right, makeIndexInfo() needs to be adjusted.
>
> > newInfo = makeIndexInfo(oldInfo->ii_NumIndexAttrs,
> > oldInfo->ii_NumIndexKeyAttrs,
> > oldInfo->ii_Am,
> > indexExprs,
> > indexPreds,
> > oldInfo->ii_Unique,
> > oldInfo->ii_NullsNotDistinct,
> > false, /* not ready for inserts */
> > true,
> > indexRelation->rd_indam->amsummarizing,
> > oldInfo->ii_WithoutOverlaps);
>
> Is it ok we pass isready == false if !concurrent?
> Also, we pass concurrent == true even if concurrently == false - feels
> strange and probably wrong.
You're right, both arguments are wrong.
> > This difference does has no impact on XidInMVCCSnapshot().
> Should it be "This difference has no impact"?
ok
> > * pgoutput_cluster.c
> > * src/backend/replication/pgoutput_cluster/pgoutput_cluster.c
> it is pgoutput_trepack.c :)
ok
I'll fix all the problems in the next version. Thanks!
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-11-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-02 00:50 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-02 16:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-02 00:50 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
On Mon, Nov 3, 2025 at 8:56 AM Antonin Houska <ah@cybertec.at> wrote:
> I'll fix all the problems in the next version. Thanks!
A few more moments I mentioned:
> switch ((vis = HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf)))
vis is unused, also to double braces.
> LockBuffer(buf, BUFFER_LOCK_UNLOCK);
> continue;
> }
> /*
> * In the concurrent case, we have a copy of the tuple, so we
> * don't worry whether the source tuple will be deleted / updated
> * after we release the lock.
> */
> LockBuffer(buf, BUFFER_LOCK_UNLOCK);
>}
I think locking and comments are a little bit confusing here.
I think we may use single LockBuffer(buf, BUFFER_LOCK_UNLOCK); before
`if (isdead)` as it was before.
Also, I am not sure "we have a copy" is the correct point here, I
think motivation is mostly the same as in
heapam_index_build_range_scan.
Also, I think it is a good idea to add tests for index-based and
sort-based repack.
Also, for sort-based I think we need to also call
repack_decode_concurrent_changes during insertion phase
> is_system_catalog && !concurrent
2 places, always true, feels strange.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-11-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-02 00:50 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-02 16:14 ` Antonin Houska <ah@cybertec.at>
2025-12-02 16:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-12-02 16:14 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hello, Antonin!
>
> On Mon, Nov 3, 2025 at 8:56 AM Antonin Houska <ah@cybertec.at> wrote:
> > I'll fix all the problems in the next version. Thanks!
>
> A few more moments I mentioned:
>
> > switch ((vis = HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf)))
> vis is unused, also to double braces.
>
> > LockBuffer(buf, BUFFER_LOCK_UNLOCK);
> > continue;
> > }
>
> > /*
> > * In the concurrent case, we have a copy of the tuple, so we
> > * don't worry whether the source tuple will be deleted / updated
> > * after we release the lock.
> > */
> > LockBuffer(buf, BUFFER_LOCK_UNLOCK);
> >}
>
> I think locking and comments are a little bit confusing here.
> I think we may use single LockBuffer(buf, BUFFER_LOCK_UNLOCK); before
> `if (isdead)` as it was before.
> Also, I am not sure "we have a copy" is the correct point here, I
> think motivation is mostly the same as in
> heapam_index_build_range_scan.
All these problems are due to incorrect separation of the "preserve
visibility" part of the patch series. Will be fixed in the next version.
> Also, I think it is a good idea to add tests for index-based and
> sort-based repack.
Not sure, cluster.sql already seems to do the same.
> Also, for sort-based I think we need to also call
> repack_decode_concurrent_changes during insertion phase
I miss the point. The current coding is such that this part
if (concurrent)
{
XLogRecPtr end_of_wal;
end_of_wal = GetFlushRecPtr(NULL);
if ((end_of_wal - end_of_wal_prev) > wal_segment_size)
{
repack_decode_concurrent_changes(decoding_ctx, end_of_wal);
end_of_wal_prev = end_of_wal;
}
}
gets called regardless the value of 'tuplesort' above.
> > is_system_catalog && !concurrent
> 2 places, always true, feels strange.
ok
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-11-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-02 00:50 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-02 16:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-02 16:22 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-02 16:22 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi!
On Tue, Dec 2, 2025 at 5:14 PM Antonin Houska <ah@cybertec.at> wrote:
> Not sure, cluster.sql already seems to do the same.
I think in the case of CONCURRENTLY it may behave a little bit
different, but not sure.
> I miss the point. The current coding is such that this part
I mean call it periodically in both loops: scan loop and insertion loop.
Best greetings,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-11-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-02 00:50 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-02 16:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-02 16:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-03 07:56 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-12-03 07:56 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> On Tue, Dec 2, 2025 at 5:14 PM Antonin Houska <ah@cybertec.at> wrote:
> > Not sure, cluster.sql already seems to do the same.
> I think in the case of CONCURRENTLY it may behave a little bit
> different, but not sure.
>
> > I miss the point. The current coding is such that this part
> I mean call it periodically in both loops: scan loop and insertion loop.
ok, that makes sense. I'll add that to the next version. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-11-05 02:48 ` jian he <jian.universality@gmail.com>
2025-11-05 05:10 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
3 siblings, 1 reply; 416+ messages in thread
From: jian he @ 2025-11-05 02:48 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>
On Fri, Oct 31, 2025 at 7:17 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Hello,
>
> Here's a new installment of this series, v25, including the CONCURRENTLY
> part, which required some conflict fixes on top of the much-changed
> v24-0001 patch.
>
<refnamediv>
<refname>pg_repackdb</refname>
<refpurpose>repack and analyze a <productname>PostgreSQL</productname>
database</refpurpose>
</refnamediv>
but with --all option specified, it's doing repack whole cluster.
(more than one database).
I am not fully sure this description is OK.
I think pg_repackdb Synopsis section:
pg_repackdb [connection-option...] [option...] [ -t | --table table [(
column [,...] )] ] ... [ dbname | -a | --all ]
pg_repackdb [connection-option...] [option...] [ -n | --schema schema
] ... [ dbname | -a | --all ]
pg_repackdb [connection-option...] [option...] [ -N | --exclude-schema
schema ] ... [ dbname | -a | --all ]
can be simplified the same way as as pg_dump:
pg_repackdb [connection-option...] [option...] [ dbname | -a | --all ]
------------------------
[-d] dbname
[--dbname=]dbname
what do you think to expand it as below:
dbname
-d dbname
--dbname=dbname
--------------------
+ printf(_(" --index[=INDEX] repack following an index\n"));
should it be
+ printf(_("--index[=INDEX] repack following an index\n"));
?
similar to pg_dump:
printf(_("\nIf no database name is supplied, then the PGDATABASE
environment\n"
"variable value is used.\n\n"));
in pg_repackdb help section, we can mention:
printf(_("\nIf no database name is supplied and --all option not
specified then the PGDATABASE environment\n"
"variable value is used.\n\n"));
Do you think it's necessary?
what the expectation of
pg_repackdb --index=index_name, the doc is not very helpful.
pg_repackdb --analyze --index=zz --verbose
pg_repackdb: repacking database "src3"
pg_repackdb: error: processing of database "src3" failed: ERROR: "zz"
is not an index for table "tenk1"
select pg_get_indexdef ('zz'::regclass);
pg_get_indexdef
---------------------------------------------------
CREATE INDEX zz ON public.tenk2 USING btree (two)
------
jian he
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-05 02:48 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
@ 2025-11-05 05:10 ` Robert Treat <rob@xzilla.net>
2025-11-05 07:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-11-05 08:46 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Robert Treat @ 2025-11-05 05:10 UTC (permalink / raw)
To: jian he <jian.universality@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Tue, Nov 4, 2025 at 9:48 PM jian he <jian.universality@gmail.com> wrote:
> On Fri, Oct 31, 2025 at 7:17 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > Hello,
> >
> > Here's a new installment of this series, v25, including the CONCURRENTLY
> > part, which required some conflict fixes on top of the much-changed
> > v24-0001 patch.
> >
>
> <refnamediv>
> <refname>pg_repackdb</refname>
> <refpurpose>repack and analyze a <productname>PostgreSQL</productname>
> database</refpurpose>
> </refnamediv>
>
> but with --all option specified, it's doing repack whole cluster.
> (more than one database).
> I am not fully sure this description is OK.
>
This wording came from vacuumdb, which operates the same way, and I
don't think it's lead to confusion. And while I don't think we need to
take away the option, I see no reason to encourage the idea that
people should be doing cluster wide full database repacks. On that
note, I'd take the "and analyze" from the refpurpose as well; the more
I look at it, I see pg_repackdb as a replacement for clusterdb, with
selected bells and whistles from vacuum full or external repack-type
tooling, but at the end of the day that's a simpler model for
operators, and helps draw a distinction for which features we DONT
need to include, like -Z (ie. analyze only; if you want that, use
vacuumdb, not pg_repackdb)
>
> I think pg_repackdb Synopsis section:
> pg_repackdb [connection-option...] [option...] [ -t | --table table [(
> column [,...] )] ] ... [ dbname | -a | --all ]
> pg_repackdb [connection-option...] [option...] [ -n | --schema schema
> ] ... [ dbname | -a | --all ]
> pg_repackdb [connection-option...] [option...] [ -N | --exclude-schema
> schema ] ... [ dbname | -a | --all ]
>
> can be simplified the same way as as pg_dump:
>
> pg_repackdb [connection-option...] [option...] [ dbname | -a | --all ]
>
I think it's laid out that way in vacuumdb to indicate that those
options are exclusive of one another. I'm not sure how convincing that
is, but the above would need to do more to make the switch imo.
> ------------------------
> [-d] dbname
> [--dbname=]dbname
>
> what do you think to expand it as below:
> dbname
> -d dbname
> --dbname=dbname
not sure i am following this one, but the brackets are the standard
way we should items to be optional, which in either case they are.
> --------------------
>
> + printf(_(" --index[=INDEX] repack following an index\n"));
> should it be
> + printf(_("--index[=INDEX] repack following an index\n"));
> ?
>
I believe this is included for alignment, since this option has no
shorthand version.
>
> similar to pg_dump:
> printf(_("\nIf no database name is supplied, then the PGDATABASE
> environment\n"
> "variable value is used.\n\n"));
>
> in pg_repackdb help section, we can mention:
> printf(_("\nIf no database name is supplied and --all option not
> specified then the PGDATABASE environment\n"
> "variable value is used.\n\n"));
> Do you think it's necessary?
>
no. (again, looking first at clusterdb, and also vacuumdb, neither of
which have it).
>
> what the expectation of
> pg_repackdb --index=index_name, the doc is not very helpful.
>
> pg_repackdb --analyze --index=zz --verbose
> pg_repackdb: repacking database "src3"
> pg_repackdb: error: processing of database "src3" failed: ERROR: "zz"
> is not an index for table "tenk1"
>
> select pg_get_indexdef ('zz'::regclass);
> pg_get_indexdef
> ---------------------------------------------------
> CREATE INDEX zz ON public.tenk2 USING btree (two)
>
Hmm... yes, this is a bit confusing. I didn't verify it in the code,
but from memory I think the --index option is meant to be used only in
conjunction with --table, in which case it would repack the table
using the specified index. I could be overlooking something though.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-05 02:48 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-11-05 05:10 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-11-05 07:12 ` Antonin Houska <ah@cybertec.at>
2025-11-09 22:13 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-11-05 07:12 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: jian he <jian.universality@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Robert Treat <rob@xzilla.net> wrote:
> On Tue, Nov 4, 2025 at 9:48 PM jian he <jian.universality@gmail.com> wrote:
> > what the expectation of
> > pg_repackdb --index=index_name, the doc is not very helpful.
> >
> > pg_repackdb --analyze --index=zz --verbose
> > pg_repackdb: repacking database "src3"
> > pg_repackdb: error: processing of database "src3" failed: ERROR: "zz"
> > is not an index for table "tenk1"
> >
> > select pg_get_indexdef ('zz'::regclass);
> > pg_get_indexdef
> > ---------------------------------------------------
> > CREATE INDEX zz ON public.tenk2 USING btree (two)
> >
>
> Hmm... yes, this is a bit confusing. I didn't verify it in the code,
> but from memory I think the --index option is meant to be used only in
> conjunction with --table, in which case it would repack the table
> using the specified index. I could be overlooking something though.
The corresponding code is:
+ /*
+ * In REPACK mode, if the 'using_index' option was given but no index
+ * name, filter only tables that have an index with indisclustered set.
+ * (If an index name is given, we trust the user to pass a reasonable list
+ * of tables.)
+ *
+ * XXX it may be worth printing an error if an index name is given with no
+ * list of tables.
+ */
+ if (vacopts->mode == MODE_REPACK &&
+ vacopts->using_index && !vacopts->indexname)
+ {
+ appendPQExpBufferStr(&catalog_query,
+ " AND EXISTS (SELECT 1 FROM pg_catalog.pg_index\n"
+ " WHERE indrelid = c.oid AND indisclustered)\n");
+ }
I'm not sure if it's worth allowing the --index option to have an
argument. Since the user can specify multiple tables, he should also be able
to specify multiple indexes. And then the question would be: what should
happen if the user forgot to specify (or just mistyped) the index name for a
table which does not yet have the clustering index set? Skip that table (and
print out a warning)? Or consider it an error?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-05 02:48 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-11-05 05:10 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-11-05 07:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-11-09 22:13 ` Robert Treat <rob@xzilla.net>
0 siblings, 0 replies; 416+ messages in thread
From: Robert Treat @ 2025-11-09 22:13 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: jian he <jian.universality@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Wed, Nov 5, 2025 at 2:12 AM Antonin Houska <ah@cybertec.at> wrote:
> Robert Treat <rob@xzilla.net> wrote:
> > On Tue, Nov 4, 2025 at 9:48 PM jian he <jian.universality@gmail.com> wrote:
>
> > > what the expectation of
> > > pg_repackdb --index=index_name, the doc is not very helpful.
> > >
> > > pg_repackdb --analyze --index=zz --verbose
> > > pg_repackdb: repacking database "src3"
> > > pg_repackdb: error: processing of database "src3" failed: ERROR: "zz"
> > > is not an index for table "tenk1"
> > >
> > > select pg_get_indexdef ('zz'::regclass);
> > > pg_get_indexdef
> > > ---------------------------------------------------
> > > CREATE INDEX zz ON public.tenk2 USING btree (two)
> > >
> >
> > Hmm... yes, this is a bit confusing. I didn't verify it in the code,
> > but from memory I think the --index option is meant to be used only in
> > conjunction with --table, in which case it would repack the table
> > using the specified index. I could be overlooking something though.
>
> The corresponding code is:
>
> + /*
> + * In REPACK mode, if the 'using_index' option was given but no index
> + * name, filter only tables that have an index with indisclustered set.
> + * (If an index name is given, we trust the user to pass a reasonable list
> + * of tables.)
> + *
> + * XXX it may be worth printing an error if an index name is given with no
> + * list of tables.
> + */
> + if (vacopts->mode == MODE_REPACK &&
> + vacopts->using_index && !vacopts->indexname)
> + {
> + appendPQExpBufferStr(&catalog_query,
> + " AND EXISTS (SELECT 1 FROM pg_catalog.pg_index\n"
> + " WHERE indrelid = c.oid AND indisclustered)\n");
> + }
>
> I'm not sure if it's worth allowing the --index option to have an
> argument. Since the user can specify multiple tables, he should also be able
> to specify multiple indexes. And then the question would be: what should
> happen if the user forgot to specify (or just mistyped) the index name for a
> table which does not yet have the clustering index set? Skip that table (and
> print out a warning)? Or consider it an error?
>
Ah, yes, this is something completely different. So, we do need a way
to differentiate between "vacuum full" vs "cluster" all tables... as
well as "vacuum full" vs "cluster" of a specific table (including the
idea of "vacuum full" of a previously clustered table, and the
existing code handles all that (though I might quibble with the option
name).
As for having an --index= option, I'd love to hear the use case;
something like partitions or maybe some client per schema situation
comes to mind, but ISTM in all those cases the user would also know
(or be expected to know) the table name, so I agree with Antonin that
the extra complexity doesn't seem worth supporting to me. (It's even
worse the more you think about it, what if some table has the index
named above, but is clustered on a different index, then what should
we do?)
As for the use case I was thinking of, specifying a table and index in
order to repack using that index (and setting indisclustered if not
already); while I feel like that would be a useful option, if it isn't
currently supported I don't see a strong argument for adding it now.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-05 02:48 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-11-05 05:10 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2025-11-05 08:46 ` jian he <jian.universality@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: jian he @ 2025-11-05 08:46 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>
On Wed, Nov 5, 2025 at 1:11 PM Robert Treat <rob@xzilla.net> wrote:
> > --------------------
> >
> > + printf(_(" --index[=INDEX] repack following an index\n"));
> > should it be
> > + printf(_("--index[=INDEX] repack following an index\n"));
> > ?
> >
>
> I believe this is included for alignment, since this option has no
> shorthand version.
>
if you compare pg_dump --help, pg_repackdb --help
then you will see the inconsistency.
This is legacy behavior, but can we move some of the error checks in
do_analyze_rel to an earlier point?
we call cluster_rel before analyze_rel, cluster_rel is way more time-consuming,
a failure in analyze_rel means all the previous work (cluster_rel) is wasted.
+ else if (HeadMatches("REPACK", "(*") &&
+ !HeadMatches("REPACK", "(*)"))
+ {
+ /*
+ * This fires if we're in an unfinished parenthesized option list.
+ * get_previous_words treats a completed parenthesized option list as
+ * one word, so the above test is correct.
+ */
+ if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
+ COMPLETE_WITH("VERBOSE");
+ else if (TailMatches("VERBOSE"))
+ COMPLETE_WITH("ON", "OFF");
+ }
this part can also support the ANALYZE option?
ClusterStmt
should be removed from src/tools/pgindent/typedefs.list?
doc/src/sgml/ref/clusterdb.sgml
<para>
<application>clusterdb</application> has been superceded by
<application>pg_repackdb</application>.
</para>
google told me, "superceded" should be "superseded"
--
jian he
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-12-04 17:43 ` Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
3 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-12-04 17:43 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> Here's a new installment of this series, v25, including the CONCURRENTLY
> part, which required some conflict fixes on top of the much-changed
> v24-0001 patch.
v26 attached here. It's been rebased and reflects most of the feedback.
A few incomplete items are marked as TBD here [1] and [2] is another thing
that needs discussion.
Besides that, I've done some refactoring in 0004: 1) move more code to
setup_logical_decoding(), and 2) reduced the number of arguments of
process_concurrent_changes() by using a new structure. Both these changes are
a preparation for a background worker that will perform the logical decoding,
but seem to be useful as such. (I have a PoC of the worker but will post it
later, it doesn't seem to be the priority for now.)
I've also removed support for decoding TRUNCATE because I realized that this
command uses AccessExclusiveLock, so it cannot be executed on a table that
REPACK (CONCURRENTLY) is just processing.
Also I tried to fix TAB completion in psql.
> I have not yet addressed Robert Treat's feedback from October 12th.
These are still pending.
[1] https://www.postgresql.org/message-id/23631.1764855372%40localhost
[2] https://www.postgresql.org/message-id/CAJSLCQ2_jX8WmNOC4eu6hL5QyNHceOkgPbGhKHFw2X5onVEKDQ%40mail.gma...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-05 00:03 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-08 07:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-05 00:03 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
On Thu, Dec 4, 2025 at 6:43 PM Antonin Houska <ah@cybertec.at> wrote:
> v26 attached here. It's been rebased and reflects most of the feedback.
Some comments on 0001-0002:
1)
> cluster_rel(stmt->command, rel, indexOid, params);
cluster_rel closes relation, and after it is dereferenced a few lines after.
Technically it may be correct, but feels a little bit strange.
2)
> if (vacopts->mode == MODE_VACUUM)
I think for better compatibility it is better to handle new value in
if - (vacopts->mode == MODE_REPACK) to keep old cases unchanged
3)
> case T_RepackStmt:
> tag = CMDTAG_REPACK;
> break;
should we use instead:
case T_RepackStmt:
if (((RepackStmt *) parsetree)->command == REPACK_COMMAND_CLUSTER)
tag = CMDTAG_CLUSTER;
else
tag = CMDTAG_REPACK;
break;
or delete CMDTAG_CLUSTER - since it not used anymore
4)
"has been superceded by"
typo
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-06 18:16 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-08 09:51 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-06 18:16 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
Some comments for 0003:
> /* allocate in transaction context */
It may be any context now, because it is a function now.
> result = CopySnapshot(snapshot);
> /* Restore the original values so the source is intact. */
> snapshot->xip = oldxip;
> snapshot->xcnt = oldxcnt;
I think it is worth to call pfree(newxip) here.
> "This difference does has no impact"
should be "This difference has no impact"?
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-07 16:03 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-07 16:03 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, comments so far on 0004:
---
> ind_oids_new = build_new_indexes(NewHeap, OldHeap, ind_oids_old);
I think the biggest issue we have so far -
repack_decode_concurrent_changes is not called while new indexes are
built (the build itself creates a huge amount of WAL and takes days
sometimes). Looks like a way to catastrophic scenarios :)
Some small parts of it may be related to reset snapshots tech in CIC case:
1) if we build new indexes concurrently in REPACK case
2) and reset snapshots every so often
3) we may use the same callback to also process WAL every so often
4) but it still not applies to some phases of index building (batch
insertion phase, for example)
Or should we move repack_decode_concurrent_changes calls into some
kind of worker instead?
---
> if (OldHeap->rd_rel->reltoastrelid)
> LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
I think we should pass mode from rebuild_relation here - because
AccessExclusiveLock will break "CONCURRENTLY" totally.
And also upgrade before swap probably.
---
> cluster_is_permitted_for_relation(RepackCommand cmd, Oid relid, Oid userid)
Should be check CheckSlotPermissions(); here? Aso, maybe it is worth
mentioning in docs.
---
> REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
Some paths (without index) are not covered in any way in tests at the moment.
Also, I think some TOAST-related scenarios too.
> * Alternatively, we can lock all the indexes now in a mode that blocks
> * all the ALTER INDEX commands (ShareUpdateExclusiveLock ?), and keep
I think it's better to lock.
---
> rebuild_relation(RepackCommand cmd, Relation OldHeap, Relation index,
"cmd" is not used.
---
> apply_concurrent_update
> apply_concurrent_delete
> apply_concurrent_insert
"change" is not used, but I think it is intentionally for the MVCC-safe case.
---
> rebuild_relation(RepackCommand cmd, Relation OldHeap, Relation index,
> bool verbose, bool concurrent)
"concurrent" is "concurrently" in definition.
---
> TM_FailureData *tmfd, bool changingPart,
> bool wal_logical);
Maybe "walLogical" to keep it aligned with "changingPart"?
---
> subtransacion
typo
---
> Should we check a the end
"a" is "at"?
---
> Note that <command>REPACK</command> with the
> the <literal>CONCURRENTLY</literal> option does not try to order the
double "the"
---
> if (size >= 0x3FFFFFFF)
if (size >= MaxAllocSize)
---
> extern bool HeapTupleMVCCInserted(HeapTuple htup, Snapshot snapshot,
> Buffer buffer);
> extern bool HeapTupleMVCCNotDeleted(HeapTuple htup, Snapshot snapshot,
> Buffer buffer);
Looks like this from another patch.
---
src/backend/utils/cache/relcache.c
> #include "commands/cluster.h"
may be removed
---
> during any of the preceding
> phase.
"phases"
---
> # Prefix the system columns with underscore as they are not allowed as column
> # names.
Should it be removed?
---
> "Failed to find target tuple"
This and multiple other new error messages should start with lowercase
---
> Copyright (c) 2012-2024, PostgreSQL Global Development Group
in pgoutput_repack - maybe it is time to adjust.
---
src/test/modules/injection_points/logical.conf
Better to add newline
---
> SELECT injection_points_detach('repack-concurrently-before-lock');
Uses spaces, need to be tabs.
Next step in my plan - rebase MVCC-safe commit and test it with some
amount of stress tests.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-09 18:52 ` Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2025-12-09 18:52 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hello, comments so far on 0004:
>
> ---
> > ind_oids_new = build_new_indexes(NewHeap, OldHeap, ind_oids_old);
>
> I think the biggest issue we have so far -
> repack_decode_concurrent_changes is not called while new indexes are
> built (the build itself creates a huge amount of WAL and takes days
> sometimes). Looks like a way to catastrophic scenarios :)
Indeed, that may be a problem.
> Some small parts of it may be related to reset snapshots tech in CIC case:
> 1) if we build new indexes concurrently in REPACK case
> 2) and reset snapshots every so often
> 3) we may use the same callback to also process WAL every so often
> 4) but it still not applies to some phases of index building (batch
> insertion phase, for example)
I prefer not to depend on other improvements.
> Or should we move repack_decode_concurrent_changes calls into some
> kind of worker instead?
Worker makes more sense to me - the initial implementation is in 0005.
> ---
> > if (OldHeap->rd_rel->reltoastrelid)
> > LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
>
> I think we should pass mode from rebuild_relation here - because
> AccessExclusiveLock will break "CONCURRENTLY" totally.
Good point, I missed this.
> And also upgrade before swap probably.
rebuild_relation_finish_concurrent() already does that.
> ---
> > cluster_is_permitted_for_relation(RepackCommand cmd, Oid relid, Oid userid)
>
> Should be check CheckSlotPermissions(); here? Aso, maybe it is worth
> mentioning in docs.
setup_logical_decoding() does that, but I'm not sure if we should really
require the REPLICATION user attribute for REPACK. I need to think about this,
perhaps ACL_MAINTAIN is enough.
> ---
> > REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
>
> Some paths (without index) are not covered in any way in tests at the moment.
> Also, I think some TOAST-related scenarios too.
I added test for TOAST to "injection_points" and hit a serious problem: when
applying concurrent changes to the new table, REPACK tried to delete rows from
the new one. The point is that the "swap TOAST by content" technique cannot be
used here. Fixed, thanks for this suggestion!
> > * Alternatively, we can lock all the indexes now in a mode that blocks
> > * all the ALTER INDEX commands (ShareUpdateExclusiveLock ?), and keep
>
> I think it's better to lock.
ok, changed
> ---
> > rebuild_relation(RepackCommand cmd, Relation OldHeap, Relation index,
>
> "cmd" is not used.
Fixed (not specific to 0004).
> ---
> > apply_concurrent_update
> > apply_concurrent_delete
> > apply_concurrent_insert
>
> "change" is not used, but I think it is intentionally for the MVCC-safe case.
Not sure if it's necessary for the MVCC-safe case, I consider it leftover from
some previous version. Removed.
> ---
> > rebuild_relation(RepackCommand cmd, Relation OldHeap, Relation index,
> > bool verbose, bool concurrent)
>
> "concurrent" is "concurrently" in definition.
Fixed.
> ---
>
> > TM_FailureData *tmfd, bool changingPart,
> > bool wal_logical);
> Maybe "walLogical" to keep it aligned with "changingPart"?
ok
> ---
> > subtransacion
> typo
>
I removed the related code. It was a workaround for plan_cluster_use_sort()
not to leave locks behind. However, as REPACK (CONCURRENTLY) does not unlock
the relation anymore, this is not needed as well.
> ---
> > Should we check a the end
>
> "a" is "at"?
Removed when addressing one of the previous comments.
>
> ---
> > Note that <command>REPACK</command> with the
> > the <literal>CONCURRENTLY</literal> option does not try to order the
>
> double "the"
Fixed.
> ---
> > if (size >= 0x3FFFFFFF)
> if (size >= MaxAllocSize)
Fixed.
> ---
> > extern bool HeapTupleMVCCInserted(HeapTuple htup, Snapshot snapshot,
> > Buffer buffer);
> > extern bool HeapTupleMVCCNotDeleted(HeapTuple htup, Snapshot snapshot,
> > Buffer buffer);
>
> Looks like this from another patch.
Right, this is from the "MVCC safety part".
> ---
> src/backend/utils/cache/relcache.c
> > #include "commands/cluster.h"
>
> may be removed
Yes, this belongs to some of the following patches of the series.
> ---
> > during any of the preceding
> > phase.
>
> "phases"
Fixed.
> ---
> > # Prefix the system columns with underscore as they are not allowed as column
> > # names.
>
> Should it be removed?
Done. (Belongs to the "MVCC-safety" part, where the test check xmin, xmax,
...)
> ---
> > "Failed to find target tuple"
>
> This and multiple other new error messages should start with lowercase
Fixed.
> ---
> > Copyright (c) 2012-2024, PostgreSQL Global Development Group
>
> in pgoutput_repack - maybe it is time to adjust.
Done.
> ---
> src/test/modules/injection_points/logical.conf
>
> Better to add newline
Done.
> ---
> > SELECT injection_points_detach('repack-concurrently-before-lock');
>
> Uses spaces, need to be tabs.
ok
Thanks for the review!
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-09 19:22 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2025-12-09 19:22 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, many thanks for the new version. Here's a very quick proposal
for a new top-of-file comment on cluster.c,
* cluster.c
* Implementation of REPACK [CONCURRENTLY], also known as CLUSTER and
* VACUUM FULL.
*
* There are two somewhat different ways to rewrite a table. In non-
* concurrent mode, it's easy: take AccessExclusiveLock, create a new
* transient relation, copy the tuples over to the relfilenode of the
* new relation, swap the relfilenodes, then drop the old relation.
*
* In concurrent mode, we lock the table with only ShareUpdateExclusiveLock,
* then do an initial copy as above. However, while the tuples are being
* copied, concurrent transactions could modify the table, and to cope
* with those changes, we rely on logical decoding to obtain them from WAL.
* A bgworker consumes WAL while the initial copy is ongoing (to prevent
* excessive WAL from being reserved), and accumulates the changes in
* a tuplestore. Once the initial copy is complete, we read the changes
* from the tuplestore and re-apply them on the new heap. Then we
* upgrade our ShareUpdateExclusiveLock to AccessExclusiveLock and swap
* the relfilenodes. This way, the time we hold a strong lock on the
* table is much reduced, and the bloat is greatly reduced.
I haven't read build_relation_finish_concurrent() yet to understand how
exactly do we do the lock upgrade, which I think is an important point
we should address in this comment. Also not addressed is how exactly we
handle indexes. Feel free to correct this, reword it or include any
additional details that you think are important.
(At this point we could just as well rename the file to repack.c, since
very little of the original remains. But let's discuss that later.)
Thanks,
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Doing what he did amounts to sticking his fingers under the hood of the
implementation; if he gets his fingers burnt, it's his problem." (Tom Lane)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-12-13 18:48 ` Antonin Houska <ah@cybertec.at>
2025-12-13 19:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2025-12-13 18:48 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> Hello, many thanks for the new version. Here's a very quick proposal
> for a new top-of-file comment on cluster.c,
The comment matches 0005, but I had to adjust it for 0004 (no background
worker there). Also, the worker writes the changes to a file rather than
tuplestore (storage/sharedfileset.h seems to me an easier way to pass the data
from one process to another) Besides that I made the following changes:
"bloat is greatly reduced" -> "bloat is eliminated"
and
"table, and to cope with" -> "table. To cope with"
> I haven't read build_relation_finish_concurrent() yet to understand how
> exactly do we do the lock upgrade, which I think is an important point
> we should address in this comment. Also not addressed is how exactly we
> handle indexes. Feel free to correct this, reword it or include any
> additional details that you think are important.
ok, I'll get back to the earlier parts of the set, including this, in the
beginning of January. Regarding indexes, one thing I've noticed recently that
they get locked in build_new_indexes(), but maybe it should happen earlier.
> (At this point we could just as well rename the file to repack.c, since
> very little of the original remains. But let's discuss that later.)
ok. Do you mean only the file or the functions as well? (I'm not going to do
that now, w/o that discussion.)
Attached here is a new version of the patch set. Its rebased and extended one
more time: 0006 is a PoC of the "snapshot resetting" technique, as discussed
elsewhere with Mihail Nikalayeu and Matthias van de Meent. The way snapshot
are generated here is different though: we need the snapshots from logical
replication's snapbuild.c, not those from procarray.c. More information is in
the commit message.
I do not insist that this should go to PG 19, just needed some confidence that
it's doable, as well as some feedback. There are no tests for this yet, but
I've played with it for a while and checked the behavior using debugger. I'm
curious to hear if the design is sound.
While working on that, I fixed some problems in 0004 and 0005 too. It
shouldn't be difficult to identify them using git, if needed.
Even if 0005 and 0006 won't land in PG19, these parts show that some
refactoring may be needed regarding the AM callback
table_relation_copy_for_cluster(). The parts 0004, 0005 and 0006 each change
the argument list. It wouldn't be perfect if both PG 19 and 20 changed the
API. I think we should reconsider which arguments are generic and which are
rather AM-specific. Maybe we should then add an opaque pointer (void *) for
the AM-specific information. REPACK could then use it to pass the
CONCURRENTLY-specific information.
I'm now going to prioritize the parts <= 0004.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-13 19:01 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-13 19:01 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
On Sat, Dec 13, 2025 at 7:48 PM Antonin Houska <ah@cybertec.at> wrote:
> Attached here is a new version of the patch set. Its rebased and extended one
> more time: 0006 is a PoC of the "snapshot resetting" technique, as discussed
> elsewhere with Mihail Nikalayeu and Matthias van de Meent. The way snapshot
> are generated here is different though: we need the snapshots from logical
> replication's snapbuild.c, not those from procarray.c. More information is in
> the commit message.
Have you seen my feedback for 0004? Do you plan to check it? Asking to
understand if it is worth reviewing now or later.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-15 14:25 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2025-12-15 14:25 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2025-Dec-13, Antonin Houska wrote:
> From 6279394135f2b693b6fffd174822509e0a067cbf Mon Sep 17 00:00:00 2001
> From: Antonin Houska <ah@cybertec.at>
> Date: Sat, 13 Dec 2025 19:27:18 +0100
> Subject: [PATCH 4/6] Add CONCURRENTLY option to REPACK command.
> diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
> index cc03f0706e9..a956892f42f 100644
> --- a/src/backend/replication/logical/decode.c
> +++ b/src/backend/replication/logical/decode.c
> @@ -472,6 +473,88 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
> + /*
> + * Second, skip records which do not contain sufficient information for
> + * the decoding.
> + *
> + * The problem we solve here is that REPACK CONCURRENTLY generates WAL
> + * when doing changes in the new table. Those changes should not be useful
> + * for any other user (such as logical replication subscription) because
> + * the new table will eventually be dropped (after REPACK CONCURRENTLY has
> + * assigned its file to the "old table").
> + */
> + switch (info)
> + {
> + case XLOG_HEAP_INSERT:
> + {
> + xl_heap_insert *rec;
> +
> + rec = (xl_heap_insert *) XLogRecGetData(buf->record);
> +
> + /*
> + * This does happen when 1) raw_heap_insert marks the TOAST
> + * record as HEAP_INSERT_NO_LOGICAL, 2) REPACK CONCURRENTLY
> + * replays inserts performed by other backends.
> + */
> + if ((rec->flags & XLH_INSERT_CONTAINS_NEW_TUPLE) == 0)
> + return;
> +
> + break;
> + }
> +
> + case XLOG_HEAP_HOT_UPDATE:
> + case XLOG_HEAP_UPDATE:
> + {
> + xl_heap_update *rec;
> +
> + rec = (xl_heap_update *) XLogRecGetData(buf->record);
> + if ((rec->flags &
> + (XLH_UPDATE_CONTAINS_NEW_TUPLE |
> + XLH_UPDATE_CONTAINS_OLD_TUPLE |
> + XLH_UPDATE_CONTAINS_OLD_KEY)) == 0)
> + return;
> +
> + break;
> + }
> +
> + case XLOG_HEAP_DELETE:
> + {
> + xl_heap_delete *rec;
> +
> + rec = (xl_heap_delete *) XLogRecGetData(buf->record);
> + if (rec->flags & XLH_DELETE_NO_LOGICAL)
> + return;
> + break;
> + }
> + }
I'm confused as to the purpose of this addition. I took this whole
block out, and no tests seem to fail. Moreover, some of the cases that
are being skipped because of this, would already be skipped by code in
DecodeInsert / DecodeUpdate anyway. The case for XLOG_HEAP_DELETE seems
to have no effect (that is, the "return" there never hits for any tests
as far as I can tell.)
The reason I ask is that the line immediately below does this:
> ReorderBufferProcessXid(ctx->reorder, xid, buf->origptr);
which means the Xid is tracked for snapshot building purposes. Which is
probably important, because of what the comment right below it says:
/*
* If we don't have snapshot or we are just fast-forwarding, there is no
* point in decoding data changes. However, it's crucial to build the base
* snapshot during fast-forward mode (as is done in
* SnapBuildProcessChange()) because we require the snapshot's xmin when
* determining the candidate catalog_xmin for the replication slot. See
* SnapBuildProcessRunningXacts().
*/
So what happens here is that we would skip processing the Xid of a xlog
record during snapshot-building, on the grounds that it doesn't contain
logical changes. I'm not sure this is okay. If we do indeed need this,
then perhaps it should be done after ReorderBufferProcessXid().
Or did you intend to make this conditional on the backend running
REPACK?
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-01-05 10:31 ` Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-05 10:31 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2025-Dec-13, Antonin Houska wrote:
>
> > From 6279394135f2b693b6fffd174822509e0a067cbf Mon Sep 17 00:00:00 2001
> > From: Antonin Houska <ah@cybertec.at>
> > Date: Sat, 13 Dec 2025 19:27:18 +0100
> > Subject: [PATCH 4/6] Add CONCURRENTLY option to REPACK command.
>
> > diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
> > index cc03f0706e9..a956892f42f 100644
> > --- a/src/backend/replication/logical/decode.c
> > +++ b/src/backend/replication/logical/decode.c
> > @@ -472,6 +473,88 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
>
> > + /*
> > + * Second, skip records which do not contain sufficient information for
> > + * the decoding.
> > + *
> > + * The problem we solve here is that REPACK CONCURRENTLY generates WAL
> > + * when doing changes in the new table. Those changes should not be useful
> > + * for any other user (such as logical replication subscription) because
> > + * the new table will eventually be dropped (after REPACK CONCURRENTLY has
> > + * assigned its file to the "old table").
> > + */
> > + switch (info)
> > + {
> > + case XLOG_HEAP_INSERT:
> > + {
> > + xl_heap_insert *rec;
> > +
> > + rec = (xl_heap_insert *) XLogRecGetData(buf->record);
> > +
> > + /*
> > + * This does happen when 1) raw_heap_insert marks the TOAST
> > + * record as HEAP_INSERT_NO_LOGICAL, 2) REPACK CONCURRENTLY
> > + * replays inserts performed by other backends.
> > + */
> > + if ((rec->flags & XLH_INSERT_CONTAINS_NEW_TUPLE) == 0)
> > + return;
> > +
> > + break;
> > + }
> > +
> > + case XLOG_HEAP_HOT_UPDATE:
> > + case XLOG_HEAP_UPDATE:
> > + {
> > + xl_heap_update *rec;
> > +
> > + rec = (xl_heap_update *) XLogRecGetData(buf->record);
> > + if ((rec->flags &
> > + (XLH_UPDATE_CONTAINS_NEW_TUPLE |
> > + XLH_UPDATE_CONTAINS_OLD_TUPLE |
> > + XLH_UPDATE_CONTAINS_OLD_KEY)) == 0)
> > + return;
> > +
> > + break;
> > + }
> > +
> > + case XLOG_HEAP_DELETE:
> > + {
> > + xl_heap_delete *rec;
> > +
> > + rec = (xl_heap_delete *) XLogRecGetData(buf->record);
> > + if (rec->flags & XLH_DELETE_NO_LOGICAL)
> > + return;
> > + break;
> > + }
> > + }
>
> I'm confused as to the purpose of this addition. I took this whole
> block out, and no tests seem to fail.
This is just an optimization, to avoid unnecessary decoding of data changes
that the output plugin would ignore anyway. Note that REPACK (CONCURRENTLY)
can generate a huge amount of WAL itself.
> Moreover, some of the cases that
> are being skipped because of this, would already be skipped by code in
> DecodeInsert / DecodeUpdate anyway.
By checking earlier I tried to avoid calling ReorderBufferProcessXid()
unnecessarily.
> The case for XLOG_HEAP_DELETE seems
> to have no effect (that is, the "return" there never hits for any tests
> as far as I can tell.)
The current tests do not cover this, but it should be hit by backends
performing logical decoding unrelated to REPACK. The typical case is that WAL
sender involved in logical replication reads a DELETE record generated by
REPACK (CONCURRENTLY) due to replaying a DELETE statement on the new relation.
> The reason I ask is that the line immediately below does this:
>
> > ReorderBufferProcessXid(ctx->reorder, xid, buf->origptr);
>
> which means the Xid is tracked for snapshot building purposes. Which is
> probably important, because of what the comment right below it says:
>
> /*
> * If we don't have snapshot or we are just fast-forwarding, there is no
> * point in decoding data changes. However, it's crucial to build the base
> * snapshot during fast-forward mode (as is done in
> * SnapBuildProcessChange()) because we require the snapshot's xmin when
> * determining the candidate catalog_xmin for the replication slot. See
> * SnapBuildProcessRunningXacts().
> */
>
> So what happens here is that we would skip processing the Xid of a xlog
> record during snapshot-building, on the grounds that it doesn't contain
> logical changes. I'm not sure this is okay.
I think I missed the fact that SnapBuildProcessChange() relies on
ReorderBufferProcessXid() having been called.
> If we do indeed need this,
> then perhaps it should be done after ReorderBufferProcessXid().
... and after SnapBuildProcessChange(). Thus the changes being discussed here
should be removed from the patch. I'll do that in the next version. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-05 10:40 ` Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-05 10:40 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > If we do indeed need this,
> > then perhaps it should be done after ReorderBufferProcessXid().
>
> ... and after SnapBuildProcessChange(). Thus the changes being discussed here
> should be removed from the patch. I'll do that in the next version. Thanks.
Actually the check of XLH_DELETE_NO_LOGICAL should not be discarded. I think
it should be added to DecodeDelete().
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-08 16:59 ` Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-08 16:59 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Antonin Houska <ah@cybertec.at> wrote:
>
> > Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > > If we do indeed need this,
> > > then perhaps it should be done after ReorderBufferProcessXid().
> >
> > ... and after SnapBuildProcessChange(). Thus the changes being discussed here
> > should be removed from the patch. I'll do that in the next version. Thanks.
>
> Actually the check of XLH_DELETE_NO_LOGICAL should not be discarded. I think
> it should be added to DecodeDelete().
v29 tries to fix the problem.
Besides that, it reflects the recent Mihail's comments [1], [2].
[1]
https://www.postgresql.org/message-id/CADzfLwXp4c-MJx7yVDxAGNNxPbX4o9dqyivxavtHvmUsdXYqBQ@mail.gmail...
[2] https://www.postgresql.org/message-id/CADzfLwWNz_jwi7KVOmJ9D97+zwxsiwDSqSUUJ9oqUCOqkbGnRA@mail.gmail...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-10 17:37 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-10 17:37 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
On Thu, Jan 8, 2026 at 7:59 PM Antonin Houska <ah@cybertec.at> wrote:
> v29 tries to fix the problem.
Some comments for 0001-0004.
------ 0001 -----
> src/bin/scripts/t/103_repackdb.pl:1:
> # Copyright (c) 2021-2025
Update year for 2026.
> * FIXME: this is missing a way to specify the index to use to repack one
> * table, or whether to pass a WITH INDEX clause when multiple tables are
> * used. Something like --index[=indexname]. Adding that bleeds into
> * vacuuming.c as well.
Comments look stale.
> return "???";
I think it is better to add Assert(false); before (done that way in a
few places).
> command <link linkend="sql-repack"><command>REPACK</command></link> There
need .
> “An utility”
Should be “A utility”
> else if (pg_strcasecmp(cmd, "CLUSTER") == 0)
> cmdtype = PROGRESS_COMMAND_CLUSTER;
Should we set PROGRESS_COMMAND_REPACK here? Because cluster is not
used anywhere. Probably we may even delete PROGRESS_COMMAND_CLUSTER.
> CLUOPT_RECHECK_ISCLUSTERED
It is not set anymore... Probably something is wrong here or we need
to just remove that constant and check for it.
------ 0002 -----
> rebuild_relation(Relation OldHeap, Relation index, bool verbose)
It removes unused cmd parameter, but I think it is better to not add
it in the previous commit.
------ 0003 -----
> int newxcnt = 0;
I think it is better to use uint32 for consistency here.
Also, I think it is worth adding Assert(snapshot->snapshot_type ==
SNAPSHOT_HISTORIC_MVCC)
------ 0004 -----
> /* Is REPACK (CONCURRENTLY) being run by this backend? */
> if (am_decoding_for_repack())
We should check change_useless_for_repack here - to avoid looking and
TRUNCATE of unrelated tables.
> /* For the same reason, unlock TOAST relation. */
> if (OldHeap->rd_rel->reltoastrelid)
> LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
Hm, we are locking here instead of unlocking ;)
> (errhint("Relation \"%s\" has no identity index.",
> RelationGetRelationName(rel)))));
One level of braces may be removed.
> * to decode on behalf of REPACK (CONCURRENT)?
CONCURRENTLY
> * If recheck is required, it must have been preformed on the source
"performed"
> * On exit,'*scan_p' contains the scan descriptor used. The caller must close
> * it when he no longer needs the tuple returned.
There is no scan_p argument here.
> * Copyright (c) 2012-2025, PostgreSQL Global Development Group
2026
> newtuple = change->data.tp.newtuple != NULL ?
> change->data.tp.newtuple : NULL;
> oldtuple = change->data.tp.oldtuple != NULL ?
> change->data.tp.oldtuple : NULL;
> newtuple = change->data.tp.newtuple != NULL ?
> change->data.tp.newtuple : NULL;
Hm, should it be just x = y ?
> apply_concurrent_insert
Double newline at function start.
> heap2_decode
Should we check for change_useless_for_repack here also? (for multi
insert, for example).
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-12 16:33 ` Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-12 16:33 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> On Thu, Jan 8, 2026 at 7:59 PM Antonin Houska <ah@cybertec.at> wrote:
> > v29 tries to fix the problem.
>
> Some comments for 0001-0004.
Thanks.
> ------ 0001 -----
> > * FIXME: this is missing a way to specify the index to use to repack one
> > * table, or whether to pass a WITH INDEX clause when multiple tables are
> > * used. Something like --index[=indexname]. Adding that bleeds into
> > * vacuuming.c as well.
>
> Comments look stale.
This is an open question, see [1].
> > return "???";
> I think it is better to add Assert(false); before (done that way in a
> few places).
This is not really uncommon, see for example event_trigger.c. Added the
comment
/* keep compiler quiet */
> > “An utility”
> Should be “A utility”
Not sure it *should be* [2], but "a utility" appears to be much more common in
the tree. Changed.
> > else if (pg_strcasecmp(cmd, "CLUSTER") == 0)
> > cmdtype = PROGRESS_COMMAND_CLUSTER;
>
> Should we set PROGRESS_COMMAND_REPACK here? Because cluster is not
> used anywhere. Probably we may even delete PROGRESS_COMMAND_CLUSTER.
Good point. Actually we do not need this branch at all as there's no
pg_stat_get_progress_info('CLUSTER') call in system_views.sql. Removed.
> > CLUOPT_RECHECK_ISCLUSTERED
> It is not set anymore... Probably something is wrong here or we need
> to just remove that constant and check for it.
Yes, it got lost somehow. I added it where I think it's appropriate.
> ------ 0002 -----
>
> > rebuild_relation(Relation OldHeap, Relation index, bool verbose)
> It removes unused cmd parameter, but I think it is better to not add
> it in the previous commit.
Yes, the changes were not correctly split into diffs. Fixed.
> ------ 0003 -----
>
> > int newxcnt = 0;
>
> I think it is better to use uint32 for consistency here.
This diff only moves code across functions, I'm not going to do other changes
now.
> Also, I think it is worth adding Assert(snapshot->snapshot_type ==
> SNAPSHOT_HISTORIC_MVCC)
ok
> ------ 0004 -----
>
> > /* Is REPACK (CONCURRENTLY) being run by this backend? */
> > if (am_decoding_for_repack())
>
> We should check change_useless_for_repack here - to avoid looking and
> TRUNCATE of unrelated tables.
In v29, if XLOG_HEAP_TRUNCATE of an unrelated table is seen here, we'll raise
ERROR unnecessarily instead of truncating the table. That's obviously wrong as
well. On the other hand, it's not trivial to teach change_useless_for_repack()
to filter the TRUNCATE records by file locator. So besides adding a call of
change_useless_for_repack(), I removed that ereport(ERROR) from heap_decode()
and added a comment to plugin_change() explaining why TRUNCATE on the table
being repacked should fire the Assert() statement: TRUNCATE shouldn't appear
here due to locking.
> > /* For the same reason, unlock TOAST relation. */
> > if (OldHeap->rd_rel->reltoastrelid)
> > LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
>
> Hm, we are locking here instead of unlocking ;)
Copy-pasto, the lock level is incorrect as well. Actually the whole idea of
unlocking index and TOAST relation is probably wrong: if some transaction
already locked the table with a lock that does not conflict with
ShareUpdateExclusiveLock, it should not need to wait for
ShareUpdateExclusiveLock on index / TOAST relation. At least I don't recall a
case where index / TOAST requires stronger lock than the main table. So I
removed the unlocking altogether.
> > (errhint("Relation \"%s\" has no identity index.",
> > RelationGetRelationName(rel)))));
>
> One level of braces may be removed.
> > * to decode on behalf of REPACK (CONCURRENT)?
>
> CONCURRENTLY
> > * If recheck is required, it must have been preformed on the source
>
> "performed"
> > * On exit,'*scan_p' contains the scan descriptor used. The caller must close
> > * it when he no longer needs the tuple returned.
>
> There is no scan_p argument here.
> > * Copyright (c) 2012-2025, PostgreSQL Global Development Group
>
> 2026
ok
> > newtuple = change->data.tp.newtuple != NULL ?
> > change->data.tp.newtuple : NULL;
>
> > oldtuple = change->data.tp.oldtuple != NULL ?
> > change->data.tp.oldtuple : NULL;
> > newtuple = change->data.tp.newtuple != NULL ?
> > change->data.tp.newtuple : NULL;
>
> Hm, should it be just x = y ?
Thanks for spotting this. The reason is that significant portion of this patch
is copied from the pg_squeeze extension, and there it originally looked like:
oldtuple = change->data.tp.oldtuple != NULL ?
&change->data.tp.oldtuple->tuple : NULL;
I failed to notice the unnecessary complexity when adapting the code to the
commit 08e6344fd642 in postgres core, and then copied it to the REPACK
patch. Fixed now.
> > apply_concurrent_insert
>
> Double newline at function start.
ok
> > heap2_decode
>
> Should we check for change_useless_for_repack here also? (for multi
> insert, for example).
Yes, done, thanks. While doing that, I've done the same for XLOG_HEAP_CONFIRM
in heap_decode() as it'd be bad for reorderbuffer.c to receive the CONFIRM
record w/o previously receiving the actual (speculative) INSERT.
[1] https://www.postgresql.org/message-id/7224.1762326739%40localhost
[2] https://en.wiktionary.org/wiki/an#Usage_notes
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-12 18:20 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 18:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-20 15:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 3 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-12 18:20 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
More comments - now for 0005 (but v29, but I think they are mostly up to date).
--- 0005 ---
> potentiallly
extra 'l' in commit message
> Memory the queue is located int.
"in"?
> again if its still eligible
if it's still eligible
> int initialized;
probably better to be bool (as in shared)
> DecodingWorkerState
such type does not exists in commit
> REPACK_WORKER_MAIN
Not used in code anywhere.
> int64 timeout = 0;
> WaitLSNResult res;
formatting issue here (tab vs space)
> if (size >= MaxAllocSize)
Seems like we lost that check, I think it may be executed on storing
the data or before "tup = (HeapTuple) palloc(HEAPTUPLESIZE + t_len);"
in apply_concurrent_changes
> bool done;
I still think it is a confusing name.
> chgdst.file_seq = WORKER_FILE_SNAPSHOT + 1;
I think it is better to increment it once a snapshot is received. And
rename to last_processed/last_improrted to be aligned with
last_exported.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-12 18:54 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-15 16:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-12 18:54 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Also, there are some crashes of stress tests for v30 (for both single
snapshot and multiple snapshot versions).
---------------------
Looks like something is leaking, but not sure.
https://cirrus-ci.com/task/5577209672368128?logs=test_world#L277 (multiple
snapshots)
https://cirrus-ci.com/task/6439044873191424 (without multiple snapshots)
[17:49:07.251] # Failed test 'concurrent operations with REINDEX/CREATE
INDEX CONCURRENTLY stderr /(?^:^$)/'
[17:49:07.251] # at /tmp/cirrus-ci-build/contrib/amcheck/t/
007_repack_concurrently.pl line 56.
[17:49:07.251] # 'pgbench: error: client 0 script 0
aborted in command 6 query 0: ERROR: out of background worker slots
[17:49:07.251] # HINT: You might need to increase "max_worker_processes".
[17:49:07.251] # pgbench: error: Run was aborted due to an error in thread 0
-------------------
This one showed something goes wrong, the sum of the table is broken. It
may be 0 because non-MVCC safe, but I checked the logs:
2026-01-12 18:41:11.656 UTC client backend[76247] 007_repack_concurrently.pl
LOG: statement: SELECT (490588) / 0;
And also
backend[54349] 007_repack_concurrently.pl ERROR: could not create unique
index "tbl_pkey_repacknew"
2026-01-12 18:41:12.477 UTC client backend[54349] 007_repack_concurrently.pl
DETAIL: Key (i)=(942) is duplicated.
2026-01-12 18:41:12.477 UTC client backend[54349] 007_repack_concurrently.pl
STATEMENT: REPACK (CONCURRENTLY) tbl;
https://cirrus-ci.com/task/4521496594350080 (single snapshot version)
https://cirrus-ci.com/task/6157569896480768 (single snapshot version)
[18:36:17.466] # at /Users/admin/pgsql/contrib/amcheck/t/
007_repack_concurrently.pl line 56.
[18:36:17.466] # 'pgbench: error: client 21 script 0
aborted in command 31 query 0: ERROR: division by zero
[18:36:17.466] # pgbench: error: Run was aborted due to an error in thread 2
---------------------
https://cirrus-ci.com/task/5682762788634624 (multiple snapshots)
Failed test 'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY
stderr /(?^:^$)/'
[18:02:06.938] # at t/007_repack_concurrently.pl line 56.
[18:02:06.938] # 'pgbench: error: client 6 aborted in
command 4 (SQL) of script 0; perhaps the backend died while processing
[18:02:06.938] # pgbench: error: Run was aborted due to an error in thread 0
[18:02:06.938] # WARNING: terminating connection because of crash of
another server process
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 18:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-15 16:36 ` Antonin Houska <ah@cybertec.at>
2026-01-15 17:05 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-15 16:36 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Also, there are some crashes of stress tests for v30 (for both single snapshot and multiple snapshot versions).
>
> ---------------------
>
> Looks like something is leaking, but not sure.
>
> https://cirrus-ci.com/task/5577209672368128?logs=test_world#L277 (multiple snapshots)
> https://cirrus-ci.com/task/6439044873191424 (without multiple snapshots)
As the test runs pgbench with --client=30 and the default value of
max_worker_processes is 8, I'm not sure this is a leak. I've increased this
parameter I couldn't see the error anymore.
> This one showed something goes wrong, the sum of the table is broken. It may be 0 because non-MVCC safe, but I checked the logs:
>
> 2026-01-12 18:41:11.656 UTC client backend[76247] 007_repack_concurrently.pl LOG: statement: SELECT (490588) / 0;
I agree that this is due to the missing MVCC safety feature. I commented that
check in the script for now.
Besides that, I saw some deadlocks. I think this was due to the fact that
multiple rows are updated per transaction, and that the keys are random, so it
can happen that two transactions try to update the same rows in different
order. I increased the number of rows in the test table to 10000 and don't see
the deadlocks anymore.
> backend[54349] 007_repack_concurrently.pl ERROR: could not create unique index "tbl_pkey_repacknew"
> 2026-01-12 18:41:12.477 UTC client backend[54349] 007_repack_concurrently.pl DETAIL: Key (i)=(942) is duplicated.
> 2026-01-12 18:41:12.477 UTC client backend[54349] 007_repack_concurrently.pl STATEMENT: REPACK (CONCURRENTLY) tbl;
This is tricky. I could reproduce the problem on my FreeBSD box a few times,
never on Linux (no idea if the OS makes the difference since HW is also quite
different, but CI also seemed to fail more often on FreeBSD.)
Something seems to be wrong about UPDATE, but I'm failing to understand how it
could relate to REPACK. This is an example of a duplicate value i=6118
SELECT i, j, xmin, xmax, ctid FROM tbl WHERE i=6118;
i | j | xmin | xmax | ctid
------+--------+--------+--------+---------
6118 | 445435 | 102317 | 103702 | (1,216)
6118 | 391135 | 103702 | 0 | (56,62)
According to log, xid=102317 is the transaction used by REPACK and xid=103702
one of the test. pageinspect shows that the old version has not only
HEAP_XMIN_COMMITTED in t_infomask, but also HEAP_XMAX_INVALID.
So far I could not reproduce the duplicities with the REPACK (CONCURRENTLY)
command commented out in the test script, but that does not prove much (even
with REPACK, not every run fails). Also I noticed that REPACK incorrectly sets
cmin/cmax to 1 instead of 0 and it needs to be fixed, but I have no idea why
this bug should cause exactly this weird behavior.
I even added quite a few logging messages to reveal where in the code the
HEAP_XMAX_INVALID flag is set for particular ctid, but after a failure I could
not find the message for the problematic tuples. Ideas are appreciated.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 18:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-15 16:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-15 17:05 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-16 18:18 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-15 17:05 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
Antonin Houska <ah@cybertec.at>:
As the test runs pgbench with --client=30 and the default value of
max_worker_processes is 8, I'm not sure this is a leak. I've increased this
parameter I couldn't see the error anymore.
Hm, as far as I remember only single repack may be executed in test
(because of locking on test itself and also REPACK).
At least still feel suspicious.
I agree that this is due to the missing MVCC safety feature. I commented
that
check in the script for now.
I don't think so. In case of non-MVCC safety we should see 0 or correct
sum. But script failed with 490588...
But should see 500500 (if I correctly calculated sum of numbers from 1 to
1000)...
Besides that, I saw some deadlocks. I think this was due to the fact that
multiple rows are updated per transaction, and that the keys are random, so
it
can happen that two transactions try to update the same rows in different
order. I increased the number of rows in the test table to 10000 and don't
see
the deadlocks anymore.
I think better to use min/max in updates to be sure (update lower id first).
This is tricky. I could reproduce the problem on my FreeBSD box a few times,
never on Linux (no idea if the OS makes the difference since HW is also
quite
different, but CI also seemed to fail more often on FreeBSD.)
You may try to play with no_hot parameter in test - maybe it will provide
some clue.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 18:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-15 16:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-15 17:05 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-16 18:18 ` Antonin Houska <ah@cybertec.at>
2026-01-19 16:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-16 18:18 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
>Antonin Houska <ah@cybertec.at>:
>>
>> As the test runs pgbench with --client=30 and the default value of
>> max_worker_processes is 8, I'm not sure this is a leak. I've increased this
>> parameter I couldn't see the error anymore.
>
> Hm, as far as I remember only single repack may be executed in test (because
> of locking on test itself and also REPACK).
The only problem is that the logical decoding system needs to wait during the
setup for all the running transactions to finish. So if REPACK (CONCURRENTLY)
is already running, the next execution will not start until the first is done.
However, that does not restrict the REPACK decoding workers from starting.
>> I agree that this is due to the missing MVCC safety feature. I commented that
>> check in the script for now.
>
> I don't think so. In case of non-MVCC safety we should see 0 or correct sum. But script failed with 490588...
> But should see 500500 (if I correctly calculated sum of numbers from 1 to 1000)...
I was referring to your statement "It may be 0 because non-MVCC
safe". Regarding the non-zero values, I think I finally understand the issue
and even could reproduce some weird behavior using debugger. Since it also
affects logical replication, I'll provide more details (and hopefully propose
a patch) in a separate thread early next week.
In short, it looks like (hopefully very) rare race condition, such that the
snapshot builder can build the initial snapshot before all the commits have
been recorded in CLOG. When that happens, visibility checks don't work
correctly.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 18:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-15 16:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-15 17:05 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-16 18:18 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-19 16:31 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-01-19 16:31 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
>
> >Antonin Houska <ah@cybertec.at>:
> >>
> >> As the test runs pgbench with --client=30 and the default value of
> >> max_worker_processes is 8, I'm not sure this is a leak. I've increased this
> >> parameter I couldn't see the error anymore.
> >
> > Hm, as far as I remember only single repack may be executed in test (because
> > of locking on test itself and also REPACK).
>
> The only problem is that the logical decoding system needs to wait during the
> setup for all the running transactions to finish. So if REPACK (CONCURRENTLY)
> is already running, the next execution will not start until the first is done.
>
> However, that does not restrict the REPACK decoding workers from starting.
>
> >> I agree that this is due to the missing MVCC safety feature. I commented that
> >> check in the script for now.
> >
> > I don't think so. In case of non-MVCC safety we should see 0 or correct sum. But script failed with 490588...
> > But should see 500500 (if I correctly calculated sum of numbers from 1 to 1000)...
>
> I was referring to your statement "It may be 0 because non-MVCC
> safe". Regarding the non-zero values, I think I finally understand the issue
> and even could reproduce some weird behavior using debugger. Since it also
> affects logical replication, I'll provide more details (and hopefully propose
> a patch) in a separate thread early next week.
This is the report:
https://www.postgresql.org/message-id/85833.1768840165%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-18 21:52 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-18 21:52 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
Some comments for 0006:
> SnapBuildSnapshotForRepack(SnapBuild *builder)
Does it also "replays" previously processed WAL to the position that
snapshot is ready to use?
I am afraid we may see some non-yet processed parts of WAL leading to
duplicate insertion.
> first_block
What is the reason for that variable? It seems to be always the first block
of relation.
Also, what if we have a huge amount of empty space at the start. In that
case the first block will be the block of the first "filled" page. But
insert may (and will) fill empty pages before first_block - out of the
range.
So, I think we should delete it and always use 0 instead.
> DecodeMultiInsert
ItemPointerSet-related logic seems missing.
> tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL);
With SnapshotAny we are going to check the same tuple multiple times.
Better to let scan logic handle it (and change snapshots used by scan code).
See [0] for a way to reset snapshots during the scan.
> if (blkno >= range_end)
I don't think it is legal to switch a snapshot while holding the tuple.
Nothing is protecting it from being pruned\reused.
Snapshots need to be switched "between" pages. You may check how it is done
at [0].
> PopActiveSnapshot();
> InvalidateCatalogSnapshot();
I think it is a good idea to add here assert for MyProc->xmin and
MyProc->xid to be invalid. To ensure we really allow the horizon to advance.
> /* Set to request a snapshot. */
> bool snapshot_requested;
We know the end or region in advance, so it should be possible to filter
before writing changes to file.
So, it is some kind of "this is the end of region, create new file and
store everything before + create new snapshot for me".
> PopActiveSnapshot();
Sometimes without InvalidateCatalogSnapshot().
> PushActiveSnapshot(GetTransactionSnapshot());
GetLatestSnapshot() feels better here.
> * The individual builds might still be a problem, but that's a
> * separate issue.
Opening the index may create a catalog snapshot, so it needs to be
invalidated after.
> * TODO Can we somehow use the fact that the new heap is not yet
> * visible to other transaction, and thus cannot be vacuumed?
Snapshot resetting [0] may work here (without CIC, just as part of the scan
+ some code to ensure catalog snapshot is managed correctly).
Also, to correctly build a unique index - some tech from [0] is required
(building a unique index with multiple snapshots is a little bit tricky).
Or we may implement some super lightweight way - just SnapshotAny without
any visibility checks (just assume everything is ok since it copied from
another relation with the same index set).
> This approach introduces one limitation though: if the USING INDEX clause
is
> specified, an explicit sort is always used. Index scan wouldn't work
because
> it does not return the tuples sorted by CTID.
Technically we may just use keys (if they are comparable) as a way to
specify regions. Instead of number of pages to switch snapshot - number of
tuples or time.
But because we don't know the region end in advance - we have to keep all
the changes in file and filter only while applying.
> Assert(XLogRecPtrIsInvalid(shared->lsn_upto));
> /* Initially we're expected to provide a snapshot and only that. */
> Assert(shared->snapshot_requested &&
> XLogRecPtrIsInvalid(shared->lsn_upto));
XLogRecPtrIsInvalid(shared->lsn_upto) assertion is duplicated here.
> range_end = repack_blocks_per_snapshot;
Should be repack_blocks_per_snapshot + ctx->first_block ? (but better to
remove the first_block at all).
> * XXX It might be worth Assert(CatalogSnapshot == NULL)
> * here, however that symbol is not external.
As said above - better assert for MyProc->xmin/xid + add
InvalidateCatalogSnapshot.
> extern Snapshot
> extern void
Some "externs" used in C files (not headers).
> ctx->block_ranges = NIL;
may be used with list_free?
> * Remember here to which pages should applied to changes recorded in
given
> * file.
"should apply"
> of pages, so that VACUUM does not get block for too long
"blocked"
> there are no restriction on block
"there is" or "restrictions"
> repack_add_block_range
extra new-line after function.
> Is REPACKED (CONCURRENTLY) is being run by this backend?
REPACK and double "is"
[0]: https://commitfest.postgresql.org/patch/6401/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-22 08:37 ` Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-22 08:37 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Some comments for 0006:
>
> > SnapBuildSnapshotForRepack(SnapBuild *builder)
> Does it also "replays" previously processed WAL to the position that snapshot is ready to use?
> I am afraid we may see some non-yet processed parts of WAL leading to duplicate insertion.
The changes present in WAL decoded prior the snapshot creation are not
replayed - these changes are visible to the snapshot. (This is not really
specific to the 0006 part.)
> > first_block
> What is the reason for that variable? It seems to be always the first block
> of relation.
Although scan usually starts at the first block, it does not have to,
especially due to synchronized sequential scans.
> Also, what if we have a huge amount of empty space at the start. In that case the first block will be the block of the first "filled" page. But
> insert may (and will) fill empty pages before first_block - out of the range.
Good catch! I think I used this "lazy initialization" because I couldn't find
the start block in TableScanDesc, and missed the problem that you describe
here.
> So, I think we should delete it and always use 0 instead.
No, we need to set first_block to heapScan->rs_startblock before the scan
starts.
> > tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL);
> With SnapshotAny we are going to check the same tuple multiple times. Better to let scan logic handle it (and change snapshots used by scan
> code).
The current API does not seem to support changing snapshot of an in-progress
scan and I don't want to change that. Plus note that the current
implementation of CLUSTER also uses SnapshotAny and then checks the visibility
separately. Finally, SnapshotAny is not really an expensive visibility check,
if it can be considered a visibility check at all.
> > if (blkno >= range_end)
> I don't think it is legal to switch a snapshot while holding the tuple. Nothing is protecting it from being pruned\reused.
Snapshot protects the table as whole from pruning live (or recently dead)
tuples, but when you have fetched a tuple, the containing buffer remains
pinned. The buffer pin itself makes prunning of the page impossible.
And especially with REPACK (CONCURRENTLY), page pruning is also restricted by
the replication slot's xmin. This is increased by calling
LogicalIncreaseXminForSlot() from the decoding worker, each time it has
created a new snapshot.
> > PopActiveSnapshot();
> > InvalidateCatalogSnapshot();
> I think it is a good idea to add here assert for MyProc->xmin and MyProc->xid to be invalid. To ensure we really allow the horizon to advance.
I've added it only for xmin. xid is valid because REPACK is executed in a
transaction. That reminds me that PROC_IN_VACUUM should be present in
MyProc->statusFlags. Fixed.
> > /* Set to request a snapshot. */
> > bool snapshot_requested;
> We know the end or region in advance, so it should be possible to filter before writing changes to file.
Yes, filtering before writing makes sense, I'll consider that.
> So, it is some kind of "this is the end of region, create new file and store everything before + create new snapshot for me".
The last se of changes does not have to be followed by a snapshot - that's the
purpose of snapshot_requested.
> > PopActiveSnapshot();
> Sometimes without InvalidateCatalogSnapshot().
[ It'd be a bit easier to find the code if you included hunk headers. ]
heapam_relation_copy_for_cluster() does not access catalogs after the first
invalidation. The following comment is related:
/*
* XXX It might be worth Assert(CatalogSnapshot == NULL) here,
* however that symbol is not external.
*/
> > PushActiveSnapshot(GetTransactionSnapshot());
> GetLatestSnapshot() feels better here.
What will then happen to code that uses GetActiveSnapshot() ?
> > * The individual builds might still be a problem, but that's a
> > * separate issue.
> Opening the index may create a catalog snapshot, so it needs to be invalidated after.
It'll be invalidated in the next iteration. The point of this invalidation is
to use one snapshot per index.
> > * TODO Can we somehow use the fact that the new heap is not yet
> > * visible to other transaction, and thus cannot be vacuumed?
> Snapshot resetting [0] may work here (without CIC, just as part of the scan + some code to ensure catalog snapshot is managed correctly).
> Also, to correctly build a unique index - some tech from [0] is required (building a unique index with multiple snapshots is a little bit tricky).
> Or we may implement some super lightweight way - just SnapshotAny without any visibility checks (just assume everything is ok since it
> copied from another relation with the same index set).
ok, I'll check your patch.
> > This approach introduces one limitation though: if the USING INDEX clause is
> > specified, an explicit sort is always used. Index scan wouldn't work because
> > it does not return the tuples sorted by CTID.
>
> Technically we may just use keys (if they are comparable) as a way to specify regions. Instead of number of pages to switch snapshot -
> number of tuples or time.
> But because we don't know the region end in advance - we have to keep all the changes in file and filter only while applying.
Good idea. Unfortunately it questions your proposal to filter the changes
before writing, as suggested above.
> > range_end = repack_blocks_per_snapshot;
> Should be repack_blocks_per_snapshot + ctx->first_block ?
Indeed, I also failed to avoid assuming first_block==0 :-)
> > * XXX It might be worth Assert(CatalogSnapshot == NULL)
> > * here, however that symbol is not external.
> As said above - better assert for MyProc->xmin/xid + add InvalidateCatalogSnapshot.
I proposed the Assert above, but still thinking about it.
> > Is REPACKED (CONCURRENTLY) is being run by this backend?
> REPACK and double "is"
Other comments accepted. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-22 11:30 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-22 11:30 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
> The changes present in WAL decoded prior the snapshot creation are not
> replayed - these changes are visible to the snapshot. (This is not really
> specific to the 0006 part.)
OK, just want to be sure it still works the same way if we build multiple
snapshots for the same slot that way.
> The current API does not seem to support changing snapshot of an
in-progress
> scan and I don't want to change that. Plus note that the current
> implementation of CLUSTER also uses SnapshotAny and then checks the
visibility
> separately. Finally, SnapshotAny is not really an expensive visibility
check,
> if it can be considered a visibility check at all.
But we will require a real check for each tuple. Including dead one,
multiple versions of the same HOT, etc.
> I've added it only for xmin. xid is valid because REPACK is executed in a
> transaction. That reminds me that PROC_IN_VACUUM should be present in
> MyProc->statusFlags. Fixed.
Yes, xid is required for repack. I think it is better to introduce a new
flag instead of PROC_IN_VACCUUM.
> > > PushActiveSnapshot(GetTransactionSnapshot());
> > GetLatestSnapshot() feels better here.
> What will then happen to code that uses GetActiveSnapshot() ?
O, I mean PushActiveSnapshot(GetLatestSnapshot())
> > Also, to correctly build a unique index - some tech from [0] is
required (building a unique index with multiple snapshots is a little bit
tricky).
> ok, I'll check your patch.
I realized building a unique index is still done with a single snapshot, so
it should be OK for that case. But still check the patch :)
> I proposed the Assert above, but still thinking about it.
Hm... Do we really need these asserts if PROC_IN_VACUUM is set? I was
proposing a way it is used for index building (to ensure nothing is
propagated into xmin).
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-25 16:31 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-25 16:31 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
PART 1:
I started rebasing the MVCC-safe version on top of the multi-snapshot
version and realized it becomes complex.
But, what's really bad about MVCC-unsafety is the ability to access
*incorrect* data and break some logic (or even constraints).
If we may *prevent* such data access with some kind of error (which is
going to be very infrequent) - I don't see any sense to achieve true
MVCC-safety.
I remembered a way it works with indcheckxmin for indexes. And made
something similar for pg_class: it records the rewriting transaction XID
and causes the executor to raise an error if a transaction with an older
snapshot attempts to access the rewritten relation.
For the normal case - check is never executed, no performance regression
here. Also, the flag is automatically cleared by VACUUM once the
transaction ID is frozen.
It also "fixes" ALTER TABLE, not only REPACK concurrently.
Attached patch contains more details (some in the commit message).
PART 2:
I have continued working with stress tests. This time I added your WIP
patch to fix the LR\CLOG race.
I made the following configs:
1) just REPACK CONCURRENTLY - ok
2) + relcheckxmin (see PART1) - ok
3) + worker - ok
4) + multiple snapshots - broken in multiple ways.
You may see example of run here -
https://cirrus-ci.com/build/6359048020295680
Some examples:
1) 'pgbench: error: client 11 script 0 aborted in command 20 query 0:
ERROR: could not read blocks 0..0 in file "base/5/16414": read only 0 of
8192 bytes
2) at /home/postgres/postgres/contrib/amcheck/t/008_repack_concurrently.pl
line 51.
[15:36:37.204] # 'pgbench: error: client 5 script 0
aborted in command 28 query 0: ERROR: division by zero
3) 'pgbench: error: client 12 script 0 aborted in command 6 query 0:
ERROR: cache lookup failed for relation 17400
Attachments:
[application/x-patch] nocfbot-stress_tests_for_repack_concurrently.patch (6.6K, ../../CADzfLwUEH5+LjCN+6kRfSsXwuou8rKXyVV42Wi-O_TG0360Kug@mail.gmail.com/3-nocfbot-stress_tests_for_repack_concurrently.patch)
download | inline diff:
Subject: [PATCH] stress tests for repack concurrently
---
Index: contrib/amcheck/meson.build
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
--- a/contrib/amcheck/meson.build (revision 26b7a7cbedd8c0848c1056ef820bed2ddb5f522c)
+++ b/contrib/amcheck/meson.build (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
@@ -50,6 +50,8 @@
't/004_verify_nbtree_unique.pl',
't/005_pitr.pl',
't/006_verify_gin.pl',
+ 't/007_repack_concurrently.pl',
+ 't/008_repack_concurrently.pl',
],
},
}
Index: contrib/amcheck/t/007_repack_concurrently.pl
===================================================================
diff --git a/contrib/amcheck/t/007_repack_concurrently.pl b/contrib/amcheck/t/007_repack_concurrently.pl
new file mode 100644
--- /dev/null (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/contrib/amcheck/t/007_repack_concurrently.pl (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
@@ -0,0 +1,111 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+max_worker_processes = 32
+));
+
+my $n=1000;
+my $no_hot = int(rand(2));
+
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl(i int PRIMARY KEY, j int)));
+
+if ($no_hot)
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(j);));
+}
+else
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(i);));
+}
+
+
+# Load amcheck
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+
+# Insert $n rows into tbl
+$node->safe_psql('postgres', qq(
+ INSERT INTO tbl SELECT i, i FROM generate_series(1,$n) i
+));
+
+my $sum = $node->safe_psql('postgres', q(
+ SELECT SUM(j) AS sum FROM tbl
+));
+
+
+$node->pgbench(
+'--no-vacuum --client=15 --jobs=4 --exit-on-abort --transactions=5000',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REPACK CONCURRENTLY',
+{
+ 'concurrent_ops' => qq(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \\gset
+ \\if :gotlock
+ REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl USING INDEX test_idx;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ SELECT pg_advisory_unlock(42);
+ \\else
+ \\set num_a random(1, $n)
+ \\set num_b random(1, $n)
+ \\set diff random(1, 10000)
+ BEGIN;
+ UPDATE tbl SET j = j + :diff WHERE i = :num_a;
+ \\sleep 1 ms
+ UPDATE tbl SET j = j - :diff WHERE i = :num_b;
+ \\sleep 1 ms
+ COMMIT;
+
+ BEGIN
+ --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ ;
+ SELECT 1;
+ \\sleep 1 ms
+ SELECT COALESCE(SUM(j), 0) AS sum FROM tbl \\gset p_
+ \\if :p_sum != $sum
+ COMMIT;
+ SELECT (:p_sum) / 0;
+ \\endif
+
+ COMMIT;
+ \\endif
+ )
+});
+
+$node->stop;
+done_testing();
Index: contrib/amcheck/t/008_repack_concurrently.pl
===================================================================
diff --git a/contrib/amcheck/t/008_repack_concurrently.pl b/contrib/amcheck/t/008_repack_concurrently.pl
new file mode 100644
--- /dev/null (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/contrib/amcheck/t/008_repack_concurrently.pl (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
@@ -0,0 +1,102 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+max_worker_processes = 32
+));
+
+my $no_hot = int(rand(2));
+
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl(i SERIAL PRIMARY KEY, j int)));
+if ($no_hot)
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(j);));
+}
+else
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(i);));
+}
+
+# Load amcheck
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+
+my $sum = $node->safe_psql('postgres', q(
+ SELECT SUM(j) AS sum FROM tbl
+));
+
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE last_j START 1 INCREMENT 1;));
+
+
+$node->pgbench(
+'--no-vacuum --client=15 --jobs=4 --exit-on-abort --transactions=1000',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REPACK CONCURRENTLY',
+{
+ 'concurrent_ops' => qq(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \\gset
+ \\if :gotlock
+ REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl USING INDEX test_idx;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ SELECT pg_advisory_unlock(42);
+ \\else
+ SELECT pg_advisory_lock(43);
+ BEGIN;
+ INSERT INTO tbl(j) VALUES (nextval('last_j')) RETURNING j \\gset p_
+ COMMIT;
+ SELECT pg_advisory_unlock(43);
+ \\sleep 1 ms
+
+ BEGIN
+ --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ ;
+ SELECT 1;
+ \\sleep 1 ms
+ SELECT COUNT(*) AS count FROM tbl WHERE j <= :p_j \\gset p_
+ \\if :p_count != :p_j
+ COMMIT;
+ SELECT (:p_count) / 0;
+ \\endif
+
+ COMMIT;
+ \\endif
+ )
+});
+
+$node->stop;
+done_testing();
[application/x-patch] nocfbot-Add_`relcheckxmin`_to_track_and_enforce_tuple_visibility_in_`pg_class`.patch (18.4K, ../../CADzfLwUEH5+LjCN+6kRfSsXwuou8rKXyVV42Wi-O_TG0360Kug@mail.gmail.com/4-nocfbot-Add_%60relcheckxmin%60_to_track_and_enforce_tuple_visibility_in_%60pg_class%60.patch)
download | inline diff:
Subject: [PATCH] Add `relcheckxmin` to track and enforce tuple visibility in `pg_class`
---
Index: doc/src/sgml/mvcc.sgml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
--- a/doc/src/sgml/mvcc.sgml (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/doc/src/sgml/mvcc.sgml (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -1833,22 +1833,30 @@
<title>Caveats</title>
<para>
- Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the
- table-rewriting forms of <link linkend="sql-altertable"><command>ALTER
- TABLE</command></link> and <command>REPACK</command> with
- the <literal>CONCURRENTLY</literal> option, are not
- MVCC-safe. This means that after the truncation or rewrite commits, the
+ Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>,
+ are not MVCC-safe. This means that after the truncation commits, the
table will appear empty to concurrent transactions, if they are using a
- snapshot taken before the command committed. This will only be an
- issue for a transaction that did not access the table in question
- before the command started — any transaction that has done so
- would hold at least an <literal>ACCESS SHARE</literal> table lock,
- which would block the truncating or rewriting command until that transaction completes.
- So these commands will not cause any apparent inconsistency in the
- table contents for successive queries on the target table, but they
+ snapshot taken before the command committed. This will only be an issue
+ for a transaction that did not access the table in question before the
+ command started — any transaction that has done so would hold at
+ least an <literal>ACCESS SHARE</literal> table lock, which would block
+ the <command>TRUNCATE</command> command until that transaction completes.
+ So <command>TRUNCATE</command> will not cause any apparent inconsistency
+ in the table contents for successive queries on the target table, but it
could cause visible inconsistency between the contents of the target
table and other tables in the database.
</para>
+
+ <para>
+ The table-rewriting forms of
+ <link linkend="sql-altertable"><command>ALTER TABLE</command></link>
+ and <link linkend="sql-repack"><command>REPACK</command></link>
+ with the <literal>CONCURRENTLY</literal> option are also not MVCC-safe.
+ If a transaction attempts to access a relation that was rewritten
+ after the transaction's snapshot was taken, an error will be raised.
+ As above, this will only be an issue for a transaction that did not
+ access the table before the rewriting command started.
+ </para>
<para>
Support for the Serializable transaction isolation level has not yet
Index: src/backend/catalog/heap.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
--- a/src/backend/catalog/heap.c (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/catalog/heap.c (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -953,6 +953,7 @@
values[Anum_pg_class_relrewrite - 1] = ObjectIdGetDatum(rd_rel->relrewrite);
values[Anum_pg_class_relfrozenxid - 1] = TransactionIdGetDatum(rd_rel->relfrozenxid);
values[Anum_pg_class_relminmxid - 1] = MultiXactIdGetDatum(rd_rel->relminmxid);
+ values[Anum_pg_class_relcheckxmin - 1] = TransactionIdGetDatum(rd_rel->relcheckxmin);
if (relacl != (Datum) 0)
values[Anum_pg_class_relacl - 1] = relacl;
else
@@ -1023,6 +1024,8 @@
/* relispartition is always set by updating this tuple later */
new_rel_reltup->relispartition = false;
+ new_rel_reltup->relcheckxmin = InvalidTransactionId;
+
/* fill rd_att's type ID with something sane even if reltype is zero */
new_rel_desc->rd_att->tdtypeid = new_type_oid ? new_type_oid : RECORDOID;
new_rel_desc->rd_att->tdtypmod = -1;
Index: src/backend/commands/cluster.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
--- a/src/backend/commands/cluster.c (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/commands/cluster.c (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -1031,7 +1031,7 @@
* rebuild the target's indexes and throw away the transient table.
*/
finish_heap_swap(tableOid, OIDNewHeap, is_system_catalog,
- swap_toast_by_content, false, true, true,
+ swap_toast_by_content, false, true, true, InvalidTransactionId,
frozenXid, cutoffMulti,
relpersistence);
}
@@ -1429,6 +1429,7 @@
swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
bool swap_toast_by_content,
bool is_internal,
+ TransactionId check_xmin,
TransactionId frozenXid,
MultiXactId cutoffMulti,
Oid *mapped_tables)
@@ -1611,6 +1612,9 @@
relform2->relallfrozen = swap_allfrozen;
}
+ relform1->relcheckxmin = check_xmin;
+ relform2->relcheckxmin = check_xmin;
+
/*
* Update the tuples in pg_class --- unless the target relation of the
* swap is pg_class itself. In that case, there is zero point in making
@@ -1688,6 +1692,7 @@
target_is_pg_class,
swap_toast_by_content,
is_internal,
+ InvalidTransactionId,
frozenXid,
cutoffMulti,
mapped_tables);
@@ -1791,6 +1796,7 @@
target_is_pg_class,
swap_toast_by_content,
is_internal,
+ check_xmin,
InvalidTransactionId,
InvalidMultiXactId,
mapped_tables);
@@ -1814,6 +1820,7 @@
bool check_constraints,
bool is_internal,
bool reindex,
+ TransactionId check_xmin,
TransactionId frozenXid,
MultiXactId cutoffMulti,
char newrelpersistence)
@@ -1835,7 +1842,7 @@
*/
swap_relation_files(OIDOldHeap, OIDNewHeap,
(OIDOldHeap == RelationRelationId),
- swap_toast_by_content, is_internal,
+ swap_toast_by_content, is_internal, check_xmin,
frozenXid, cutoffMulti, mapped_tables);
/*
@@ -3335,6 +3342,7 @@
false, /* swap_toast_by_content */
true,
InvalidTransactionId,
+ InvalidTransactionId,
InvalidMultiXactId,
mapped_tables);
@@ -3371,7 +3379,7 @@
finish_heap_swap(old_table_oid, new_table_oid,
is_system_catalog,
false, /* swap_toast_by_content */
- false, true, false,
+ false, true, false, GetCurrentTransactionId(),
frozenXid, cutoffMulti,
relpersistence);
}
Index: src/backend/commands/matview.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
--- a/src/backend/commands/matview.c (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/commands/matview.c (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -892,7 +892,7 @@
static void
refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersistence)
{
- finish_heap_swap(matviewOid, OIDNewHeap, false, false, true, true, true,
+ finish_heap_swap(matviewOid, OIDNewHeap, false, false, true, true, true, InvalidTransactionId,
RecentXmin, ReadNextMultiXactId(), relpersistence);
}
Index: src/backend/commands/tablecmds.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
--- a/src/backend/commands/tablecmds.c (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/commands/tablecmds.c (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -6026,6 +6026,7 @@
false, false, true,
!OidIsValid(tab->newTableSpace),
true,
+ GetCurrentTransactionId(),
RecentXmin,
ReadNextMultiXactId(),
persistence);
Index: src/backend/commands/vacuum.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
--- a/src/backend/commands/vacuum.c (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/commands/vacuum.c (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -1534,6 +1534,9 @@
if (update)
{
pgcform->relfrozenxid = frozenxid;
+ /* Clear relcheckxmin if frozenxid is higher, now it is safe */
+ if (TransactionIdPrecedes(pgcform->relcheckxmin, pgcform->relfrozenxid))
+ pgcform->relcheckxmin = InvalidTransactionId;
dirty = true;
if (frozenxid_updated)
*frozenxid_updated = true;
Index: src/backend/executor/execUtils.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
--- a/src/backend/executor/execUtils.c (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/executor/execUtils.c (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -865,6 +865,24 @@
estate->es_relations[rti - 1] = rel;
}
+ if (unlikely(TransactionIdIsValid(rel->rd_check_xmin)))
+ {
+ /* We know it is committed, just need to be sure it is visible for current exec */
+ TransactionId xmin = rel->rd_check_xmin;
+ /* Fast check first */
+ if (unlikely(!TransactionIdIsCurrentTransactionId(xmin) &&
+ !TransactionIdPrecedes(xmin, TransactionXmin)))
+ {
+ Snapshot snapshot = GetActiveSnapshot();
+ /* Test remaining rules */
+ if (XidInMVCCSnapshot(xmin, snapshot))
+ ereport(ERROR,
+ errcode(ERRCODE_RELATION_UNAVAILABLE_FOR_CURRENT_TRANSACTION),
+ errmsg("relation \"%s\" cannot be accessed in current transaction",
+ RelationGetRelationName(rel)));
+ }
+ }
+
return rel;
}
Index: src/backend/utils/cache/relcache.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
--- a/src/backend/utils/cache/relcache.c (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/utils/cache/relcache.c (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -1203,6 +1203,8 @@
*/
RelationBuildTupleDesc(relation);
+ relation->rd_check_xmin = relp->relcheckxmin;
+
/* foreign key data is not loaded till asked for */
relation->rd_fkeylist = NIL;
relation->rd_fkeyvalid = false;
Index: src/backend/utils/errcodes.txt
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/utils/errcodes.txt b/src/backend/utils/errcodes.txt
--- a/src/backend/utils/errcodes.txt (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/backend/utils/errcodes.txt (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -259,6 +259,7 @@
25P02 E ERRCODE_IN_FAILED_SQL_TRANSACTION in_failed_sql_transaction
25P03 E ERRCODE_IDLE_IN_TRANSACTION_SESSION_TIMEOUT idle_in_transaction_session_timeout
25P04 E ERRCODE_TRANSACTION_TIMEOUT transaction_timeout
+25P05 E ERRCODE_RELATION_UNAVAILABLE_FOR_CURRENT_TRANSACTION relation_unavailable_for_current_transaction
Section: Class 26 - Invalid SQL Statement Name
Index: src/include/catalog/catversion.h
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
--- a/src/include/catalog/catversion.h (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/include/catalog/catversion.h (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -57,6 +57,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202601221
+#define CATALOG_VERSION_NO 202601242
#endif
Index: src/include/catalog/pg_class.h
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
--- a/src/include/catalog/pg_class.h (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/include/catalog/pg_class.h (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -131,6 +131,9 @@
/* all multixacts in this rel are >= this; it is really a MultiXactId */
TransactionId relminmxid BKI_DEFAULT(1); /* FirstMultiXactId */
+ /* need to check to prevent MVCC violations */
+ TransactionId relcheckxmin BKI_DEFAULT(0); /* InvalidTransactionId */
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: These fields are not present in a relcache entry's rd_rel field. */
/* access permissions */
@@ -146,7 +149,7 @@
/* Size of fixed part of pg_class tuples, not counting var-length fields */
#define CLASS_TUPLE_SIZE \
- (offsetof(FormData_pg_class,relminmxid) + sizeof(TransactionId))
+ (offsetof(FormData_pg_class,relcheckxmin) + sizeof(TransactionId))
/* ----------------
* Form_pg_class corresponds to a pointer to a tuple with
Index: src/include/commands/cluster.h
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
--- a/src/include/commands/cluster.h (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/include/commands/cluster.h (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -130,6 +130,7 @@
bool check_constraints,
bool is_internal,
bool reindex,
+ TransactionId check_xmin,
TransactionId frozenXid,
MultiXactId cutoffMulti,
char newrelpersistence);
Index: src/include/utils/rel.h
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
--- a/src/include/utils/rel.h (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/include/utils/rel.h (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -143,6 +143,11 @@
*/
TransactionId rd_partdesc_nodetached_xmin;
+ /*
+ * Used at execution phase to prevent violation of MVCC rules.
+ */
+ TransactionId rd_check_xmin;
+
/* data managed by RelationGetPartitionQual: */
List *rd_partcheck; /* partition CHECK quals */
bool rd_partcheckvalid; /* true if list has been computed */
Index: src/test/modules/injection_points/Makefile
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
--- a/src/test/modules/injection_points/Makefile (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/test/modules/injection_points/Makefile (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -16,6 +16,7 @@
inplace \
repack \
repack_toast \
+ repack_mvcc \
syscache-update-pruned \
heap_lock_update
ISOLATION_OPTS = --temp-config $(top_srcdir)/src/test/modules/injection_points/logical.conf
Index: src/test/modules/injection_points/expected/repack_mvcc.out
===================================================================
diff --git a/src/test/modules/injection_points/expected/repack_mvcc.out b/src/test/modules/injection_points/expected/repack_mvcc.out
new file mode 100644
--- /dev/null (revision e5311f14c463819fabe6c57fa74512f10c881508)
+++ b/src/test/modules/injection_points/expected/repack_mvcc.out (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -0,0 +1,19 @@
+Parsed test spec with 2 sessions
+
+starting permutation: begin repack check
+step begin:
+ BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+ SELECT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step repack:
+ REPACK (CONCURRENTLY) repack_test;
+
+step check:
+ SELECT * FROM repack_test;
+
+ERROR: relation "repack_test" cannot be accessed in current transaction
Index: src/test/modules/injection_points/meson.build
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
--- a/src/test/modules/injection_points/meson.build (revision 14bf8dbc0ac0d52a6121be2484f652fc0c04732d)
+++ b/src/test/modules/injection_points/meson.build (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -47,6 +47,7 @@
'inplace',
'repack',
'repack_toast',
+ 'repack_mvcc',
'syscache-update-pruned',
'heap_lock_update',
],
Index: src/test/modules/injection_points/specs/repack_mvcc.spec
===================================================================
diff --git a/src/test/modules/injection_points/specs/repack_mvcc.spec b/src/test/modules/injection_points/specs/repack_mvcc.spec
new file mode 100644
--- /dev/null (revision e5311f14c463819fabe6c57fa74512f10c881508)
+++ b/src/test/modules/injection_points/specs/repack_mvcc.spec (revision e5311f14c463819fabe6c57fa74512f10c881508)
@@ -0,0 +1,38 @@
+# REPACK (CONCURRENTLY);
+#
+# Test handling of preventing access to non-mvcc safe data.
+setup
+{
+ CREATE EXTENSION injection_points;
+
+ CREATE TABLE repack_test(i int PRIMARY KEY);
+ INSERT INTO repack_test(i) VALUES (1), (2), (3);
+}
+
+teardown
+{
+ DROP TABLE repack_test;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+step repack
+{
+ REPACK (CONCURRENTLY) repack_test;
+}
+
+session s2
+step begin
+{
+ BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+ SELECT 1;
+}
+step check
+{
+ SELECT * FROM repack_test;
+}
+
+permutation
+ begin
+ repack
+ check
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-26 07:34 ` Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-26 07:34 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> PART 1:
>
> I started rebasing the MVCC-safe version on top of the multi-snapshot version and realized it becomes complex.
> But, what's really bad about MVCC-unsafety is the ability to access *incorrect* data and break some logic (or even constraints).
>
> If we may *prevent* such data access with some kind of error (which is going to be very infrequent) - I don't see any sense to achieve true
> MVCC-safety.
>
> I remembered a way it works with indcheckxmin for indexes. And made something similar for pg_class: it records the rewriting transaction
> XID and causes the executor to raise an error if a transaction with an older snapshot attempts to access the rewritten relation.
>
> For the normal case - check is never executed, no performance regression here. Also, the flag is automatically cleared by VACUUM once the
> transaction ID is frozen.
>
> It also "fixes" ALTER TABLE, not only REPACK concurrently.
>
> Attached patch contains more details (some in the commit message).
A few days ago, when thinking about it, I realized that my implementation of
MVCC safety is not correct, as it does not preserve the whole HOT chains as
CLUSTER / VACUUM FULL does. To resolve that, we should not allow access to the
new table until the parts of HOT chains not copied by REPACK are DEAD.
Better solution might be to improve rewriteheap.c (which does keep the HOT
chains) so it 1) works w/o AccessExclusiveLock on the table, 2) does not copy
tuples which will eventually be retrieved by the logical decoding output
plugin, 3) allows snapshot switching/resetting in 2). I think these
requirements are rather hard to implement.
I've noticed recently that the MVCC safety patch you posted is not exactly
what I wrote, but maybe the copying part is identical. So it's possible that
the problem you saw is related to what I try to describe here.
Let's see in the future if the demand for the MVCC safety will ever justify
the effort to implement it.
> PART 2:
>
> I have continued working with stress tests. This time I added your WIP patch to fix the LR\CLOG race.
>
> I made the following configs:
> 1) just REPACK CONCURRENTLY - ok
> 2) + relcheckxmin (see PART1) - ok
> 3) + worker - ok
> 4) + multiple snapshots - broken in multiple ways.
>
> You may see example of run here - https://cirrus-ci.com/build/6359048020295680
>
> Some examples:
>
> 1) 'pgbench: error: client 11 script 0 aborted in command 20 query 0: ERROR: could not read blocks 0..0 in file "base/5/16414": read only 0
> of 8192 bytes
> 2) at /home/postgres/postgres/contrib/amcheck/t/008_repack_concurrently.pl line 51.
> [15:36:37.204] # 'pgbench: error: client 5 script 0 aborted in command 28 query 0: ERROR: division by zero
> 3) 'pgbench: error: client 12 script 0 aborted in command 6 query 0: ERROR: cache lookup failed for relation 17400
Thanks, I'll check these.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-27 10:57 ` Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-01-27 10:57 UTC (permalink / raw)
To: ; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
>
> > PART 2:
> >
> > I have continued working with stress tests. This time I added your WIP patch to fix the LR\CLOG race.
> >
> > I made the following configs:
> > 1) just REPACK CONCURRENTLY - ok
> > 2) + relcheckxmin (see PART1) - ok
> > 3) + worker - ok
> > 4) + multiple snapshots - broken in multiple ways.
> >
> > You may see example of run here - https://cirrus-ci.com/build/6359048020295680
> >
> > Some examples:
> >
> > 1) 'pgbench: error: client 11 script 0 aborted in command 20 query 0: ERROR: could not read blocks 0..0 in file "base/5/16414": read only 0
> > of 8192 bytes
> > 2) at /home/postgres/postgres/contrib/amcheck/t/008_repack_concurrently.pl line 51.
> > [15:36:37.204] # 'pgbench: error: client 5 script 0 aborted in command 28 query 0: ERROR: division by zero
> > 3) 'pgbench: error: client 12 script 0 aborted in command 6 query 0: ERROR: cache lookup failed for relation 17400
>
> Thanks, I'll check these.
PROC_IN_VACUUM shouldn't be used for the same reason StartupDecodingContext()
avoids setting PROC_IN_LOGICAL_DECODING in transaction. I've removed that and
the tests work for me. Especially the "cache lookup failed" error is almost
certainly related. Please let me know if you still get the other errors
(Except for 2, which is probably due to the MVCC-unsafe behavior, as discussed
earlier.)
The 0006 part needs more work (definitely beyond PG 19). For now I've
summarized the problem in the code this way:
+ * As there is no snapshot, our xmin should be invalid now.
+ *
+ * TODO xid can still be valid. We can mark our transaction with the
+ * PROC_IN_VACUUM flag, but at the same time we need to make sure that
+ * anything we write is ignored by VACUUM: since our xid is >= xmin of
+ * our replication slot, the slot does not help. Other transaction
+ * might use their RecentXmin to check if our xact is still running
+ * (see TransactionIdIsInProgress) before they check CLOG. By using
+ * PROC_IN_VACUUM we'd let their RecentXmin skip our xid. Thus our
+ * xact would appear not running anymore, but not yet marked committed
+ * in CLOG either, therefore aborted: it's o.k. for VACUUM to clean up
+ * tuples written by aborted transaction.
+ *
+ * Perhaps we can add a new field 'relisvalid' to pg_class and
+ * something alike to pg_index and make sure that neither queries nor
+ * VACUUM can use tables / indexes which do not have this flag set
+ * (The existing pg_index(indisvalid) field probably should not
+ * control whether VACUUM is allowed or not). Then we can do the
+ * catalog changes in separate transactions. Only the transaction that
+ * copies the heap would then use the PROC_IN_VACUUM flag. However,
+ * even then it would probably be appropriate to do regular
+ * (MVCC-safe) rewriting, i.e. avoid setting the xid of the rewriting
+ * transaction in the tuple headers.
Thanks for your testing!
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-28 02:06 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 07:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-01-28 02:06 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
> PROC_IN_VACUUM shouldn't be used for the same reason
StartupDecodingContext()
> avoids setting PROC_IN_LOGICAL_DECODING in transaction. I've removed
that and
> the tests work for me. Especially the "cache lookup failed" error is
almost
> certainly related. Please let me know if you still get the other errors
Yes, now it is passing.
> (Except for 2, which is probably due to the MVCC-unsafe behavior, as
discussed
> earlier.)
Not happening too. BTW, it was non MVCC-related, because in that case
relcheckxmin would catch it.
What if:
1) add new PROC_IN_REPACK flag
2) use it in catalog horizon, but not in data (like was done in [0] for
PROC_IN_SAFE_IC)
And after we have options:
3) do not "table_close(NewHeap, NoLock);" -
keep ShareUpdateExclusiveLock all the time to prevent VACUUM enter
4) do not heap_page_prune_opt in repack transaction (just using simple
flag)
Or
3) preserve xmin/xmax of original transaction in repacked data
4) but better to keep ShareUpdateExclusiveLock anyway
Seems to be enough.
> The 0006 part needs more work (definitely beyond PG 19).
This is sad, because if you are in a situation then you need REPACK -
pinning the horizon for too long may just finish your DB....
And also, even with 0006 we still need to build indexes, which might pin it
for long (even duration caused by a single index).
Mikhail.
[0]:
https://github.com/postgres/postgres/commit/d9d076222f5b94a85e0e318339cfc44b8f26022d
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-30 19:33 ` Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-30 19:33 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > PROC_IN_VACUUM shouldn't be used for the same reason StartupDecodingContext()
> > avoids setting PROC_IN_LOGICAL_DECODING in transaction. I've removed that and
> > the tests work for me. Especially the "cache lookup failed" error is almost
> > certainly related. Please let me know if you still get the other errors
>
> Yes, now it is passing.
>
> > (Except for 2, which is probably due to the MVCC-unsafe behavior, as discussed
> > earlier.)
>
> Not happening too. BTW, it was non MVCC-related, because in that case relcheckxmin would catch it.
>
> What if:
>
> 1) add new PROC_IN_REPACK flag
> 2) use it in catalog horizon, but not in data (like was done in [0] for PROC_IN_SAFE_IC)
>
> And after we have options:
> 3) do not "table_close(NewHeap, NoLock);" - keep ShareUpdateExclusiveLock all the time to prevent VACUUM enter
> 4) do not heap_page_prune_opt in repack transaction (just using simple flag)
> Or
> 3) preserve xmin/xmax of original transaction in repacked data
> 4) but better to keep ShareUpdateExclusiveLock anyway
I've been thinking of another approach. Note that REPACK creates a new table
only to eventually swap the relation files and drop it. Thus the transactions
needs to get XID assigned very soon.
I'm considering a special kind of relation whose catalog entries remain in the
catalog cache and are never written to the catalog tables. (Unlike temporary
relation, it'd be WAL logged so that REPACK can be replayed on standby.)
If we eventually implement the MVCC safety, XID will neither be needed during
data copying. And it shouldn't even be needed to build indexes, as long as
their catalog entries are also "cache only". Thus the transaction REPACK is
running in would not need XID until the data has been copied, indexes built
and even (most of) the concurrent data changes replayed. Only the final
catalog changes would require XID, but those should take very short time.
Without XID and with the snapshot resetting, REPACK should not really block
the progress of the VACUUM xmin horizon.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-01 19:46 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-01 22:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-02 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-01 19:46 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
PART 1:
--------------
Something still wrong with 0006, check:
'pgbench: error: client 12 script 0 aborted in command 2 query 0: ERROR:
attempted to overwrite invisible tuple
https://cirrus-ci.com/task/6385612527239168?logs=test_world#L300
But it is hard to reproduce - happened once.
--------------
Also, once I got
[16:25:18.641] # at /tmp/cirrus-ci-build/contrib/amcheck/t/
007_repack_concurrently.pl line 57.
[16:25:18.641] # 'pgbench: error: client 6 script 0
aborted in command 2 query 0: ERROR: relation 21856 deleted while still in
use
https://cirrus-ci.com/task/4686014242881536?logs=test_world#L384
It was the PROC_IN_REPACK version (see below), but I think it is not
related to it. But I'm not 100% sure.
PART 2:
> I'm considering a special kind of relation whose catalog entries remain
in the
> catalog cache and are never written to the catalog tables. (Unlike
temporary
> relation, it'd be WAL logged so that REPACK can be replayed on standby.)
I think it is too complicated, especially including replication logic.
Approach with catalog-only xid is much simpler, it was even committed (yes,
reverted but because of another reason).
Essentially we have two issues:
1) make sure catalog entities are not dropped because the vacuum
2) make sure data in new table is not vacuumed also
For the first PROC_IN_REPACK is enough.
For second - depends if MVCC-safe (original xmin/xmax) are preserved. If
yes - looks like nothing more needed.
If not - just prevent the vacuum from touching the table (but, looks like
it is done already, because lock is held on NewHeap until commit).
And additionally reset snapshots during the index building itself, but it
is scope of another patch.
I have implemented PROC_IN_REPACK POC in the attached patch.
Also, I am still not sure if MVCC-safe implementation is worth
its complexity compared with "relcheckxmin"approach [0].
[0]:
https://www.postgresql.org/message-id/CADzfLwUEH5%2BLjCN%2B6kRfSsXwuou8rKXyVV42Wi-O_TG0360Kug%40mail...
Best regards,
Mikhail.
Attachments:
[application/octet-stream] vX-0001-Handle-VACUUM-interaction-with-REPACK-CONCURRENTL.patch (6.8K, ../../CADzfLwUukiGOPoUkDgf6oEB-Y0TnNy6UFUN4obnU-AN5W1N=sw@mail.gmail.com/3-vX-0001-Handle-VACUUM-interaction-with-REPACK-CONCURRENTL.patch)
download | inline diff:
From 2edaf6d9a427c9f683989bd8eb1ed4da9579541a Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 1 Feb 2026 14:21:41 +0100
Subject: [PATCH vX] Handle VACUUM interaction with REPACK CONCURRENTLY and
separate catalog/data horizons, based on d9d076222f5
---
src/backend/commands/cluster.c | 32 ++++++++++++++++++++++++-
src/backend/storage/ipc/procarray.c | 36 +++++++++++++++++++++++------
src/include/storage/proc.h | 6 +++--
3 files changed, 64 insertions(+), 10 deletions(-)
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 403bdcda32e..5afd6e8088f 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -292,7 +292,7 @@ static void export_snapshot(Snapshot snapshot,
DecodingWorkerShared *shared);
static void ProcessRepackMessage(StringInfo msg);
static const char *RepackCommandAsString(RepackCommand cmd);
-
+static void set_in_repack_procflags(void);
#define REPL_PLUGIN_NAME "pgoutput_repack"
@@ -355,6 +355,14 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
parser_errposition(pstate, opt->location));
}
+ if ((params.options & CLUOPT_CONCURRENT) != 0)
+ {
+ InvalidateCatalogSnapshot();
+ PopActiveSnapshot();
+ set_in_repack_procflags();
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
/*
* Determine the lock mode expected by cluster_rel().
*
@@ -509,6 +517,12 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
continue;
}
+ if ((params.options & CLUOPT_CONCURRENT) != 0)
+ {
+ InvalidateCatalogSnapshot();
+ set_in_repack_procflags();
+ }
+
/* functions in indexes may want a snapshot set */
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4357,3 +4371,19 @@ ProcessRepackMessage(StringInfo msg)
}
}
}
+
+static void
+set_in_repack_procflags(void)
+{
+ /*
+ * This should only be called before installing xid or xmin in MyProc;
+ * otherwise, concurrent processes could see an Xmin that moves backwards.
+ */
+ Assert(MyProc->xid == InvalidTransactionId &&
+ MyProc->xmin == InvalidTransactionId);
+
+ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
+ MyProc->statusFlags |= PROC_IN_REPACK;
+ ProcGlobal->statusFlags[MyProc->pgxactoff] = MyProc->statusFlags;
+ LWLockRelease(ProcArrayLock);
+}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 6be565155ab..9cede06338a 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -1642,7 +1642,12 @@ TransactionIdIsInProgress(TransactionId xid)
* relations that's not required, since only backends in my own database could
* ever see the tuples in them. Also, we can ignore concurrently running lazy
* VACUUMs because (a) they must be working on other tables, and (b) they
- * don't need to do snapshot-based lookups.
+ * don't need to do snapshot-based lookups. Similarly, for the non-catalog
+ * horizon, we can ignore REPACK CONCURRENTLY for the
+ * same reasons and because they can't run in transaction blocks. (They are
+ * not possible to ignore for catalogs, because REPACK do some catalog
+ * operations.) Do note that this means that REPACK must use a lock level
+ * that conflicts with VACUUM.
*
* This also computes a horizon used to truncate pg_subtrans. For that
* backends in all databases have to be considered, and concurrently running
@@ -1689,9 +1694,6 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
bool in_recovery = RecoveryInProgress();
TransactionId *other_xids = ProcGlobal->xids;
- /* inferred after ProcArrayLock is released */
- h->catalog_oldest_nonremovable = InvalidTransactionId;
-
LWLockAcquire(ProcArrayLock, LW_SHARED);
h->latest_completed = TransamVariables->latestCompletedXid;
@@ -1711,6 +1713,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->oldest_considered_running = initial;
h->shared_oldest_nonremovable = initial;
+ h->catalog_oldest_nonremovable = initial;
h->data_oldest_nonremovable = initial;
/*
@@ -1809,11 +1812,26 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
(statusFlags & PROC_AFFECTS_ALL_HORIZONS) ||
in_recovery)
{
- h->data_oldest_nonremovable =
- TransactionIdOlder(h->data_oldest_nonremovable, xmin);
+ /*
+ * We can ignore this backend if it's running REPACK
+ * CONCURRENTLY - but
+ * only on vacuums of user-defined tables.
+ */
+ if (!(statusFlags & PROC_IN_REPACK))
+ h->data_oldest_nonremovable =
+ TransactionIdOlder(h->data_oldest_nonremovable, xmin);
+
+ /* Catalog tables need to consider all backends in this db */
+ h->catalog_oldest_nonremovable =
+ TransactionIdOlder(h->catalog_oldest_nonremovable, xmin);
+
}
}
+ /* catalog horizon should never be later than data */
+ Assert(TransactionIdPrecedesOrEquals(h->catalog_oldest_nonremovable,
+ h->data_oldest_nonremovable));
+
/*
* If in recovery fetch oldest xid in KnownAssignedXids, will be applied
* after lock is released.
@@ -1835,6 +1853,8 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
TransactionIdOlder(h->shared_oldest_nonremovable, kaxmin);
h->data_oldest_nonremovable =
TransactionIdOlder(h->data_oldest_nonremovable, kaxmin);
+ h->catalog_oldest_nonremovable =
+ TransactionIdOlder(h->catalog_oldest_nonremovable, kaxmin);
/* temp relations cannot be accessed in recovery */
}
@@ -1862,7 +1882,9 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
h->shared_oldest_nonremovable =
TransactionIdOlder(h->shared_oldest_nonremovable,
h->slot_catalog_xmin);
- h->catalog_oldest_nonremovable = h->data_oldest_nonremovable;
+ h->catalog_oldest_nonremovable =
+ TransactionIdOlder(h->catalog_oldest_nonremovable,
+ h->slot_xmin);
h->catalog_oldest_nonremovable =
TransactionIdOlder(h->catalog_oldest_nonremovable,
h->slot_catalog_xmin);
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 039bc8353be..5396680d2b9 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -66,16 +66,18 @@ struct XidCache
#define PROC_AFFECTS_ALL_HORIZONS 0x20 /* this proc's xmin must be
* included in vacuum horizons
* in all databases */
+#define PROC_IN_REPACK 0x40
+
/* flags reset at EOXact */
#define PROC_VACUUM_STATE_MASK \
- (PROC_IN_VACUUM | PROC_IN_SAFE_IC | PROC_VACUUM_FOR_WRAPAROUND)
+ (PROC_IN_VACUUM | PROC_IN_SAFE_IC | PROC_VACUUM_FOR_WRAPAROUND | PROC_IN_REPACK)
/*
* Xmin-related flags. Make sure any flags that affect how the process' Xmin
* value is interpreted by VACUUM are included here.
*/
-#define PROC_XMIN_FLAGS (PROC_IN_VACUUM | PROC_IN_SAFE_IC)
+#define PROC_XMIN_FLAGS (PROC_IN_VACUUM | PROC_IN_SAFE_IC | PROC_IN_REPACK)
/*
* We allow a limited number of "weak" relation locks (AccessShareLock,
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-01 22:31 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-01 22:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-02-01 22:31 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Feb-01, Mihail Nikalayeu wrote:
> Also, I am still not sure if MVCC-safe implementation is worth
> its complexity compared with "relcheckxmin"approach [0].
I'm not sure it's acceptable to cause other sessions to raise errors if
they query the table being repacked (or a table repacked recently).
That sounds extremely unpleasant. Imagine a long-running transactions
that runs enormous queries for many hours or even days, being killed
near the end because some DBA decided to run REPACK on a table.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-01 22:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-01 22:37 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:17 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-01 22:37 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
> I'm not sure it's acceptable to cause other sessions to raise errors if
> they query the table being repacked (or a table repacked recently).
> That sounds extremely unpleasant. Imagine a long-running transactions
> that runs enormous queries for many hours or even days, being killed
> near the end because some DBA decided to run REPACK on a table.
It will not. It raises an error only for the case table will be "empty"
because REPACK switched to new with all tuples with REPACK xid and our
transaction treats that xid as running.
So, there is no regression here, it just changes from "see an empty table
because of MVCC violation" to "get error" in the exact situation.
I prefer the second.
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-01 22:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-01 22:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-02 09:17 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-02 09:17 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
> > I'm not sure it's acceptable to cause other sessions to raise errors if
> > they query the table being repacked (or a table repacked recently).
> > That sounds extremely unpleasant. Imagine a long-running transactions
> > that runs enormous queries for many hours or even days, being killed
> > near the end because some DBA decided to run REPACK on a table.
> It will not. It raises an error only for the case table will be "empty"
because REPACK switched to new with all tuples with REPACK xid and our
transaction treats that xid as running.
> So, there is no regression here, it just changes from "see an empty table
because of MVCC violation" to "get error" in the exact situation.
> I prefer the second.
But I agree that a MVCC-safe solution is better. But to receive the error
you need to be really unlucky (subtle race - in *first* access to a
repacked table in transaction you need to get a statement snapshot in the
moment of table swap) or use non READ-COMMITED isolation method.
So, we prevent silent READ of MVCC-violated data - I think it is enough, at
least for start.
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-02 09:35 ` Antonin Houska <ah@cybertec.at>
2026-02-02 10:04 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-02 09:35 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> PART 1:
>
> --------------
>
> Something still wrong with 0006, check:
>
> 'pgbench: error: client 12 script 0 aborted in command 2 query 0: ERROR: attempted to overwrite invisible tuple
> https://cirrus-ci.com/task/6385612527239168?logs=test_world#L300
>
> But it is hard to reproduce - happened once.
>
> --------------
>
> Also, once I got
> [16:25:18.641] # at /tmp/cirrus-ci-build/contrib/amcheck/t/007_repack_concurrently.pl line 57.
> [16:25:18.641] # 'pgbench: error: client 6 script 0 aborted in command 2 query 0: ERROR: relation 21856 deleted while still in use
> https://cirrus-ci.com/task/4686014242881536?logs=test_world#L384
>
> It was the PROC_IN_REPACK version (see below), but I think it is not related to it. But I'm not 100% sure.
I think it *is* related. My earlier patch version, which used the
PROC_IN_VACUUM flag improperly [1] was also causing visibility issues. Please
let me know if you manage to reproduce the issue with v32.
> PART 2:
>
> > I'm considering a special kind of relation whose catalog entries remain in the
> > catalog cache and are never written to the catalog tables. (Unlike temporary
> > relation, it'd be WAL logged so that REPACK can be replayed on standby.)
>
> I think it is too complicated, especially including replication logic.
I'm confused by hearing a complaint about complexity of code that I haven't
posted yet. And I don't understand the relationship to "replication logic":
REPACK (CONCURRENTLY) tries to avoid decoding of data changes in the *new*
(transient) relation anyway.
> Essentially we have two issues:
> 1) make sure catalog entities are not dropped because the vacuum
> 2) make sure data in new table is not vacuumed also
3) XID assigned early due to creation of catalog entries for the new table -
that XID prevents the VACUUM xmin horizon from advancing till the end of the
transaction, i.e. till the end of REPACK execution.
> Also, I am still not sure if MVCC-safe implementation is worth its complexity compared with "relcheckxmin"approach [0].
IMO it's better for users to see the correct data than ERROR. But it still
needs work.
[1] https://www.postgresql.org/message-id/88003.1769511456%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-02 10:04 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 19:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 16:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-02 10:04 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
> I think it *is* related. My earlier patch version, which used the
> PROC_IN_VACUUM flag improperly [1] was also causing visibility issues.
Please
> let me know if you manage to reproduce the issue with v32.
Will try. Just to highlight - first error happened on v31 *without*
PROC_IN_REPACK.
Second error had PROC_IN_REPACK code, but it wasn't executed (flag wasn't
set) - that's why I think it is not related.
> I'm confused by hearing a complaint about complexity of code that I
haven't
> posted yet. And I don't understand the relationship to "replication
logic":
> REPACK (CONCURRENTLY) tries to avoid decoding of data changes in the *new*
> (transient) relation anyway.
I am not about complexity of code, but more about complexity of approach
(introducing new things like cache-only relations).
"Replication logic" - is about the fact you mentioned that such a relation
is going to be replicated to standby (as result, some replication-related
code is affected too, probably standby promotion also).
Compared to the PROC_IN_REPACK flag - it feels overly complicated for me.
PROC_IN_REPACK is the simplest thing here - just exclude XID from
data-horizon, but keep it in catalog. That's all.
Also, maybe I sound a little bit rude, sorry, it is just because of the
language barrier.
> 3) XID assigned early due to creation of catalog entries for the new
table -
> that XID prevents the VACUUM xmin horizon from advancing till the end of
the
> transaction, i.e. till the end of REPACK execution.
Yes, but PROC_IN_REPACK covers it as well. That xid only in the catalog
horizon.
> IMO it's better for users to see the correct data than ERROR. But it still
> needs work.
Agreed, for me it is ordered like this (from bad to good):
1) silently see incorrect data in rear race
2) receive error instead in that race <----- acceptable for me
3) no error, data is correct
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 10:04 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-02 19:39 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-02-02 19:39 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > I think it *is* related. My earlier patch version, which used the
> > PROC_IN_VACUUM flag improperly [1] was also causing visibility issues. Please
> > let me know if you manage to reproduce the issue with v32.
>
> Will try. Just to highlight - first error happened on v31 *without* PROC_IN_REPACK.
> Second error had PROC_IN_REPACK code, but it wasn't executed (flag wasn't set) - that's why I think it is not related.
ok, v31 is the one that uses PROC_IN_VACUUM incorrectly.
> > I'm confused by hearing a complaint about complexity of code that I haven't
> > posted yet. And I don't understand the relationship to "replication logic":
> > REPACK (CONCURRENTLY) tries to avoid decoding of data changes in the *new*
> > (transient) relation anyway.
>
> I am not about complexity of code, but more about complexity of approach (introducing new things like cache-only relations).
> "Replication logic" - is about the fact you mentioned that such a relation is going to be replicated to standby (as result, some
> replication-related code is affected too, probably standby promotion also).
I thought you mean logical replication. Regarding streaming replication, I
mentioned it rather for the record. I need to check details to see if it
requires special attention.
> Compared to the PROC_IN_REPACK flag - it feels overly complicated for me.
> PROC_IN_REPACK is the simplest thing here - just exclude XID from data-horizon, but keep it in catalog. That's all.
My preference is to avoid hacking procarray.c if a reasonable alternative
exists.
> Also, maybe I sound a little bit rude, sorry, it is just because of the language barrier.
No, that's fine. Since we've met at pgconf.eu, I think you're not a bad guy
:-) Technical discussions are mostly about problems, so they tend to sound
negative as such.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 10:04 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-06 16:29 ` Antonin Houska <ah@cybertec.at>
2026-02-07 14:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-06 16:29 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > I think it *is* related. My earlier patch version, which used the
> > PROC_IN_VACUUM flag improperly [1] was also causing visibility issues. Please
> > let me know if you manage to reproduce the issue with v32.
>
> Will try. Just to highlight - first error happened on v31 *without* PROC_IN_REPACK.
I spent some time running the test with your branch (based on v32 as you told
me off-list), but couldn't reproduce the problem.
The related code is in heap_inplace_lock():
/* no known way this can happen */
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg_internal("attempted to overwrite invisible tuple")));
The only path REPACK (CONCURRENTLY) uses in-place update seems to be:
cluster.c:build_new_indexes() -> index_create_copy() -> index_create() ->
index_build() -> index_update_stats() -> systable_inplace_update_begin()
However I've got no idea how this can be related to REPACK. Since the new
index is not visible to other transactions until REPACK is done, VACUUM should
be the only process able to change the tuple before
heap_inplace_lock(). Indeed, the server log seems to indicate relationship to
VACUUM:
2026-02-01 16:44:58.878 UTC autovacuum worker[22589] LOG: automatic vacuum of table "postgres.pg_catalog.pg_class": index scans: 1
...
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: COMMIT;
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: SELECT pg_try_advisory_lock(42)::integer AS gotlock
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: SELECT pg_advisory_lock(43);
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: BEGIN;
2026-02-01 16:44:58.884 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: INSERT INTO tbl(j) VALUES (nextval('last_j')) RETURNING j
2026-02-01 16:44:58.885 UTC client backend[12727] 008_repack_concurrently.pl LOG: statement: SELECT COUNT(*) AS count FROM tbl WHERE j <= 14148
2026-02-01 16:44:58.885 UTC client backend[12734] 008_repack_concurrently.pl LOG: statement: SELECT COUNT(*) AS count FROM tbl WHERE j <= 14145
2026-02-01 16:44:58.885 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: COMMIT;
2026-02-01 16:44:58.885 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: SELECT pg_advisory_unlock(43);
2026-02-01 16:44:58.887 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: BEGIN
--TRANSACTION ISOLATION LEVEL REPEATABLE READ
;
2026-02-01 16:44:58.887 UTC client backend[12737] 008_repack_concurrently.pl LOG: statement: SELECT 1;
2026-02-01 16:44:58.891 UTC REPACK decoding worker[22621] FATAL: terminating background worker "REPACK decoding worker" due to administrator command
2026-02-01 16:44:58.896 UTC client backend[12740] 008_repack_concurrently.pl LOG: statement: SELECT COUNT(*) AS count FROM tbl WHERE j <= 14146
2026-02-01 16:44:58.896 UTC client backend[12722] 008_repack_concurrently.pl ERROR: attempted to overwrite invisible tuple
2026-02-01 16:44:58.896 UTC client backend[12722] 008_repack_concurrently.pl STATEMENT: REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;
However, VACUUM should not touch the tuple because the scan in
systable_inplace_update_begin() should leave the containing buffer pinned. I
wonder if you managed to hit another existing bug.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 10:04 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-06 16:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-07 14:16 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-09 11:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-07 14:16 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi!
> Indeed, the server log seems to indicate relationship to
> VACUUM:
> 2026-02-01 16:44:58.878 UTC autovacuum worker[22589] LOG: automatic
vacuum of table "postgres.pg_catalog.pg_class": index scans: 1
O, it's a good clue!
I have added some vacuum calls for pg_class in a stress test - and now it
fails much more often (check attachment).
It is "ERROR: cache lookup failed for relation" - but I think it may share
the cause with "attempted to overwrite invisible tuple.
See:
https://cirrus-ci.com/build/4852126532239360 - with "Use multiple snapshots
to copy the data."
https://cirrus-ci.com/build/6429084491710464 - with "Use background worker
to do logical decoding."
But I am unable to reproduce the issue with only "Add CONCURRENTLY option
to REPACK command."
https://cirrus-ci.com/build/6467070524653568
Best regards,
Mikhail.
Attachments:
[application/x-patch] nocfbot-vX-0001-stress-tests-for-repack-concurrently.patch (6.7K, ../../CADzfLwWNv5QDn6qmxCRV-p_ijSTGwNcEZFCOXt09+RmpSG2=+w@mail.gmail.com/3-nocfbot-vX-0001-stress-tests-for-repack-concurrently.patch)
download | inline diff:
From cbb3fc402e8a0ce14abee46f329b15df6157b7f2 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Tue, 27 Jan 2026 21:41:56 +0100
Subject: [PATCH vX 1/3] stress tests for repack concurrently
---
contrib/amcheck/meson.build | 2 +
contrib/amcheck/t/007_repack_concurrently.pl | 116 +++++++++++++++++++
contrib/amcheck/t/008_repack_concurrently.pl | 106 +++++++++++++++++
3 files changed, 224 insertions(+)
create mode 100644 contrib/amcheck/t/007_repack_concurrently.pl
create mode 100644 contrib/amcheck/t/008_repack_concurrently.pl
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index d5137ef691d..f726db2ffe0 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -50,6 +50,8 @@ tests += {
't/004_verify_nbtree_unique.pl',
't/005_pitr.pl',
't/006_verify_gin.pl',
+ 't/007_repack_concurrently.pl',
+ 't/008_repack_concurrently.pl',
],
},
}
diff --git a/contrib/amcheck/t/007_repack_concurrently.pl b/contrib/amcheck/t/007_repack_concurrently.pl
new file mode 100644
index 00000000000..fb110b0b57d
--- /dev/null
+++ b/contrib/amcheck/t/007_repack_concurrently.pl
@@ -0,0 +1,116 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+max_worker_processes = 32
+));
+
+my $n=1000;
+my $no_hot = int(rand(2));
+
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl(i int PRIMARY KEY, j int)));
+
+if ($no_hot)
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(j);));
+}
+else
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(i);));
+}
+
+
+# Load amcheck
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+
+# Insert $n rows into tbl
+$node->safe_psql('postgres', qq(
+ INSERT INTO tbl SELECT i, i FROM generate_series(1,$n) i
+));
+
+my $sum = $node->safe_psql('postgres', q(
+ SELECT SUM(j) AS sum FROM tbl
+));
+
+
+$node->pgbench(
+'--no-vacuum --client=15 --jobs=4 --exit-on-abort --transactions=5000',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REPACK CONCURRENTLY',
+{
+ 'concurrent_ops' => qq(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \\gset
+ \\if :gotlock
+ REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl USING INDEX test_idx;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ SELECT pg_advisory_unlock(42);
+ \\else
+ \\set num_a random(1, $n)
+ \\set num_b random(1, $n)
+ \\set diff random(1, 10000)
+ BEGIN;
+ UPDATE tbl SET j = j + :diff WHERE i = :num_a;
+ \\sleep 1 ms
+ UPDATE tbl SET j = j - :diff WHERE i = :num_b;
+ \\sleep 1 ms
+ COMMIT;
+
+ \\set v random(1, 300)
+ \\ if :v = 1
+ VACUUM ANALYZE pg_catalog.pg_class;
+ \\ endif
+
+ BEGIN
+ --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ ;
+ SELECT 1;
+ \\sleep 1 ms
+ SELECT COALESCE(SUM(j), 0) AS sum FROM tbl \\gset p_
+ \\if :p_sum != $sum
+ COMMIT;
+ SELECT (:p_sum) / 0;
+ \\endif
+
+ COMMIT;
+ \\endif
+ )
+});
+
+$node->stop;
+done_testing();
diff --git a/contrib/amcheck/t/008_repack_concurrently.pl b/contrib/amcheck/t/008_repack_concurrently.pl
new file mode 100644
index 00000000000..329e6d7ed47
--- /dev/null
+++ b/contrib/amcheck/t/008_repack_concurrently.pl
@@ -0,0 +1,106 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+max_worker_processes = 32
+));
+
+my $no_hot = int(rand(2));
+
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl(i SERIAL PRIMARY KEY, j int)));
+if ($no_hot)
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(j);));
+}
+else
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(i);));
+}
+
+# Load amcheck
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+
+my $sum = $node->safe_psql('postgres', q(
+ SELECT SUM(j) AS sum FROM tbl
+));
+
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE last_j START 1 INCREMENT 1;));
+
+
+$node->pgbench(
+'--no-vacuum --client=15 --jobs=4 --exit-on-abort --transactions=5000',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REPACK CONCURRENTLY',
+{
+ 'concurrent_ops' => qq(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \\gset
+ \\if :gotlock
+ REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl USING INDEX test_idx;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ SELECT pg_advisory_unlock(42);
+ \\else
+ SELECT pg_advisory_lock(43);
+ BEGIN;
+ INSERT INTO tbl(j) VALUES (nextval('last_j')) RETURNING j \\gset p_
+ COMMIT;
+ SELECT pg_advisory_unlock(43);
+ \\sleep 1 ms
+ \\set v random(1, 300)
+ \\ if :v = 1
+ VACUUM ANALYZE pg_catalog.pg_class;
+ \\ endif
+
+ BEGIN
+ --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ ;
+ SELECT 1;
+ \\sleep 1 ms
+ SELECT COUNT(*) AS count FROM tbl WHERE j <= :p_j \\gset p_
+ \\if :p_count != :p_j
+ COMMIT;
+ SELECT (:p_count) / 0;
+ \\endif
+
+ COMMIT;
+ \\endif
+ )
+});
+
+$node->stop;
+done_testing();
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 10:04 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-06 16:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-07 14:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-09 11:50 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-02-09 11:50 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > Indeed, the server log seems to indicate relationship to
> > VACUUM:
> > 2026-02-01 16:44:58.878 UTC autovacuum worker[22589] LOG: automatic vacuum of table "postgres.pg_catalog.pg_class": index scans: 1
>
> O, it's a good clue!
>
> I have added some vacuum calls for pg_class in a stress test - and now it fails much more often (check attachment).
>
> It is "ERROR: cache lookup failed for relation" - but I think it may share the cause with "attempted to overwrite invisible tuple.
I've just reported one issue [1] that causes this, but that does not seem to
be related to the "attempted to overwrite invisible tuple" error.
> See:
> https://cirrus-ci.com/build/4852126532239360 - with "Use multiple snapshots to copy the data."
> https://cirrus-ci.com/build/6429084491710464 - with "Use background worker to do logical decoding."
>
> But I am unable to reproduce the issue with only "Add CONCURRENTLY option to REPACK command."
> https://cirrus-ci.com/build/6467070524653568
No idea why VACUUM makes the issue happen too often. Maybe it's related to the
PD_ALL_VISIBLE flage, but I've got no detailed explanation. I also don't know
why it does not reproduce w/o the logical decoding worker.
Thanks again for your testing!
[1] https://www.postgresql.org/message-id/61812.1770637345%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-02 07:25 ` Antonin Houska <ah@cybertec.at>
2026-02-02 09:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-02 07:25 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > The 0006 part needs more work (definitely beyond PG 19).
>
> This is sad, because if you are in a situation then you need REPACK - pinning the horizon for too long may just finish your DB....
> And also, even with 0006 we still need to build indexes, which might pin it for long (even duration caused by a single index).
I suppose "to finish database" refers to XID wraparound - a problem that you
keep mentioning again and again. (Yes, the wraparound is a problem, but not
exactly a "final" state of the database.)
As far as I know, it's not uncommon for DBAs to use the pg_repack extension,
and this extension also restricts the progress of the VACUUM xmin horizon. Are
you sure that users do complain about having ended up in the XID wraparound
situation?
I don't really pay attention to pg_repack, but I do pay quite some attention
to the pg_squeeze extension (which I wrote and maintain). I recall that some
users were surprised by the amount of disk space consumed (as the earlier
versions of pg_squeeze were "too lazy" about WAL decoding), but I do not
recall a single complaint about pg_squeeze causing the XID wraparound
situation.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 07:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-02 09:18 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-02 09:18 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Helllo!
> I don't really pay attention to pg_repack, but I do pay quite some
attention
> to the pg_squeeze extension (which I wrote and maintain). I recall that
some
> users were surprised by the amount of disk space consumed (as the earlier
> versions of pg_squeeze were "too lazy" about WAL decoding), but I do not
> recall a single complaint about pg_squeeze causing the XID wraparound
> situation.
For "finish" I mean get out of space (in other write-heavy tables) or high
CPU usage (due to slow index scan checking the same rows again and again).
Also, you REPACK one table - and add a lot of bloat in others, in some
cases with negative impact in total.
But yes, agree about pg_squeeze here - if it is usable with such a long
transaction - REPACK CONCURRENTLY will be too.
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-06 15:08 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-15 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-02-06 15:08 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Here's a v33, where pg_repackdb has been removed, per multiple
discussions off-list. This is not a statement that we will never have
such a tool; just that for the time being we should not let ourselves be
distracted by it. If we have time to get something done about it for
v19, then it's fine to bring it back; but I kinda doubt it. I think
getting bits done including the addition of the CONCURRENTLY option
trumps that. We can add it in v20 if we decide to; no great loss.
I didn't include Antonin's 0006 "Use multiple snapshots to copy the
data" either. It seems a bit too experimental yet. I think it would be
good to have it in v19 also, but it seems less critical than the rest.
I haven't looked at Mihail's patch downthread either.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Los cuentos de hadas no dan al niño su primera idea sobre los monstruos.
Lo que le dan es su primera idea de la posible derrota del monstruo."
(G. K. Chesterton)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-15 16:03 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-15 16:03 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
Some feedback for v33.
> else if (pg_strcasecmp(cmd, "REPACK") == 0)
> cmdtype = PROGRESS_COMMAND_REPACK;
src/backend/utils/adt/pgstatfuncs.c:290
I think we need to add "CLUSTER" here too to avoid regression.
-----------
> ConditionVariablePrepareToSleep(&shared->cv);
> for (;;)
> {
> bool initialized;
>
> SpinLockAcquire(&shared->mutex);
> initialized = shared->initialized;
> SpinLockRelease(&shared->mutex);
src/backend/commands/cluster.c:3663
I think we should check GetBackgroundWorkerPid for worker status, to
not wait forever in case of some issue with the worker.
-----------
> /* Error queue. */
> shm_mq *error_mq;
src/backend/commands/cluster.c:210.
Not used anywhere.
-----------
> finish_heap_swap(old_table_oid, new_table_oid,
> is_system_catalog,
> false, /* swap_toast_by_content */
> false, true, false,
> frozenXid, cutoffMulti,
> relpersistence);
src/backend/commands/cluster.c
I think we should add comments for other boolean parameters.
-----------
> elog(ERROR, "Incomplete insert info.");
> elog(ERROR, "Incomplete update info.");
src/backend/replication/pgoutput_repack/pgoutput_repack.c:118,132
Should be non-capitalized?
-----------
> # Copyright (c) 2022-2024, PostgreSQL Global Development Group
src/backend/replication/pgoutput_repack/meson.build
2022-2026
-----------
> int newxcnt = 0;
src/backend/replication/logical/snapbuild.c:53
uint32 is better here.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-15 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-16 07:47 ` Antonin Houska <ah@cybertec.at>
2026-02-16 11:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-16 07:47 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Some feedback for v33.
Thanks again for your review!
> > else if (pg_strcasecmp(cmd, "REPACK") == 0)
> > cmdtype = PROGRESS_COMMAND_REPACK;
> src/backend/utils/adt/pgstatfuncs.c:290
>
> I think we need to add "CLUSTER" here too to avoid regression.
What kind of regression? There is no pg_stat_get_progress_info('CLUSTER') call
in system_views.sql.
> -----------
>
> > ConditionVariablePrepareToSleep(&shared->cv);
> > for (;;)
> > {
> > bool initialized;
> >
> > SpinLockAcquire(&shared->mutex);
> > initialized = shared->initialized;
> > SpinLockRelease(&shared->mutex);
> src/backend/commands/cluster.c:3663
>
> I think we should check GetBackgroundWorkerPid for worker status, to
> not wait forever in case of some issue with the worker.
ConditionVariableSleep() calls CHECK_FOR_INTERRUPTS(). That should process
error messages from the worker.
> -----------
>
> > /* Error queue. */
> > shm_mq *error_mq;
> src/backend/commands/cluster.c:210.
>
> Not used anywhere.
ok
> -----------
>
> > finish_heap_swap(old_table_oid, new_table_oid,
> > is_system_catalog,
> > false, /* swap_toast_by_content */
> > false, true, false,
> > frozenXid, cutoffMulti,
> > relpersistence);
> src/backend/commands/cluster.c
>
> I think we should add comments for other boolean parameters.
Perhaps, it wouldn't hurt.
> -----------
>
> > elog(ERROR, "Incomplete insert info.");
> > elog(ERROR, "Incomplete update info.");
> src/backend/replication/pgoutput_repack/pgoutput_repack.c:118,132
>
> Should be non-capitalized?
ok
> -----------
>
> > # Copyright (c) 2022-2024, PostgreSQL Global Development Group
> src/backend/replication/pgoutput_repack/meson.build
>
> 2022-2026
ok
> -----------
>
> > int newxcnt = 0;
> src/backend/replication/logical/snapbuild.c:53
>
> uint32 is better here.
This was already discussed:
https://www.postgresql.org/message-id/137668.1768235610%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-15 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-16 11:16 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 15:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-16 11:16 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
> What kind of regression? There is no pg_stat_get_progress_info('CLUSTER') call
> in system_views.sql.
I am about a direct call to that function (some monitoring tools may use it).
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-15 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-16 11:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-02-16 15:05 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 15:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-02-16 15:05 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Feb-16, Mihail Nikalayeu wrote:
> Hello!
>
> > What kind of regression? There is no pg_stat_get_progress_info('CLUSTER')
> > call in system_views.sql.
>
> I am about a direct call to that function (some monitoring tools may
> use it).
We don't promise compatibility using that interface, as far as I know.
Tools can update to accomodate the new definition -- assuming any exist
that actually do this.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-15 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-16 11:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 15:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-16 15:44 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-02-16 15:44 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Feb-16, Mihail Nikalayeu wrote:
>
> > Hello!
> >
> > > What kind of regression? There is no pg_stat_get_progress_info('CLUSTER')
> > > call in system_views.sql.
> >
> > I am about a direct call to that function (some monitoring tools may
> > use it).
>
> We don't promise compatibility using that interface, as far as I know.
At least this particular function is not mentioned in the user documentation,
AFAICS.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-16 19:56 ` Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-16 19:56 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> Here's a v33 ...
This is v33 rebased, pgindented and with Mihail's review [1] incorporated.
[1] https://www.postgresql.org/message-id/CADzfLwXJLypkRdpwapQr+pZQnv1-NvkJ9DpzWhNwudQgirCE0Q@mail.gmail...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-23 18:23 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-02-23 18:23 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Rebased again, due to recent conflicting changes. I also noticed a
leftover "See also" line for nonexistent pg_repackdb in ref/repack.sgml;
removed.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Tom: There seems to be something broken here.
Teodor: I'm in sackcloth and ashes... Fixed.
http://postgr.es/m/482D1632.8010507@sigaev.ru
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-24 18:29 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-02-24 18:29 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Feb-23, Alvaro Herrera wrote:
Looking at this function in pgoutput_repack.c:
> +/* Store concurrent data change. */
> +static void
> +store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
> + HeapTuple tuple)
> +{
[...] we have this:
> + size = VARHDRSZ + SizeOfConcurrentChange;
> +
> + /*
> + * ReorderBufferCommit() stores the TOAST chunks in its private memory
> + * context and frees them after having called apply_change(). Therefore
> + * we need flat copy (including TOAST) that we eventually copy into the
> + * memory context which is available to decode_concurrent_changes().
> + */
> + if (HeapTupleHasExternal(tuple))
> + {
> + /*
> + * toast_flatten_tuple_to_datum() might be more convenient but we
> + * don't want the decompression it does.
> + */
> + tuple = toast_flatten_tuple(tuple, dstate->tupdesc);
> + flattened = true;
> + }
> +
> + size += tuple->t_len;
> + if (size >= MaxAllocSize)
> + elog(ERROR, "Change is too big.");
> +
> + /* Construct the change. */
> + change_raw = (char *) palloc0(size);
> + SET_VARSIZE(change_raw, size);
I wonder if this isn't problematic with large tuples. If a row has some
very wide columns, each of which individually is less than 1 GB, then it
might happen that the sum of their sizes exceeds 1 GB, causing palloc()
to complain and abort the whole repack operation. This wouldn't be very
nice, so I think we need to address it somehow.
Another thing I'm not very keen on, is the fact that we have to memcpy()
the tuple contents a few lines below:
> + /*
> + * Copy the tuple.
> + *
> + * Note: change->tup_data.t_data must be fixed on retrieval!
> + */
> + memcpy(&change.tup_data, tuple, sizeof(HeapTupleData));
> + memcpy(dst, &change, SizeOfConcurrentChange);
> + dst += SizeOfConcurrentChange;
> + memcpy(dst, tuple->t_data, tuple->t_len);
> + /* Store as tuple of 1 bytea column. */
> + values[0] = PointerGetDatum(change_raw);
> + isnull[0] = false;
> + tuplestore_putvalues(dstate->tstore, dstate->tupdesc_change,
> + values, isnull);
To make matters worse, tuplestore_putvalues does a
heap_form_minimal_tuple() on this and copies the data again. This seems
pretty wasteful.
I think we need some new APIs to avoid all this copying. It appears
that it all starts with reorderbuffer doing something unhelpful with the
memory context of the TOAST chunks. Maybe we should address this by
"fixing" reorderbuffer so that it doesn't do this, instead of playing so
many games to cope.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-25 08:55 ` Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-02-25 08:55 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Feb-23, Alvaro Herrera wrote:
>
> Looking at this function in pgoutput_repack.c:
>
> > +/* Store concurrent data change. */
> > +static void
> > +store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
> > + HeapTuple tuple)
> > +{
>
> [...] we have this:
>
> > + size = VARHDRSZ + SizeOfConcurrentChange;
> > +
> > + /*
> > + * ReorderBufferCommit() stores the TOAST chunks in its private memory
> > + * context and frees them after having called apply_change(). Therefore
> > + * we need flat copy (including TOAST) that we eventually copy into the
> > + * memory context which is available to decode_concurrent_changes().
> > + */
> > + if (HeapTupleHasExternal(tuple))
> > + {
> > + /*
> > + * toast_flatten_tuple_to_datum() might be more convenient but we
> > + * don't want the decompression it does.
> > + */
> > + tuple = toast_flatten_tuple(tuple, dstate->tupdesc);
> > + flattened = true;
> > + }
> > +
> > + size += tuple->t_len;
> > + if (size >= MaxAllocSize)
> > + elog(ERROR, "Change is too big.");
> > +
> > + /* Construct the change. */
> > + change_raw = (char *) palloc0(size);
> > + SET_VARSIZE(change_raw, size);
In 0005 ("Use background worker to do logical decoding"), the function is a
bit simpler because here the decoding worker uses temporary file to send the
data to the REPACKing backend, rather than tuplestore. sharedtuplestore.h
would also work but I think we do not need its functionality, and AFAICS it
always writes the data into a file anyway (i.e. it does not use memory even if
the amount of data is small).
(Perhaps 0004 should use the file too, in case 0005 does not make it into PG
19.)
> I wonder if this isn't problematic with large tuples. If a row has some
> very wide columns, each of which individually is less than 1 GB, then it
> might happen that the sum of their sizes exceeds 1 GB, causing palloc()
> to complain and abort the whole repack operation. This wouldn't be very
> nice, so I think we need to address it somehow.
I agree.
> I think we need some new APIs to avoid all this copying. It appears
> that it all starts with reorderbuffer doing something unhelpful with the
> memory context of the TOAST chunks. Maybe we should address this by
> "fixing" reorderbuffer so that it doesn't do this, instead of playing so
> many games to cope.
What I see is that reorderbuffer.c collects the TOAST pointers
(ReorderBufferToastAppendChunk) and then, before passing the tuple to the
output plugin, it copies the TOAST chunks referenced by the tuple to memory
and replaces the "on-disk" TOAST pointers in the tuple with "external
indirect" ones, pointing to the in-memory TOAST chunks
(ReorderBufferToastReplace).
For REPACK, I suggest a variant of toast_flatten_tuple() that writes the
output to a file, and a corresponding function that reads it while allocating
separate chunks of memory for the individual TOASTed attributes - the restored
tuple would reference the chunks using the "external indirect" TOAST pointers,
as if it had been processed by ReorderBufferToastReplace(). Does that make
sense to you?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-25 13:54 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-02-25 13:54 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Feb-25, Antonin Houska wrote:
> In 0005 ("Use background worker to do logical decoding"), the function is a
> bit simpler because here the decoding worker uses temporary file to send the
> data to the REPACKing backend, rather than tuplestore.
Ah, that patch changes the implementation rather substantially then; I
didn't realize that because I haven't been looking at it yet, as I
wanted to have a good idea of the status of 0004 before proceeding
further with the next one. However, given that these changes are so
extensive, I think it might be better to review 0004+0005 squashed as
one unit, rather than separately.
> For REPACK, I suggest a variant of toast_flatten_tuple() that writes the
> output to a file, and a corresponding function that reads it while allocating
> separate chunks of memory for the individual TOASTed attributes - the restored
> tuple would reference the chunks using the "external indirect" TOAST pointers,
> as if it had been processed by ReorderBufferToastReplace(). Does that make
> sense to you?
Hmm, so on the apply side when reading the file, we would first reach
each toast attribute value, which we know to insert directly to the
toast table (keeping track of each individually toast pointer as we do
so); then we reach the heap tuple itself, we [... somehow ...] interpret
these external indirect toast pointers and substitute the toast pointers
that we created. So we never have to construct the entire tuple, or
indeed do anything else with the toasted values other than insert them
into the toast table.
Actually, can't we simply insert the toasted values directly in the
decoding worker into the new toast table? That could save a lot of
writing to the file, since we only save the raw heap tuples with no
toasted contents; but it's not clear to me that this is valid. (And we
might create extra bloat if a tuple is inserted and later deleted
concurrently with the repack; but that would happen with the original
approach as well, no?)
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"We’ve narrowed the problem down to the customer’s pants being in a situation
of vigorous combustion" (Robert Haas, Postgres expert extraordinaire)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-25 16:03 ` Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-25 16:03 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Feb-25, Antonin Houska wrote:
> > For REPACK, I suggest a variant of toast_flatten_tuple() that writes the
> > output to a file, and a corresponding function that reads it while allocating
> > separate chunks of memory for the individual TOASTed attributes - the restored
> > tuple would reference the chunks using the "external indirect" TOAST pointers,
> > as if it had been processed by ReorderBufferToastReplace(). Does that make
> > sense to you?
>
> Hmm, so on the apply side when reading the file, we would first reach
> each toast attribute value, which we know to insert directly to the
> toast table (keeping track of each individually toast pointer as we do
> so); then we reach the heap tuple itself, we [... somehow ...] interpret
> these external indirect toast pointers and substitute the toast pointers
> that we created. So we never have to construct the entire tuple, or
> indeed do anything else with the toasted values other than insert them
> into the toast table.
Yes, that's what I mean.
> Actually, can't we simply insert the toasted values directly in the
> decoding worker into the new toast table? That could save a lot of
> writing to the file, since we only save the raw heap tuples with no
> toasted contents; but it's not clear to me that this is valid. (And we
> might create extra bloat if a tuple is inserted and later deleted
> concurrently with the repack; but that would happen with the original
> approach as well, no?)
The problem I see here is that for UPDATE you need the old tuple to determine
if its TOAST value should be deleted or if the new tuple should reuse it -
this is how I understand toast_tuple_init(). So the worker would have to store
all the changes somewhere temporarily until it can fully apply the changes
(i.e. until the initial copy and index build is complete).
Besides that, if the worker had to switch between the past (for the decoding)
and present (for the TOAST operations), it would have to invalidate system
caches repeatedly. 0004 does that, but 0005 makes that unnecessary. (I don't
know if the repeated cache invalidation would be a serious performance
problem, but from coding perspective I find it more convenient if the worker
only deals with decoding and does not have to do this time travel and
invalidations at all.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-25 16:25 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-02-25 16:25 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Feb-25, Antonin Houska wrote:
> > Hmm, so on the apply side when reading the file, we would first reach
> > each toast attribute value, which we know to insert directly to the
> > toast table (keeping track of each individually toast pointer as we do
> > so); then we reach the heap tuple itself, we [... somehow ...] interpret
> > these external indirect toast pointers and substitute the toast pointers
> > that we created. So we never have to construct the entire tuple, or
> > indeed do anything else with the toasted values other than insert them
> > into the toast table.
>
> Yes, that's what I mean.
Makes sense. Would you be able to try and implement that?
> The problem I see here is that for UPDATE you need the old tuple to determine
> if its TOAST value should be deleted or if the new tuple should reuse it -
> this is how I understand toast_tuple_init(). So the worker would have to store
> all the changes somewhere temporarily until it can fully apply the changes
> (i.e. until the initial copy and index build is complete).
Ah, you're right, that won't work.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Tom: There seems to be something broken here.
Teodor: I'm in sackcloth and ashes... Fixed.
http://postgr.es/m/482D1632.8010507@sigaev.ru
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-02-25 19:04 ` Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-25 19:04 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Feb-25, Antonin Houska wrote:
>
> > > Hmm, so on the apply side when reading the file, we would first reach
> > > each toast attribute value, which we know to insert directly to the
> > > toast table (keeping track of each individually toast pointer as we do
> > > so); then we reach the heap tuple itself, we [... somehow ...] interpret
> > > these external indirect toast pointers and substitute the toast pointers
> > > that we created. So we never have to construct the entire tuple, or
> > > indeed do anything else with the toasted values other than insert them
> > > into the toast table.
> >
> > Yes, that's what I mean.
>
> Makes sense. Would you be able to try and implement that?
Yes, I'll try in the following days.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-27 18:38 ` Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-02-27 18:38 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > On 2026-Feb-25, Antonin Houska wrote:
> >
> > > > Hmm, so on the apply side when reading the file, we would first reach
> > > > each toast attribute value, which we know to insert directly to the
> > > > toast table (keeping track of each individually toast pointer as we do
> > > > so); then we reach the heap tuple itself, we [... somehow ...] interpret
> > > > these external indirect toast pointers and substitute the toast pointers
> > > > that we created. So we never have to construct the entire tuple, or
> > > > indeed do anything else with the toasted values other than insert them
> > > > into the toast table.
> > >
> > > Yes, that's what I mean.
> >
> > Makes sense. Would you be able to try and implement that?
>
> Yes, I'll try in the following days.
My proposal is in the 0005 part of this series - a separate diff just for now,
to make review easier (the diff also contains a few lines of related
refactoring, I hope it's not too disturbing).
The changes we had in 0005 ("Use background worker ...") so far are now in
0004 ("Add CONCURRENTLY option ...").
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-28 15:16 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-02-28 15:16 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
Some review comments:
------------
> attrs = palloc0_array(Datum, desc->natts);
> isnull = palloc0_array(bool, desc->natts);
It looks like there is a memory leak with those arrays.
------------
> # TOAST pointer, wich we need to update
typo
------------
> ident_idx = RelationGetReplicaIndex(rel);
> if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
check_repack_concurrently_requirements uses rd_pkindex as fallback.
But rebuild_relation_finish_concurrent does not contain such logic:
> ident_idx_old = RelationGetReplicaIndex(OldHeap);
------------
> >
> > > ConditionVariablePrepareToSleep(&shared->cv);
> > > for (;;)
> > > {
> > > bool initialized;
> > >
> > > SpinLockAcquire(&shared->mutex);
> > > initialized = shared->initialized;
> > > SpinLockRelease(&shared->mutex);
> > src/backend/commands/cluster.c:3663
> >
> > I think we should check GetBackgroundWorkerPid for worker status, to
> > not wait forever in case of some issue with the worker.
> ConditionVariableSleep() calls CHECK_FOR_INTERRUPTS(). That should process
> error messages from the worker.
Hm, yes, and RepackWorkerShutdown will detach the queue. But
ProcessRepackMessages does not react somehow to SHM_MQ_DETACHED - just
ignores. Or am I missing something?
And looks like it applies to all wait-loops related to repack.
------------
> build_identity_key
> ....
> n = ident_idx->indnatts;
Should we use indnkeyatts here?
------------
> build_identity_key
> ....
> entry->sk_collation = att->attcollation;
Should we use index collation (not heap) here?
entry->sk_collation = ident_idx_rel->rd_indcollation[i];
------------
> SnapBuildInitialSnapshotForRepack
What is about to add defensive checks like SnapBuildInitialSnapshot does?
> if (!builder->committed.includes_all_transactions)
> elog(ERROR, "cannot build an initial slot snapshot, not all transactions are monitored anymore");
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-03-02 14:39 ` Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-02 14:39 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Some review comments:
Thanks again!
> ------------
>
> > attrs = palloc0_array(Datum, desc->natts);
> > isnull = palloc0_array(bool, desc->natts);
>
> It looks like there is a memory leak with those arrays.
I suppose you mean store_change(). Yes, I tried to free the individual chunks
and forgot these. The next version uses a new, per-change memory context.
> > ident_idx = RelationGetReplicaIndex(rel);
> > if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
>
> check_repack_concurrently_requirements uses rd_pkindex as fallback.
>
> But rebuild_relation_finish_concurrent does not contain such logic:
>
> > ident_idx_old = RelationGetReplicaIndex(OldHeap);
Good point. I added a new argument to rebuild_relation_finish_concurrent() so
that the identity index is only determined once.
> ------------
>
> > >
> > > > ConditionVariablePrepareToSleep(&shared->cv);
> > > > for (;;)
> > > > {
> > > > bool initialized;
> > > >
> > > > SpinLockAcquire(&shared->mutex);
> > > > initialized = shared->initialized;
> > > > SpinLockRelease(&shared->mutex);
> > > src/backend/commands/cluster.c:3663
> > >
> > > I think we should check GetBackgroundWorkerPid for worker status, to
> > > not wait forever in case of some issue with the worker.
>
> > ConditionVariableSleep() calls CHECK_FOR_INTERRUPTS(). That should process
> > error messages from the worker.
>
> Hm, yes, and RepackWorkerShutdown will detach the queue. But
> ProcessRepackMessages does not react somehow to SHM_MQ_DETACHED - just
> ignores. Or am I missing something?
On ERROR / FATAL, RepackWorkerShutdown() should send the message before
detaching. elog.c does it via send_message_to_frontend(), due to the previous
call of pq_redirect_to_shm_mq() in RepackWorkerMain(). ProcessRepackMessages()
then only needs to care about the message, not about the worker's detaching.
> ------------
>
> > build_identity_key
> > ....
> > n = ident_idx->indnatts;
>
> Should we use indnkeyatts here?
Definitely. I missed the addition of the INCLUDE columns feature during
maintenance of pg_squeeze, and copied the bug to REPACK. Fixed.
> ------------
>
> > build_identity_key
> > ....
> > entry->sk_collation = att->attcollation;
>
> Should we use index collation (not heap) here?
> entry->sk_collation = ident_idx_rel->rd_indcollation[i];
AFAIC they should be equal, but what you propose simplifies the code a
bit. Done.
> ------------
>
> > SnapBuildInitialSnapshotForRepack
>
> What is about to add defensive checks like SnapBuildInitialSnapshot does?
>
> > if (!builder->committed.includes_all_transactions)
> > elog(ERROR, "cannot build an initial slot snapshot, not all transactions are monitored anymore");
Initially I added a header comment (XXX) to
SnapBuildInitialSnapshotForRepack() saying that some of the checks, including
this one, could be adopted. The checks were problematic in the backend
executing REPACK. However, they appear to be fine if the code is executed by
the logical decoding worker. So what I'm trying now is to add a new argument
"repack" to SnapBuildInitialSnapshot() and remove the original 0003 diff
("Move conversion of a historic to MVCC snapshot to a separate function.")
from the series altogether.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-02 17:23 ` Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-02 17:23 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > ------------
> >
> > > attrs = palloc0_array(Datum, desc->natts);
> > > isnull = palloc0_array(bool, desc->natts);
> >
> > It looks like there is a memory leak with those arrays.
>
> I suppose you mean store_change(). Yes, I tried to free the individual chunks
> and forgot these. The next version uses a new, per-change memory context.
I realize now that I forgot to reset the context at the end of the
function. I'll fix that in the next version (which will probably be posted
rather soon).
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-05 19:06 ` Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-05 19:06 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Antonin Houska <ah@cybertec.at> wrote:
>
> > Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > > ------------
> > >
> > > > attrs = palloc0_array(Datum, desc->natts);
> > > > isnull = palloc0_array(bool, desc->natts);
> > >
> > > It looks like there is a memory leak with those arrays.
> >
> > I suppose you mean store_change(). Yes, I tried to free the individual chunks
> > and forgot these. The next version uses a new, per-change memory context.
>
> I realize now that I forgot to reset the context at the end of the
> function. I'll fix that in the next version (which will probably be posted
> rather soon).
This is it. One more diff added, to engage BulkInsertState.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-07 14:28 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-03-07 14:28 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Thu, Mar 5, 2026 at 8:06 PM Antonin Houska <ah@cybertec.at> wrote:
> This is it. One more diff added, to engage BulkInsertState.
Few more comments:
> * In case functions in the index need the active snapshot and caller
> * hasn't set one.
Looks like a stale comment.
-----------
> recheck = ExecInsertIndexTuples(iistate->rri,
> iistate->estate,
> 0,
> index_slot,
> NIL, NULL);
Such code in apply_concurrent_update and apply_concurrent_insert.
AFAIU we need to call ResetPerTupleExprContext(iistate->estate); after
list_free(recheck); to avoid small per-row memory leak.
-----------------
In find_target_tuple index_rescan is called before setting the
sk_argument - it works, but feels to be incorrect for the common case.
We should call it once entry->sk_argument is ready.
-----------------
I put possible fixes into the attached patch.
Mikhail.
Attachments:
[application/octet-stream] nocfbot-review_changes.patch (2.4K, ../../CADzfLwVVxC-Ptp98B=gTQgT7uuWY1KnQDLBhGkP1NS3iz-AYcA@mail.gmail.com/2-nocfbot-review_changes.patch)
download | inline diff:
Subject: [PATCH] review changes
---
Index: src/backend/commands/cluster.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
--- a/src/backend/commands/cluster.c (revision 5936ff16dfe4765ef0dd6e18be401aefcf120d36)
+++ b/src/backend/commands/cluster.c (date 1772889987377)
@@ -2971,9 +2971,6 @@
/*
* Update indexes.
- *
- * In case functions in the index need the active snapshot and caller
- * hasn't set one.
*/
ExecStoreHeapTuple(tup, index_slot, false);
recheck = ExecInsertIndexTuples(iistate->rri,
@@ -2988,6 +2985,7 @@
* committed.)
*/
list_free(recheck);
+ ResetPerTupleExprContext(iistate->estate);
pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED, 1);
}
@@ -3032,6 +3030,7 @@
index_slot,
NIL, NULL);
list_free(recheck);
+ ResetPerTupleExprContext(iistate->estate);
}
pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_UPDATED, 1);
@@ -3260,25 +3259,24 @@
scan = index_beginscan(rel, ident_index, GetActiveSnapshot(),
NULL, dest->ident_key_nentries, 0);
- /*
- * Scan key is passed by caller, so it does not have to be constructed
- * multiple times. Key entries have all fields initialized, except for
- * sk_argument.
- */
- index_rescan(scan, dest->ident_key, dest->ident_key_nentries, NULL, 0);
-
/* Info needed to retrieve key values from heap tuple. */
ident_form = ident_index->rd_index;
ident_indkey = &ident_form->indkey;
- /* Use the incoming tuple to finalize the scan key. */
- for (int i = 0; i < scan->numberOfKeys; i++)
+ /*
+ * Scan key is passed by caller, so it does not have to be constructed
+ * multiple times. Key entries have all fields initialized, except for
+ * sk_argument.
+ *
+ * Use the incoming tuple to finalize the scan key.
+ */
+ for (int i = 0; i < dest->ident_key_nentries; i++)
{
ScanKey entry;
bool isnull;
int16 attno_heap;
- entry = &scan->keyData[i];
+ entry = &dest->ident_key[i];
attno_heap = ident_indkey->values[i];
entry->sk_argument = heap_getattr(tup_key,
attno_heap,
@@ -3286,6 +3284,8 @@
&isnull);
Assert(!isnull);
}
+
+ index_rescan(scan, dest->ident_key, dest->ident_key_nentries, NULL, 0);
if (index_getnext_slot(scan, ForwardScanDirection, ident_slot))
{
bool shouldFree;
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-03-09 10:43 ` Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-09 10:43 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> On Thu, Mar 5, 2026 at 8:06 PM Antonin Houska <ah@cybertec.at> wrote:
> > This is it. One more diff added, to engage BulkInsertState.
>
> Few more comments:
> ...
> I put possible fixes into the attached patch.
All LGTM, included in the next version. Thanks!
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-10 19:24 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-10 20:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-10 19:24 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
I have just pushed 0001 with some additional changes. Here's a rebase
of the next ones; no changes other than fixing the conflicts.
I'm seeing this warning caused by 0004, which I think is also being
reported in CI
https://cirrus-ci.com/task/6606871575920640
[281/1134] Compiling C object src/backend/postgres_lib.a.p/commands_cluster.c.o
In file included from ../../source/repack/src/include/access/htup_details.h:22,
from ../../source/repack/src/include/access/relscan.h:17,
from ../../source/repack/src/include/access/heapam.h:19,
from ../../source/repack/src/backend/commands/cluster.c:37:
In function ‘VARSIZE_ANY’,
inlined from ‘restore_tuple’ at ../../source/repack/src/backend/commands/cluster.c:3129:18,
inlined from ‘apply_concurrent_changes’ at ../../source/repack/src/backend/commands/cluster.c:2915:9,
inlined from ‘process_concurrent_changes’ at ../../source/repack/src/backend/commands/cluster.c:3386:2:
../../source/repack/src/include/varatt.h:243:51: warning: array subscript ‘varattrib_4b[0]’ is partly outside array bounds of ‘varlena[1]’ [-Warray-bounds=]
243 | ((((const varattrib_4b *) (PTR))->va_4byte.va_header >> 2) & 0x3FFFFFFF)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
../../source/repack/src/include/varatt.h:467:24: note: in expansion of macro ‘VARSIZE_4B’
467 | return VARSIZE_4B(PTR);
| ^~~~~~~~~~
../../source/repack/src/backend/commands/cluster.c: In function ‘process_concurrent_changes’:
../../source/repack/src/backend/commands/cluster.c:3121:33: note: object ‘varhdr’ of size 4
3121 | varlena varhdr;
| ^~~~~~
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-10 20:05 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 01:13 ` RE: Adding REPACK [concurrently] Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-10 20:05 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-10, Alvaro Herrera wrote:
> I'm seeing this warning caused by 0004
(Caused by 0003, not 0004, sorry)
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-10 20:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-11 01:13 ` Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>
2026-03-11 16:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 18:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Shinoda, Noriyoshi (PSD Japan FSI) @ 2026-03-11 01:13 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi, Hackers.
Thanks for developing this great feature.
The committed documentation for the pg_stat_progress_repack view was missing the explanation for the "command" column.
I've attached a small patch for monitoring.sgml.
Regards,
Noriyoshi Shinoda
-----Original Message-----
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Sent: Wednesday, March 11, 2026 5:05 AM
To: Antonin Houska <ah@cybertec.at>
Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Subject: Re: Adding REPACK [concurrently]
On 2026-Mar-10, Alvaro Herrera wrote:
> I'm seeing this warning caused by 0004
(Caused by 0003, not 0004, sorry)
--
Álvaro Herrera Breisgau, Deutschland — https://urldefense.com/v3/__https://www.EnterpriseDB.com/__;!!NpxR!jyoaRRv_itWrU5kI6TI2ew-oOOvVeDRnH...
Attachments:
[application/octet-stream] pg_stat_progress_repack_doc_v1.diff (713B, ../../LV8PR84MB37870F0F35EF2E8CB99768CBEE47A@LV8PR84MB3787.NAMPRD84.PROD.OUTLOOK.COM/2-pg_stat_progress_repack_doc_v1.diff)
download | inline diff:
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index cc014564c97..9f072c9f092 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -6805,6 +6805,15 @@ FROM pg_stat_get_backend_idset() AS backendid;
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>command</structfield> <type>text</type>
+ </para>
+ <para>
+ The command that is running. Either REPACK or VACUUM FULL.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>phase</structfield> <type>text</type>
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-10 20:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 01:13 ` RE: Adding REPACK [concurrently] Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>
@ 2026-03-11 16:06 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-11 16:06 UTC (permalink / raw)
To: Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com> wrote:
> The committed documentation for the pg_stat_progress_repack view was missing the explanation for the "command" column.
> I've attached a small patch for monitoring.sgml.
LGTM, thanks!
(I haven't added it to the next version, which now only deals with adding the
CONCURRENTLY option.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-10 20:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 01:13 ` RE: Adding REPACK [concurrently] Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>
@ 2026-03-12 18:25 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-12 19:15 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-12 18:25 UTC (permalink / raw)
To: Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-11, Shinoda, Noriyoshi (PSD Japan FSI) wrote:
> Hi, Hackers.
> Thanks for developing this great feature.
> The committed documentation for the pg_stat_progress_repack view was missing the explanation for the "command" column.
> I've attached a small patch for monitoring.sgml.
Thanks, pushed. I also made some edits to the description of the
pg_stat_progress_cluster view while there. I've been wondering for some
time whether we should drop that one for pg19.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"People get annoyed when you try to debug them." (Larry Wall ...)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-10 20:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 01:13 ` RE: Adding REPACK [concurrently] Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>
2026-03-12 18:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-12 19:15 ` Robert Treat <rob@xzilla.net>
2026-03-12 19:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Robert Treat @ 2026-03-12 19:15 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Thu, Mar 12, 2026 at 2:25 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Mar-11, Shinoda, Noriyoshi (PSD Japan FSI) wrote:
>
> > Hi, Hackers.
> > Thanks for developing this great feature.
> > The committed documentation for the pg_stat_progress_repack view was missing the explanation for the "command" column.
> > I've attached a small patch for monitoring.sgml.
>
> Thanks, pushed. I also made some edits to the description of the
> pg_stat_progress_cluster view while there. I've been wondering for some
> time whether we should drop that one for pg19.
>
ISTM the user facing docs refer to cluster as an "obsolete" variant /
spelling, rather than something marked as deprecated. This feels like
it is meant to imply that the old functionality is not planned for
removal in some future release (ie. deprecated), but that you may find
that certain bits of support for it are already removed/broken. If
that was the intention, I guess it gives justification to remove it
now; that said it does seem rather unfriendly to not give any kind of
bridge release to get from one side to the other, so I think the ideal
would be to keep it for v19 and remove it in v20.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-10 20:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 01:13 ` RE: Adding REPACK [concurrently] Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>
2026-03-12 18:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-12 19:15 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2026-03-12 19:45 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-12 19:45 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On 2026-Mar-12, Robert Treat wrote:
> ISTM the user facing docs refer to cluster as an "obsolete" variant /
> spelling, rather than something marked as deprecated. This feels like
> it is meant to imply that the old functionality is not planned for
> removal in some future release (ie. deprecated), but that you may find
> that certain bits of support for it are already removed/broken. If
> that was the intention, I guess it gives justification to remove it
> now; that said it does seem rather unfriendly to not give any kind of
> bridge release to get from one side to the other, so I think the ideal
> would be to keep it for v19 and remove it in v20.
Yeah, I think "obsolete" rather than "deprecated" is correct, because I
don't see us removing either CLUSTER or VACUUM FULL. On the other hand,
if you use pg_stat_progress_cluster while running REPACK CONCURRENTLY,
then the resulting report might not fully make sense.
But I agree it makes sense to keep the view in place until ... some
later release. (Maybe v20, but also maybe a little later wouldn't
hurt.)
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"On the other flipper, one wrong move and we're Fatal Exceptions"
(T.U.X.: Term Unit X - http://www.thelinuxreview.com/TUX/)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-11 15:54 ` Antonin Houska <ah@cybertec.at>
2026-03-11 17:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-11 15:54 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> I have just pushed 0001 with some additional changes.
Thanks!
> Here's a rebase of the next ones; no changes other than fixing the
> conflicts.
>
> I'm seeing this warning caused by 0004, which I think is also being
> reported in CI
> https://cirrus-ci.com/task/6606871575920640
>
> [281/1134] Compiling C object src/backend/postgres_lib.a.p/commands_cluster.c.o
> In file included from ../../source/repack/src/include/access/htup_details.h:22,
> from ../../source/repack/src/include/access/relscan.h:17,
> from ../../source/repack/src/include/access/heapam.h:19,
> from ../../source/repack/src/backend/commands/cluster.c:37:
> In function ‘VARSIZE_ANY’,
> inlined from ‘restore_tuple’ at ../../source/repack/src/backend/commands/cluster.c:3129:18,
> inlined from ‘apply_concurrent_changes’ at ../../source/repack/src/backend/commands/cluster.c:2915:9,
> inlined from ‘process_concurrent_changes’ at ../../source/repack/src/backend/commands/cluster.c:3386:2:
> ../../source/repack/src/include/varatt.h:243:51: warning: array subscript ‘varattrib_4b[0]’ is partly outside array bounds of ‘varlena[1]’ [-Warray-bounds=]
> 243 | ((((const varattrib_4b *) (PTR))->va_4byte.va_header >> 2) & 0x3FFFFFFF)
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> ../../source/repack/src/include/varatt.h:467:24: note: in expansion of macro ‘VARSIZE_4B’
> 467 | return VARSIZE_4B(PTR);
> | ^~~~~~~~~~
> ../../source/repack/src/backend/commands/cluster.c: In function ‘process_concurrent_changes’:
> ../../source/repack/src/backend/commands/cluster.c:3121:33: note: object ‘varhdr’ of size 4
> 3121 | varlena varhdr;
> | ^~~~~~
I'm not sure it can be fixed nicely in the REPACK (CONCURRENTLY) patch. I
think the problem is that, in the current tree, VARSIZE_ANY() is used in such
a way that the compiler cannot check the "array bounds". The restore_tuple()
function is special in that it uses VARSIZE_ANY() to check a variable
allocated from the stack, so the compiler can check the size.
I'm trying to fix that in a new diff 0002 - the point is that VARSIZE_ANY()
should not need to dereference a pointer to varattrib_4b, since the size
information is always located at the beginning of the structure. Maybe you
have better idea.
Besides that, I've done a related change in 0003 (now 0004, due to the new
diff):
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 77e301b7c63..8b5571374d0 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -3118,7 +3118,7 @@ restore_tuple(BufFile *file, Relation relation, MemoryContext cxt)
BufFileReadExact(file, &natt_ext, sizeof(natt_ext));
for (int i = 0; i < natt_ext; i++)
{
- varlena varhdr;
+ alignas(uint32) varlena varhdr;
char *ext_val;
Size ext_val_size;
And also this one in the same file, to suppress another compiler warning
(occuring when configured w/o --enable-cassert):
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
index 707940c1127..90f3a8975b9 100644
--- a/src/backend/replication/pgoutput_repack/pgoutput_repack.c
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -93,12 +93,9 @@ static void
plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Relation relation, ReorderBufferChange *change)
{
- RepackDecodingState *dstate;
-
- dstate = (RepackDecodingState *) ctx->output_writer_private;
-
/* Changes of other relation should not have been decoded. */
- Assert(RelationGetRelid(relation) == dstate->relid);
+ Assert(RelationGetRelid(relation) ==
+ ((RepackDecodingState *) ctx->output_writer_private)->relid);
/* Decode entry depending on its type */
switch (change->action)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-11 17:07 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-11 17:07 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> I'm trying to fix that in a new diff 0002 - the point is that VARSIZE_ANY()
> should not need to dereference a pointer to varattrib_4b, since the size
> information is always located at the beginning of the structure. Maybe you
> have better idea.
I realize now that I ended up with VARSIZE_4B() that does not really need the
new field in varattrib_4b. Simply thinko.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-12 19:31 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-12 19:31 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-11, Antonin Houska wrote:
> I'm not sure it can be fixed nicely in the REPACK (CONCURRENTLY) patch. I
> think the problem is that, in the current tree, VARSIZE_ANY() is used in such
> a way that the compiler cannot check the "array bounds". The restore_tuple()
> function is special in that it uses VARSIZE_ANY() to check a variable
> allocated from the stack, so the compiler can check the size.
>
> I'm trying to fix that in a new diff 0002 - the point is that VARSIZE_ANY()
> should not need to dereference a pointer to varattrib_4b, since the size
> information is always located at the beginning of the structure. Maybe you
> have better idea.
I have no immediate ideas on this.
I offer the following rather trivial fixup diffs, which I think should
be mostly be for 0002.
Other trivial things I'd like to change, if you don't mind,
- the name pgoutput_repack sounds wrong to me. I would rather say
rpck_output, repack_output, repack_plugin, ... or something. I don't
understand where the suffix "output" comes from in the name
"pgoutput", but it sounds like arbitrary nonsense to me.
- I would rename the routines in that file to also have the name
"repack", probably as prefixes.
One thing we need and is rather not trivial, is to go through the table
AM interface rather than calling heapam.c routines directly. I'm
working on this now and will present a patch later.
Another thing I noticed while going over the code was that we seem to
spill whole tuples even for things like the old tuple of an UPDATE and
for DELETE, but unless I misunderstand how this works, these shouldn't
be necessary: we just need the replica identity so that we can locate
the tuple to operate on. Especially for tuples that contain large
toasted attributes, this is likely important.
It may make sense to use the TupleTableSlot interface rather than
HeapTuple for everything. I'm not really sure about this though.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Most hackers will be perfectly comfortable conceptualizing users as entropy
sources, so let's move on." (Nathaniel Smith)
https://mail.gnu.org/archive/html/monotone-devel/2007-01/msg00080.html
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-13 08:11 ` Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-13 08:11 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> I offer the following rather trivial fixup diffs, which I think should
> be mostly be for 0002.
Thanks, just a few comments:
* 0002 - I didn't add the 'repack_' prefix because the function is static, but
realize now that it might be useful from the code browsing perspective.
* 0008 - It's possible during query execution, when you know exactly which
attributes you need to fetch from the tuple. However in store_change(), we
don't know which attributes are external (indirect) without deforming them
all.
> Other trivial things I'd like to change, if you don't mind,
> - the name pgoutput_repack sounds wrong to me. I would rather say
> rpck_output, repack_output, repack_plugin, ... or something. I don't
> understand where the suffix "output" comes from in the name
> "pgoutput", but it sounds like arbitrary nonsense to me.
No strong preference here, except that I don't like "rpck_...". How about
pgoutput/repack.c? I think I tried to put the plugin into the existing
"pgoutput" directory at some point, but don't remember why I eventually
rejected that approach.
> - I would rename the routines in that file to also have the name
> "repack", probably as prefixes.
ok
> One thing we need and is rather not trivial, is to go through the table
> AM interface rather than calling heapam.c routines directly. I'm
> working on this now and will present a patch later.
It occurred to me too, at least because copy_table_data() calls
table_relation_copy_for_cluster() rather than
heapam_relation_copy_for_cluster(). Thanks for working on it.
> Another thing I noticed while going over the code was that we seem to
> spill whole tuples even for things like the old tuple of an UPDATE and
> for DELETE, but unless I misunderstand how this works, these shouldn't
> be necessary: we just need the replica identity so that we can locate
> the tuple to operate on. Especially for tuples that contain large
> toasted attributes, this is likely important.
I think we don't need to care, as both heap_update() and heap_delete() call
ExtractReplicaIdentity(). That returns a real tuple, but it only contains the
attributes constituting the replica identity. The other ones are set to
NULL.
> It may make sense to use the TupleTableSlot interface rather than
> HeapTuple for everything. I'm not really sure about this though.
Isn't this part of the adoption of table AM? For example, table_tuple_insert()
accepts tuple slot rather than heap tuple.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-16 09:13 ` Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-16 09:13 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > - the name pgoutput_repack sounds wrong to me. I would rather say
> > rpck_output, repack_output, repack_plugin, ... or something. I don't
> > understand where the suffix "output" comes from in the name
> > "pgoutput", but it sounds like arbitrary nonsense to me.
>
> No strong preference here ...
One more problem related to the replication slot is that, due to the call of
CheckSlotPermissions() in setup_logical_decoding(), REPLICATION privilege is
required for REPACK (CONCURRENTLY) to run. That's not too user-friendly.
I think the reason to require the REPLICATION privilege is that, in generic
case, the output plugin can access data of any table in the database. However
REPACK uses one particular plugin and that plugin only decodes changes of one
particular table. Thus I think we don't really need to call
CheckSlotPermissions(). Do I seem to miss something?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-16 12:21 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-16 12:21 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-16, Antonin Houska wrote:
> One more problem related to the replication slot is that, due to the call of
> CheckSlotPermissions() in setup_logical_decoding(), REPLICATION privilege is
> required for REPACK (CONCURRENTLY) to run. That's not too user-friendly.
>
> I think the reason to require the REPLICATION privilege is that, in generic
> case, the output plugin can access data of any table in the database. However
> REPACK uses one particular plugin and that plugin only decodes changes of one
> particular table. Thus I think we don't really need to call
> CheckSlotPermissions(). Do I seem to miss something?
Yeah, I don't think it makes sense to require REPLICATION privilege to
run REPACK.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Oh, great altar of passive entertainment, bestow upon me thy discordant images
at such speed as to render linear thought impossible" (Calvin a la TV)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-16 14:21 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Matthias van de Meent @ 2026-03-16 14:21 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, 16 Mar 2026 at 13:21, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Mar-16, Antonin Houska wrote:
>
> > One more problem related to the replication slot is that, due to the call of
> > CheckSlotPermissions() in setup_logical_decoding(), REPLICATION privilege is
> > required for REPACK (CONCURRENTLY) to run. That's not too user-friendly.
> >
> > I think the reason to require the REPLICATION privilege is that, in generic
> > case, the output plugin can access data of any table in the database. However
> > REPACK uses one particular plugin and that plugin only decodes changes of one
> > particular table. Thus I think we don't really need to call
> > CheckSlotPermissions(). Do I seem to miss something?
>
> Yeah, I don't think it makes sense to require REPLICATION privilege to
> run REPACK.
I don't think it makes sense to allow just any table owner to modify
the effective_wal_level GUC; which is what the effect would be of
removing the REPLICATION requirement from roles that want to REPACK
CONCURRENTLY -- and in doing so create logical slots, which increase
effective_wal_level to logical.
Creating a replication slot requires REPLICATION privilege, because it
consumes a (possibly very) limited server resource that can't be
increased without restart and because it impacts other backends' WAL
performance through effective_wal_level. Allowing users to consume
said resource without first having the appropriate permissions makes
this flag practically meaningless.
Note that most of my argument hinges on the impact on other, unrelated
databases/tables/sessions. Replication slots have a hard cap defined
at startup, and effective_wal_level increases the WAL generated by
practically all backends. If replication slot counts were SIGHUP, and
RelationIsLogicallyLogged() returned true only in databases with LR
slots, and only for the tables actually present in publications, I'd
consider this much less problematic. However, we don't live in that
world, so I am opposed to allowing table owners without REPLICATION to
take any/all replication slots.
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
@ 2026-03-16 15:10 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 15:49 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-16 15:10 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-16, Matthias van de Meent wrote:
> Note that most of my argument hinges on the impact on other, unrelated
> databases/tables/sessions. Replication slots have a hard cap defined
> at startup, and effective_wal_level increases the WAL generated by
> practically all backends.
I'd rather have a new GUC to declare a bunch of additional slots that
are reserved exclusively for repack, set its default to something like
3, and call it a day. If all repack slots are in use, you don't get to
run repack, period.
A slot costs nothing if unused, and we really don't want to make the
interaction with regular replication more complicated than it is today.
> However, we don't live in that world, so I am opposed to allowing
> table owners without REPLICATION to take any/all replication slots.
I think requiring REPACK users to have the REPLICATION priv is rather
user unfriendly. Some potential REPACK users might not have any other
use for replication at all.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"World domination is proceeding according to plan" (Andrew Morton)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-16 15:49 ` Robert Treat <rob@xzilla.net>
2026-03-16 16:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Robert Treat @ 2026-03-16 15:49 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Mar 16, 2026 at 11:11 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Mar-16, Matthias van de Meent wrote:
>
> > However, we don't live in that world, so I am opposed to allowing
> > table owners without REPLICATION to take any/all replication slots.
>
> I think requiring REPACK users to have the REPLICATION priv is rather
> user unfriendly. Some potential REPACK users might not have any other
> use for replication at all.
>
For many users, I feel like repack concurrently making use of
replication machinery is an implementation detail that some will find
quite surprising (pg_repack doesn't use it), so I'd agree requiring
REPLICATION priv is both unfriendly and a bit counter intuitive,
especially if you need to run repack concurrently on a stand alone
server.
That said, I think Matthias is right that we can't allow "repackers"
to block "replicators"...
> > Note that most of my argument hinges on the impact on other, unrelated
> > databases/tables/sessions. Replication slots have a hard cap defined
> > at startup, and effective_wal_level increases the WAL generated by
> > practically all backends.
>
> I'd rather have a new GUC to declare a bunch of additional slots that
> are reserved exclusively for repack, set its default to something like
> 3, and call it a day. If all repack slots are in use, you don't get to
> run repack, period.
>
> A slot costs nothing if unused, and we really don't want to make the
> interaction with regular replication more complicated than it is today.
>
I'm never excited about adding GUCs, but at first thought this seems
like a decent work-around; most people are unlikely to run multiple
repack concurrently's, but they can if needed. (I think the most
likely use case is on clusters using the "database per customer"
pattern, but if we have the guc, people will have a means to deal with
it).
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 15:49 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2026-03-16 16:03 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 17:55 ` Re: Adding REPACK [concurrently] Mahendra Singh Thalor <mahi6run@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-16 16:03 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On 2026-Mar-16, Robert Treat wrote:
> I'm never excited about adding GUCs, but at first thought this seems
> like a decent work-around; most people are unlikely to run multiple
> repack concurrently's, but they can if needed. (I think the most
> likely use case is on clusters using the "database per customer"
> pattern, but if we have the guc, people will have a means to deal with
> it).
I wonder if, longer term, it would make sense to do away with the
max_replication_slots GUC (and this new one) altogether, and use dynamic
shared memory for slots instead. There's of course always the danger
that people would accumulate arbitrary numbers of slots since they would
never be forced to check. But that may be a lesser problem than having
to gauge these GUCs with any care.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"You're _really_ hosed if the person doing the hiring doesn't understand
relational systems: you end up with a whole raft of programmers, none of
whom has had a Date with the clue stick." (Andrew Sullivan)
https://postgr.es/m/20050809113420.GD2768@phlogiston.dyndns.org
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 15:49 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-03-16 16:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-16 17:55 ` Mahendra Singh Thalor <mahi6run@gmail.com>
2026-03-16 19:26 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Mahendra Singh Thalor @ 2026-03-16 17:55 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Robert Treat <rob@xzilla.net>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, 16 Mar 2026 at 21:33, Alvaro Herrera <alvherre@alvh.no-ip.org>
wrote:
>
> On 2026-Mar-16, Robert Treat wrote:
>
> > I'm never excited about adding GUCs, but at first thought this seems
> > like a decent work-around; most people are unlikely to run multiple
> > repack concurrently's, but they can if needed. (I think the most
> > likely use case is on clusters using the "database per customer"
> > pattern, but if we have the guc, people will have a means to deal with
> > it).
>
> I wonder if, longer term, it would make sense to do away with the
> max_replication_slots GUC (and this new one) altogether, and use dynamic
> shared memory for slots instead. There's of course always the danger
> that people would accumulate arbitrary numbers of slots since they would
> never be forced to check. But that may be a lesser problem than having
> to gauge these GUCs with any care.
>
> --
> Álvaro Herrera 48°01'N 7°57'E —
https://www.EnterpriseDB.com/
> "You're _really_ hosed if the person doing the hiring doesn't understand
> relational systems: you end up with a whole raft of programmers, none of
> whom has had a Date with the clue stick." (Andrew Sullivan)
> https://postgr.es/m/20050809113420.GD2768@phlogiston.dyndns.org
>
>
Hi all,
I was reading this thread and was doing some tests.
postgres=# create table test1(a int);
CREATE TABLE
postgres=# create table test2(a int);
CREATE TABLE
postgres=# *vacuum full test1 , test2;*
VACUUM
postgres=# repack test1;
REPACK
postgres=# repack test2;
REPACK
postgres=#* repack test1, test2;*
ERROR: syntax error at or near ","
LINE 1: repack test1, test2;
^
I was not expecting any error but maybe I am missing something (some patch
needs to be applied to test this query?).
Thanks and Regards
Mahendra
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 15:49 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-03-16 16:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 17:55 ` Re: Adding REPACK [concurrently] Mahendra Singh Thalor <mahi6run@gmail.com>
@ 2026-03-16 19:26 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-16 19:26 UTC (permalink / raw)
To: Mahendra Singh Thalor <mahi6run@gmail.com>; +Cc: Robert Treat <rob@xzilla.net>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On 2026-Mar-16, Mahendra Singh Thalor wrote:
> postgres=#* repack test1, test2;*
> ERROR: syntax error at or near ","
> LINE 1: repack test1, test2;
> ^
>
> I was not expecting any error but maybe I am missing something (some patch
> needs to be applied to test this query?).
Yeah, there's no support for repacking multiple tables in one command.
This is intentional. I think it'd be not very hard to add. But why?
People isn't likely to be running emergency VACUUM FULL on multiple
large tables in this way, listing multiple of them in a single command,
IMO anyway. So I think we don't need it.
(I think this is not similar to the case where normal vacuum is being
run in an emergency for XID wraparound purposes, which the ability to
run on multiple tables is more valuable. But at the same time, vacuumdb
does a much better job at that, since it can process multiple tables in
parallel.)
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-16 19:24 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 19:34 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Matthias van de Meent @ 2026-03-16 19:24 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, 16 Mar 2026 at 16:10, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Mar-16, Matthias van de Meent wrote:
>
> > Note that most of my argument hinges on the impact on other, unrelated
> > databases/tables/sessions. Replication slots have a hard cap defined
> > at startup, and effective_wal_level increases the WAL generated by
> > practically all backends.
>
> I'd rather have a new GUC to declare a bunch of additional slots that
> are reserved exclusively for repack, set its default to something like
> 3, and call it a day. If all repack slots are in use, you don't get to
> run repack, period.
>
> A slot costs nothing if unused, and we really don't want to make the
> interaction with regular replication more complicated than it is today.
There are a few overheads even for unused slots: each slot uses some
shared memory, and many (most) functions that operate on the shared
slot state have O(n_slots) overhead through e.g.
ReplicationSlotsComputeRequiredXmin(); so having a 10k slot limit
whilst using only 1 will be slower for that one active slot than if
the limit was just 10. Granted, that's not a lot of overhead, but it's
not free.
> > However, we don't live in that world, so I am opposed to allowing
> > table owners without REPLICATION to take any/all replication slots.
>
> I think requiring REPACK users to have the REPLICATION priv is rather
> user unfriendly. Some potential REPACK users might not have any other
> use for replication at all.
I agree it's not user-friendly, but that's the point of limiting
permissions. Users can't install c-functions without SUPERUSER,
because it can cause cluster instability and crashes. Users can't
create slots without REPLICATION, because they'll be able to
negatively impact the whole cluster's performance, and possibly,
stability, when taking up replication slots that otherwise would be
used for critical HA purposes.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
@ 2026-03-16 19:34 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 21:45 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-16 19:34 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-16, Matthias van de Meent wrote:
> On Mon, 16 Mar 2026 at 16:10, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > A slot costs nothing if unused, and we really don't want to make the
> > interaction with regular replication more complicated than it is today.
>
> There are a few overheads even for unused slots: each slot uses some
> shared memory, and many (most) functions that operate on the shared
> slot state have O(n_slots) overhead through e.g.
> ReplicationSlotsComputeRequiredXmin(); so having a 10k slot limit
> whilst using only 1 will be slower for that one active slot than if
> the limit was just 10. Granted, that's not a lot of overhead, but it's
> not free.
True, it's not really free in that sense. But let's put this aside, as
it's not something I'm considering for pg19 anyway.
> I agree it's not user-friendly, but that's the point of limiting
> permissions. Users can't install c-functions without SUPERUSER,
> because it can cause cluster instability and crashes. Users can't
> create slots without REPLICATION, because they'll be able to
> negatively impact the whole cluster's performance, and possibly,
> stability, when taking up replication slots that otherwise would be
> used for critical HA purposes.
Well, as I said, these repack slots would be separate from the regular
ones for replication and available only for repack, so they would not
interfere with the count of slots used for replication, so the second
point is moot, isn't it?
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 19:34 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-16 21:45 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Matthias van de Meent @ 2026-03-16 21:45 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, 16 Mar 2026 at 20:34, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Mar-16, Matthias van de Meent wrote:
> > I agree it's not user-friendly, but that's the point of limiting
> > permissions. Users can't install c-functions without SUPERUSER,
> > because it can cause cluster instability and crashes. Users can't
> > create slots without REPLICATION, because they'll be able to
> > negatively impact the whole cluster's performance, and possibly,
> > stability, when taking up replication slots that otherwise would be
> > used for critical HA purposes.
>
> Well, as I said, these repack slots would be separate from the regular
> ones for replication and available only for repack, so they would not
> interfere with the count of slots used for replication, so the second
> point is moot, isn't it?
Sorry, I misread your response as "if at all, then at least like
this", instead of "let's do this alternative in this patch".
But, yes, a pool of slots used exclusively by REPACK CONCURRENTLY
would also solve the slot availability issue.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
@ 2026-03-16 20:15 ` Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-16 20:15 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Matthias van de Meent <boekewurm+postgres@gmail.com> wrote:
> I agree it's not user-friendly, but that's the point of limiting
> permissions. Users can't install c-functions without SUPERUSER,
> because it can cause cluster instability and crashes. Users can't
> create slots without REPLICATION, because they'll be able to
> negatively impact the whole cluster's performance, and possibly,
> stability, when taking up replication slots that otherwise would be
> used for critical HA purposes.
I thought these attributes exist primarily for security purposes. If
non-SUPERUSER user could install C-functions, it'd be easy to install code
that leaks data. REPLICATION is currently the only way to limit access to the
the publisher's data as there is no ACL for publications.
And regarding resources, the REPLICATION attribute alone does not pose a limit
on resource consumption unless you limit the total number of sessions of all
the REPLICATION users at the same time.
Anyway (fortunately?), the concurrent use of slots by REPACK is limited
because, during the initialization of logical decoding, the backend needs to
wait for all the transactions having XID assigned to finish, and these include
the already running REPACK commands. See SnapBuildWaitSnapshot() and callers
if you're interested in details.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-16 21:50 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Matthias van de Meent @ 2026-03-16 21:50 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, 16 Mar 2026 at 21:15, Antonin Houska <ah@cybertec.at> wrote:
>
> Matthias van de Meent <boekewurm+postgres@gmail.com> wrote:
>
> > I agree it's not user-friendly, but that's the point of limiting
> > permissions. Users can't install c-functions without SUPERUSER,
> > because it can cause cluster instability and crashes. Users can't
> > create slots without REPLICATION, because they'll be able to
> > negatively impact the whole cluster's performance, and possibly,
> > stability, when taking up replication slots that otherwise would be
> > used for critical HA purposes.
>
> I thought these attributes exist primarily for security purposes.
Well, yes, but operational security is also security, right?
> If
> non-SUPERUSER user could install C-functions, it'd be easy to install code
> that leaks data.
Yes, as long as they're able to find the right primitives in the
available binaries. That's certainly possible, but a bit more work
than just none.
> REPLICATION is currently the only way to limit access to the
> the publisher's data as there is no ACL for publications.
> And regarding resources, the REPLICATION attribute alone does not pose a limit
> on resource consumption unless you limit the total number of sessions of all
> the REPLICATION users at the same time.
True. It's not great. And we also don't really have a (good)
distinction for logical/physical replication permissions either...
> Anyway (fortunately?), the concurrent use of slots by REPACK is limited
> because, during the initialization of logical decoding, the backend needs to
> wait for all the transactions having XID assigned to finish, and these include
> the already running REPACK commands. See SnapBuildWaitSnapshot() and callers
> if you're interested in details.
Huh, so would you be able to run more than one Repack Concurrently in
the same database? ISTM that would not be possible, apart from
possibly a mechanism comparable to the SAFE_IN_IC flag (to not wait on
those backends).
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
@ 2026-03-16 22:03 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-16 22:03 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-16, Matthias van de Meent wrote:
> On Mon, 16 Mar 2026 at 21:15, Antonin Houska <ah@cybertec.at> wrote:
> > Anyway (fortunately?), the concurrent use of slots by REPACK is limited
> > because, during the initialization of logical decoding, the backend needs to
> > wait for all the transactions having XID assigned to finish, and these include
> > the already running REPACK commands. See SnapBuildWaitSnapshot() and callers
> > if you're interested in details.
>
> Huh, so would you be able to run more than one Repack Concurrently in
> the same database? ISTM that would not be possible, apart from
> possibly a mechanism comparable to the SAFE_IN_IC flag (to not wait on
> those backends).
Yeah, this sounds kind of bad news ...
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Los dioses no protegen a los insensatos. Éstos reciben protección de
otros insensatos mejor dotados" (Luis Wu, Mundo Anillo)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-17 10:53 ` Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-17 10:53 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Mar-16, Matthias van de Meent wrote:
>
> > On Mon, 16 Mar 2026 at 21:15, Antonin Houska <ah@cybertec.at> wrote:
>
> > > Anyway (fortunately?), the concurrent use of slots by REPACK is limited
> > > because, during the initialization of logical decoding, the backend needs to
> > > wait for all the transactions having XID assigned to finish, and these include
> > > the already running REPACK commands. See SnapBuildWaitSnapshot() and callers
> > > if you're interested in details.
> >
> > Huh, so would you be able to run more than one Repack Concurrently in
> > the same database? ISTM that would not be possible, apart from
> > possibly a mechanism comparable to the SAFE_IN_IC flag (to not wait on
> > those backends).
>
> Yeah, this sounds kind of bad news ...
Admittedly, it is a problem. I tried to address this in pg_squeeze by
pre-allocating slots when it's clear (due to scheduling) that more than one
table needs to be processed. This was an effort to achieve the best possible
performance rather than a response to complaints of users about low
throughput. Nevertheless, I'm glad I happened to mention it before it's too
late.
Regarding solution, a flag like SAFE_IN_IC alone does not help. The
information that particular transaction is used by REPACK (and therefore it
does not have to be decoded) would need to be propagated to the
xl_running_xacts WAL record too.
The enhancements I wrote for PG 20 (not all of them posted yet) that aim at
eliminating the impact of REPACK on VACUUM xmin horizon should fix this
problem: due to the MVCC-safety (i.e. preserving xmin/xmax of the tuples),
REPACK will not need XID assigned (except for catalog changes, which will
happen in separate transactions), so it won't block the logical decoding setup
of other backends.
So the question is whether we should implement a workaround for PG 19, that
won't be needed in v20.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-17 19:57 ` Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-17 19:57 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > On 2026-Mar-16, Matthias van de Meent wrote:
> >
> > > On Mon, 16 Mar 2026 at 21:15, Antonin Houska <ah@cybertec.at> wrote:
> >
> > > > Anyway (fortunately?), the concurrent use of slots by REPACK is limited
> > > > because, during the initialization of logical decoding, the backend needs to
> > > > wait for all the transactions having XID assigned to finish, and these include
> > > > the already running REPACK commands. See SnapBuildWaitSnapshot() and callers
> > > > if you're interested in details.
> > >
> > > Huh, so would you be able to run more than one Repack Concurrently in
> > > the same database? ISTM that would not be possible, apart from
> > > possibly a mechanism comparable to the SAFE_IN_IC flag (to not wait on
> > > those backends).
> >
> > Yeah, this sounds kind of bad news ...
>
> Admittedly, it is a problem. I tried to address this in pg_squeeze by
> pre-allocating slots when it's clear (due to scheduling) that more than one
> table needs to be processed. This was an effort to achieve the best possible
> performance rather than a response to complaints of users about low
> throughput. Nevertheless, I'm glad I happened to mention it before it's too
> late.
>
> Regarding solution, a flag like SAFE_IN_IC alone does not help. The
> information that particular transaction is used by REPACK (and therefore it
> does not have to be decoded) would need to be propagated to the
> xl_running_xacts WAL record too.
0007 in the next version tries to implement that.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-18 19:12 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-18 20:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-03-18 19:12 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
While i was doing concurrency test onn V41 patches ,i found this crash
because of the assert failure,
TRAP: failed Assert("RelationGetRelid(relation) == ((RepackDecodingState *)
ctx->output_writer_private)->relid"), File: "pgoutput_repack.c", Line: 97,
PID: 397007
postgres: REPACK decoding worker for relation "stress_victim"
(ExceptionalCondition+0x98)[0xaaaad9361698]
/home/srinath/Desktop/pgbuild/lib/postgresql/pgoutput_repack.so(+0xfe8)[0xffff90e00fe8]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x679e14)[0xaaaad9049e14]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x689cd0)[0xaaaad9059cd0]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x68a65c)[0xaaaad905a65c]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x68b2f0)[0xaaaad905b2f0]
postgres: REPACK decoding worker for relation "stress_victim"
(ReorderBufferCommit+0x74)[0xaaaad905b374]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x671ec4)[0xaaaad9041ec4]
postgres: REPACK decoding worker for relation "stress_victim"
(xact_decode+0x1a0)[0xaaaad9040edc]
postgres: REPACK decoding worker for relation "stress_victim"
(LogicalDecodingProcessRecord+0xd4)[0xaaaad9040a80]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x33f558)[0xaaaad8d0f558]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x341ccc)[0xaaaad8d11ccc]
postgres: REPACK decoding worker for relation "stress_victim"
(RepackWorkerMain+0x1ac)[0xaaaad8d11bd4]
postgres: REPACK decoding worker for relation "stress_victim"
(BackgroundWorkerMain+0x2b0)[0xaaaad900d21c]
postgres: REPACK decoding worker for relation "stress_victim"
(postmaster_child_launch+0x1f0)[0xaaaad9012070]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x64b974)[0xaaaad901b974]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x64bc64)[0xaaaad901bc64]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x64a3e4)[0xaaaad901a3e4]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x647648)[0xaaaad9017648]
postgres: REPACK decoding worker for relation "stress_victim"
(PostmasterMain+0x160c)[0xaaaad9016d98]
postgres: REPACK decoding worker for relation "stress_victim"
(main+0x3dc)[0xaaaad8ea7a38]
/lib/aarch64-linux-gnu/libc.so.6(+0x284c4)[0xffff9c5c84c4]
/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0x98)[0xffff9c5c8598]
postgres: REPACK decoding worker for relation "stress_victim"
(_start+0x30)[0xaaaad8abc970]
2026-03-16 19:40:21.622 IST [393820] LOG: background worker "REPACK
decoding worker" (PID 397007) was terminated by signal 6: Aborted
2026-03-16 19:40:21.622 IST [393820] LOG: terminating any other active
server processes
2026-03-16 19:40:21.632 IST [397036] FATAL: the database system is in
recovery mode
This crash happens if we run REPACK (concurrently) on a table while a heavy
pgbench workload is concurrently executing multi-table(setup.sql)
transactions(dual_chaos.sql).
It triggers after a few back to back REPACK (concurrently) runs.
i think i found the cause for this crash , because there were some changes
which
slipped under the nose of the change_useless_for_repack filter , which led
some
changes which are not related to the relation which we are currently doing
REPACK (concurrently)
got decoded and added into the reorderbuffer queue, the reason for this
is repacked_rel_locator.relNumber
is by default set to InvalidOid, this is actually set to the target
relation during setup_logical_decoding
but this done after DecodingContextFindStartpoint, in
DecodingContextFindStartpoint changes are not
filtered even if its not related to the target relation , because
rm_decode->change_useless_for_repack->am_decoding_for_repack
where repacked_rel_locator.relNumber is still InvalidOid, which makes it
skip the filtering even its not the target relation,
this makes it to be added to reorder buffer queue, so during the processing
of reorder buffer plugin_change is called
where assert fails, i have attached a diff patch to solve this.
thoughts?
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] fix_diff.norobots (1.6K, ../../CAFC+b6qk3-DQTi43QMqvVLP+sudPV4vsLQm5iHfcCeObrNaVyA@mail.gmail.com/3-fix_diff.norobots)
download
[application/octet-stream] dual_chaos.sql (680B, ../../CAFC+b6qk3-DQTi43QMqvVLP+sudPV4vsLQm5iHfcCeObrNaVyA@mail.gmail.com/4-dual_chaos.sql)
download
[application/octet-stream] setup.sql (466B, ../../CAFC+b6qk3-DQTi43QMqvVLP+sudPV4vsLQm5iHfcCeObrNaVyA@mail.gmail.com/5-setup.sql)
download
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-18 20:07 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-18 20:07 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> TRAP: failed Assert("RelationGetRelid(relation) == ((RepackDecodingState *) ctx->output_writer_private)->relid"), File: "pgoutput_repack.c",
> Line: 97, PID: 397007
> This crash happens if we run REPACK (concurrently) on a table while a heavy
> pgbench workload is concurrently executing multi-table(setup.sql) transactions(dual_chaos.sql).
> It triggers after a few back to back REPACK (concurrently) runs.
>
> i think i found the cause for this crash , because there were some changes which
> slipped under the nose of the change_useless_for_repack filter , which led some
> changes which are not related to the relation which we are currently doing REPACK (concurrently)
> got decoded and added into the reorderbuffer queue, the reason for this is repacked_rel_locator.relNumber
> is by default set to InvalidOid, this is actually set to the target relation during setup_logical_decoding
> but this done after DecodingContextFindStartpoint, in DecodingContextFindStartpoint changes are not
> filtered even if its not related to the target relation , because rm_decode->change_useless_for_repack->am_decoding_for_repack
> where repacked_rel_locator.relNumber is still InvalidOid, which makes it skip the filtering even its not the target relation,
> this makes it to be added to reorder buffer queue, so during the processing of reorder buffer plugin_change is called
> where assert fails, i have attached a diff patch to solve this.
Thanks a lot! Yes, your explanation makes sense. I'll include the fix in the
next version. I think it might also explain the other crash [1] you reported
earlier. I'll try to reproduce that.
[1] https://www.postgresql.org/message-id/CAFC%2Bb6o2yzA80YmfEhmMO9puN8qvGRvr-15BBLn3UmJxPfpr2w%40mail.g...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-19 19:52 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-19 20:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-19 19:52 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi Srinath,
On 2026-Mar-19, Srinath Reddy Sadipiralla wrote:
> i think i found the cause for this crash, because there were some changes
> which slipped under the nose of the change_useless_for_repack filter, which
> led some changes which are not related to the relation which we are
> currently doing REPACK (concurrently) got decoded and added into the
> reorderbuffer queue, the reason for this is repacked_rel_locator.relNumber
> is by default set to InvalidOid, this is actually set to the target relation
> during setup_logical_decoding but this done after
> DecodingContextFindStartpoint, in DecodingContextFindStartpoint changes are
> not filtered even if its not related to the target relation, because
> rm_decode->change_useless_for_repack->am_decoding_for_repack where
> repacked_rel_locator.relNumber is still InvalidOid, which makes it skip the
> filtering even its not the target relation, this makes it to be added to
> reorder buffer queue, so during the processing of reorder buffer
> plugin_change is called where assert fails, i have attached a diff patch to
> solve this.
Ah, thanks for tracking this down -- I've been seeing this crash also.
So here's v43. Here, I've changed the CONCURRENTLY implementation to go
through table AM. This necessitated changing it to use tuples in slots
instead of HeapTuple. This is good because we can avoid repeated tuple
form/deform, which could get pretty expensive. Antonin's 0004 patch
here looks suspicious here though, because it deforms the tuple and
forms it again, which sounds unnecessary now.
Not yet committable; the change to use slots and table AM is a rather
significant modification to what was before. Barring cosmetics and code
reorganization, it should be close enough.
I included your (Srinath's) patch as 0005. The file you sent is corrupt
though, so I applied the visible part of change manually, in addition to
removing the rest of the block that the patch somehow doesn't remove.
This also includes as part of 0003 the change to table AM options I sent in
[1]. I will probably commit that change in the way that the other thread
decides, and then rebase this on top of that.
[1] https://postgr.es/m/202603171606.kf6pmhscqbqz@alvherre.pgsql
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-19 20:37 ` Antonin Houska <ah@cybertec.at>
2026-03-20 07:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-19 20:37 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> So here's v43. Here, I've changed the CONCURRENTLY implementation to go
> through table AM. This necessitated changing it to use tuples in slots
> instead of HeapTuple. This is good because we can avoid repeated tuple
> form/deform, which could get pretty expensive. Antonin's 0004 patch
> here looks suspicious here though, because it deforms the tuple and
> forms it again, which sounds unnecessary now.
I suppose you mean
v42-0004-Serialize-decoded-tuples-without-flattening.patch. This deforms the
tuple to get the external attributes and to write them to file. The tuple the
logical worker received from reorderbuffer.c cannot be passed to the backend
executing REPACK because it may contain "external indirect" attributes,
i.e. pointers to the worker's memory.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-19 20:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-20 07:59 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-23 10:20 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-20 07:59 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-19, Antonin Houska wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > So here's v43. Here, I've changed the CONCURRENTLY implementation to go
> > through table AM. This necessitated changing it to use tuples in slots
> > instead of HeapTuple. This is good because we can avoid repeated tuple
> > form/deform, which could get pretty expensive. Antonin's 0004 patch
> > here looks suspicious here though, because it deforms the tuple and
> > forms it again, which sounds unnecessary now.
>
> I suppose you mean
> v42-0004-Serialize-decoded-tuples-without-flattening.patch. This deforms the
> tuple to get the external attributes and to write them to file. The tuple the
> logical worker received from reorderbuffer.c cannot be passed to the backend
> executing REPACK because it may contain "external indirect" attributes,
> i.e. pointers to the worker's memory.
No, that patch has been absorbed in what is now v43-0003. I meant
v43-0004 "Use BulkInsertState when copying data to the new heap.",
that's why I said "patch 0004 here". In this patch, we have
reform_tuple which deforms the tuple, sets to NULL any attribute that's
marked dropped, and then forms a new one. This is wasteful and should
probably be done elsewhere, while the tuple is still in slot
representation. In fact, I suspect it may not be necessary at all
anymore.
I haven't verified whether all the code is covered by existing tests;
what I did was just run them. But to ensure that it is all trustworthy,
I'll spend some time with the coverage report to ensure there aren't any
nasty surprises anywhere. The slot/tupdesc interface is notoriously bad
at differentiating 0-based indexes of the attribute array, and 1-based
proper attribute numbers, so it's very easy to do the wrong thing.
(It's worse when you do an even number of wrong things and they cancel
each other out.)
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"This is what I like so much about PostgreSQL. Most of the surprises
are of the "oh wow! That's cool" Not the "oh shit!" kind. :)"
Scott Marlowe, http://archives.postgresql.org/pgsql-admin/2008-10/msg00152.php
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-19 20:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 07:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-23 10:20 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-23 10:20 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Mar-19, Antonin Houska wrote:
>
> > Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > > So here's v43. Here, I've changed the CONCURRENTLY implementation to go
> > > through table AM. This necessitated changing it to use tuples in slots
> > > instead of HeapTuple. This is good because we can avoid repeated tuple
> > > form/deform, which could get pretty expensive. Antonin's 0004 patch
> > > here looks suspicious here though, because it deforms the tuple and
> > > forms it again, which sounds unnecessary now.
> >
> > I suppose you mean
> > v42-0004-Serialize-decoded-tuples-without-flattening.patch. This deforms the
> > tuple to get the external attributes and to write them to file. The tuple the
> > logical worker received from reorderbuffer.c cannot be passed to the backend
> > executing REPACK because it may contain "external indirect" attributes,
> > i.e. pointers to the worker's memory.
>
> No, that patch has been absorbed in what is now v43-0003. I meant
> v43-0004 "Use BulkInsertState when copying data to the new heap.",
> that's why I said "patch 0004 here". In this patch, we have
> reform_tuple which deforms the tuple, sets to NULL any attribute that's
> marked dropped, and then forms a new one. This is wasteful and should
> probably be done elsewhere, while the tuple is still in slot
> representation. In fact, I suspect it may not be necessary at all
> anymore.
The idea to "reform" the tuple comes from VACUUM FULL / CLUSTER. I agree the
"deform" and "form" steps are no longer needed when we use tuple slots.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-20 12:35 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-20 15:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 3 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-03-20 12:35 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, everyone!
Some comments for v43:
---
src/backend/commands/cluster.c:3211
slot_attisnull(dest, i) uses a 0-based loop index but slot_attisnull
starts with "Assert(attnum > 0);".
Also, it proves it has not been tested in any way yet.
------------------------------
src/backend/storage/ipc/standby.c:1376
"if (xlrec.xcnt > 0)"
looks like it should be "xlrec.xcnt + xlrec.xcnt_repack > 0" because
of "CurrentRunningXacts->xcnt = count - subcount - count_repack;"
------------------------------
src/backend/storage/ipc/procarray.c:2673-2681
"nrepack = logical_decoding_enabled ? MAX_REPACK_XIDS : 0"
not sure it is a real issue, but if EnableLogicalDecoding is called
somehow after allocating the array - it makes it possible to overrun
the array later by MAX_REPACK_XIDS.
------------------------------
src/backend/commands/cluster.c:3006
"apply_cxt = AllocSetContextCreate(TopTransactionContext,
"REPACK - apply",
ALLOCSET_DEFAULT_SIZES);"
Should we do it before the "MakeSingleTupleTableSlot" calls?
Because MakeSingleTupleTableSlot stored CurrentMemoryContext in tts_mcxt.
------------------------------
src/backend/commands/cluster.c:3187
if (natt_ext != 0)
elog(WARNING, "have natt_ext %d, weird", natt_ext);
Should we make that ERROR?
------------------------------
src/backend/access/rmgrdesc/standbydesc.c:24
appendStringInfo(buf, "nextXid %u latestCompletedXid %u
oldestRunningXid %u oldestRunningXid %u",
"oldestRunningXidLogical" at the end?
------------------------------
src/backend/catalog/index.c:766,1464-1469
"bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;"
AFAIU gin, hash and btree (at least) still just unconditionally write
PROGRESS_CREATEIDX_* progress.
------------------------------
src/backend/catalog/toasting.c:334
"INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,"
Should we add the "progress" flag too here and move it from make_new_heap?
-------------------------------
Also just, gently remind you about [1] and a simple way to avoid any
unnoticed MVCC-related issues with REPACK proposed here [2].
Best regards,
Mikhail.
[1]: https://www.postgresql.org/message-id/flat/5k2dfckyp6zv2fiovosvtbya5onvplgviz5n4kdamxupff4vi2%40yytz...
[2]: https://www.postgresql.org/message-id/CADzfLwUEH5+LjCN+6kRfSsXwuou8rKXyVV42Wi-O_TG0360Kug@mail.gmail...
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-03-20 15:56 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-20 15:56 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-20, Mihail Nikalayeu wrote:
> Also just, gently remind you about [1] and a simple way to avoid any
> unnoticed MVCC-related issues with REPACK proposed here [2].
Ah, thanks for the reminder, I've sent a patch there.
> [1]: https://www.postgresql.org/message-id/flat/5k2dfckyp6zv2fiovosvtbya5onvplgviz5n4kdamxupff4vi2%40yytz...
> [2]: https://www.postgresql.org/message-id/CADzfLwUEH5+LjCN+6kRfSsXwuou8rKXyVV42Wi-O_TG0360Kug@mail.gmail...
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Learn about compilers. Then everything looks like either a compiler or
a database, and now you have two problems but one of them is fun."
https://twitter.com/thingskatedid/status/1456027786158776329
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-03-24 22:32 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-24 22:32 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
On 2026-Mar-20, Mihail Nikalayeu wrote:
> Hello, everyone!
>
> Some comments for v43:
Many thanks for the review. I have applied fixes for these, so here's
v44. Changes:
- 0001 now simply renames the existing function and adds the
"concurrently" flag, instead of adding a separate function. I don't
think there's a reason to keep the previous function name.
- 0002 is unchanged.
- 0003 contains the fixes to the problems pointed out by Mihail;
otherwise it's pretty much the same as in v43.
- 0004 is Antonin's bugfix from the crash reported by Srinath.
- 0005 to 0007 are my proposed changes on this round. It's mostly just
cosmetics. I intend to squash these all into a single commit (0003 to
at least 0007) when posting the next version.
- 0008 to 0010 are as posted by Antonin; they are unchanged, except for
fixes for the problems pointed out by Mihail. Antonin, I would
appreciate it if you want to change the "reform" bit in 0007 as
discussed.
For the next version I'll probably also rename the file cluster.c to
repack.c as well as assorted symbols therein, as well as studying the
coverage more closely.
Thanks,
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-25 02:57 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-03-25 02:57 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hallo Alvaro,
On Wed, Mar 25, 2026 at 4:02 AM Alvaro Herrera <alvherre@alvh.no-ip.org>
wrote:
>
> Many thanks for the review. I have applied fixes for these, so here's
> v44.
>
Thanks for the patches.
- 0004 is Antonin's bugfix from the crash reported by Srinath.
>
I think it's "0004 is Srinath's bugfix from the crash reported by Srinath."
;-)
after i provided the analysis and fix for the crash[1], Antonin tried to
reproduce
this crash using isolation tester , for this he even proposed changes to
isolation tester (so cool ... btw i reviewed it) [2] .
i have done another round of stress testing on V43 , this time with more
tests,
as i did previously [3] did concurrency test - went well,
crash test:
after crash i observed that repack worker files are cleaned during server
restart
using RemovePgTempFiles but the transient table relation files remains
not cleaned up, maybe we can do cleanup for this as well during server
restart,
I will think about this more.
physical replication test where I did REPACK (concurrently) on primary and
checked if data is intact using the 4 verifications I did here [3] on
replica - went well.
Then as suggested by Alvaro off-list I checked the lock upgrade behavior
during the table swap phase. I observed that if another transaction holds a
conflicting lock on the table when the swap is attempted, it can lead to
“transient table” data loss during a manual or timeout abort.
when a REPACK (concurrent) waits for a conflicting lock to be released and
eventually hits a
lock_timeout (or is cancelled via ctrl+c), the transaction aborts. During
this abort,
the cleanup process triggers smgrDoPendingDeletes. This results in the
removal
of all transient table relfiles and decoder worker files created during the
process.
This effectively wipes out the work done by the transient table creation
before
the swap could successfully complete, this happens because during transient
table creation we add the table to the PendingRelDelete list.
rebuild_relation→make_new_heap->heap_create_with_catalog→heap_create→table_relation_set_new_filelocator→RelationCreateStorage
/*
* Add the relation to the list of stuff to delete at abort, if we are
* asked to do so.
*/
if (register_delete)
{
PendingRelDelete *pending;
pending = (PendingRelDelete *)
MemoryContextAlloc(TopMemoryContext, sizeof(PendingRelDelete));
pending->rlocator = rlocator;
pending->procNumber = procNumber;
pending->atCommit = false; /* delete if abort */
pending->nestLevel = GetCurrentTransactionNestLevel();
pending->next = pendingDeletes;
pendingDeletes = pending;
}
and the base/5/pgsql_tmp/ files also gets unlinked during the decoding
worker cleanup,
I think this cleanup of transient table relfiles and decoder files makes
sense because
we don’t have any resume operation in which we can re-use the transient
table’s files,
please correct me if I am not getting your point here.
test scenario:
session 1:
postgres=# repack (concurrently) stress_victim;
had a breakpoint rebuild_relation_finish_concurrent->
LockRelationOid(old_table_oid, AccessExclusiveLock); just before getting
the exclusive lock.
with lock_timeout = 5s
session 2:
postgres=# BEGIN;
SELECT * FROM stress_victim LIMIT 1;
-- left it open
BEGIN
id | balance |
payload
-----+---------+---------------------------------
-------------------------------------------------
-------------------------------------------------
-------------------------------------------------
--------------
170 | 65 | d12f400c4d0d3c49818f88597e16cf29
d12f400c4d0d3c49818f88597e16cf29d12f400c4d0d3c498
18f88597e16cf29d12f400c4d0d3c49818f88597e16cf29d1
2f400c4d0d3c49818f88597e16cf29d12f400c4d0d3c49818
f88597e16cf29
(1 row)
-- this gets us a conflicting lock (AccessShareLock) on the same table,
REPACK (concurrently) is running on.
session 1:
release the breakpoint and now the backend waits for the conflicting lock
to be released.
in between if lock_timeout occurs then transaction aborts.
postgres=# repack (concurrently) stress_victim;
ERROR: canceling statement due to lock timeout
CONTEXT: waiting for AccessExclusiveLock on relation 16637 of database 5
Now we can see the transient table relfiles and decoder worker files
getting cleaned up.
[1] -
https://www.postgresql.org/message-id/CAFC%2Bb6qk3-DQTi43QMqvVLP%2BsudPV4vsLQm5iHfcCeObrNaVyA%40mail...
[2] - https://www.postgresql.org/message-id/flat/4703.1774250534%40localhost
[3] -
https://www.postgresql.org/message-id/CAFC%2Bb6o2yzA80YmfEhmMO9puN8qvGRvr-15BBLn3UmJxPfpr2w%40mail.g...
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-25 20:12 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-26 09:51 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-26 14:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 17:37 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 3 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-25 20:12 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-25, Srinath Reddy Sadipiralla wrote:
> Then as suggested by Alvaro off-list I checked the lock upgrade
> behavior during the table swap phase. I observed that if another
> transaction holds a conflicting lock on the table when the swap is
> attempted, it can lead to “transient table” data loss during a manual
> or timeout abort. when a REPACK (concurrent) waits for a conflicting
> lock to be released and eventually hits a lock_timeout (or is
> cancelled via ctrl+c), the transaction aborts. During this abort, the
> cleanup process triggers smgrDoPendingDeletes. This results in the
> removal of all transient table relfiles and decoder worker files
> created during the process. This effectively wipes out the work done
> by the transient table creation before the swap could successfully
> complete, this happens because during transient table creation we add
> the table to the PendingRelDelete list.
I think we certainly need to make the files be deleted in some
reasonable fashion if repack fails partway through.
As for lock upgrade, I wonder if the best way to handle this isn't to
hack the deadlock detector so that it causes any *other* process to die,
if they detect that they would block on REPACK. Arguably there's
nothing that you can do to a table while its undergoing REPACK
CONCURRENTLY; any alterations would have to wait until the repacking is
compelted. We can implement that idea simply enough, as shown in this
crude prototype. (I omitted the last three patches in the series, and
squashed my proposed changes into 0003, as announced in my previous
posting.)
The isolation test file is also a bit crude; I just copied repack.spec
to a new file and removed the uninteresting bits.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"I am amazed at [the pgsql-sql] mailing list for the wonderful support, and
lack of hesitasion in answering a lost soul's question, I just wished the rest
of the mailing list could be like this." (Fotis)
https://postgr.es/m/200606261359.k5QDxE2p004593@auth-smtp.hol.gr
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-26 09:51 ` Antonin Houska <ah@cybertec.at>
2026-03-26 11:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-26 23:14 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-26 09:51 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Mar-25, Srinath Reddy Sadipiralla wrote:
>
> > Then as suggested by Alvaro off-list I checked the lock upgrade
> > behavior during the table swap phase. I observed that if another
> > transaction holds a conflicting lock on the table when the swap is
> > attempted, it can lead to “transient table” data loss during a manual
> > or timeout abort. when a REPACK (concurrent) waits for a conflicting
> > lock to be released and eventually hits a lock_timeout (or is
> > cancelled via ctrl+c), the transaction aborts. During this abort, the
> > cleanup process triggers smgrDoPendingDeletes. This results in the
> > removal of all transient table relfiles and decoder worker files
> > created during the process. This effectively wipes out the work done
> > by the transient table creation before the swap could successfully
> > complete, this happens because during transient table creation we add
> > the table to the PendingRelDelete list.
>
> I think we certainly need to make the files be deleted in some
> reasonable fashion if repack fails partway through.
I think that Srinath tries to explain the cleanup in detail, but in fact
that's a normal processing of transaction abort. Not sure we need to do
anything special.
> As for lock upgrade, I wonder if the best way to handle this isn't to
> hack the deadlock detector so that it causes any *other* process to die,
> if they detect that they would block on REPACK. Arguably there's
> nothing that you can do to a table while its undergoing REPACK
> CONCURRENTLY; any alterations would have to wait until the repacking is
> compelted. We can implement that idea simply enough, as shown in this
> crude prototype. (I omitted the last three patches in the series, and
> squashed my proposed changes into 0003, as announced in my previous
> posting.)
I haven't thought of it because I'm not familiar with the deadlock detector,
but what you do seems consistent with the way blocking by autovacuum is
handled.
The only problem I noticed is that PROC_IN_CONCURRENT_REPACK is not cleared at
the end of transaction. Perhaps it should be added to PROC_VACUUM_STATE_MASK
(name of which is already misleading due to the presence of PROC_IN_SAFE_IC,
but that's another problem).
> The isolation test file is also a bit crude; I just copied repack.spec
> to a new file and removed the uninteresting bits.
Maybe just add a new permutation to repack.spec? I don't remembery if I
created repack_toast.spec as a separate file just for better readability or if
there was some other issue, but the deadlock test might fit into repack.spec.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-26 09:51 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-26 11:23 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-26 11:23 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > As for lock upgrade, I wonder if the best way to handle this isn't to
> > hack the deadlock detector so that it causes any *other* process to die,
> > if they detect that they would block on REPACK. Arguably there's
> > nothing that you can do to a table while its undergoing REPACK
> > CONCURRENTLY; any alterations would have to wait until the repacking is
> > compelted. We can implement that idea simply enough, as shown in this
> > crude prototype. (I omitted the last three patches in the series, and
> > squashed my proposed changes into 0003, as announced in my previous
> > posting.)
If we take this approach, some comments on deadlock need to be adjusted - see
my proposals in nocfbot_comments_deadlock.diff.
Besides that, nocfbot_comment_cluster_rel.diff suggests one more comment
change that does not depend on the deadlock detection - I forgot to change it
when implementing the lock upgrade.
Also the commit message of 0003 needs to be adjusted. (Does it need to mention
the problem at all?)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] nocfbot_comments_deadlock.diff (2.5K, ../../29614.1774524229@localhost/2-nocfbot_comments_deadlock.diff)
download | inline diff:
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index d5b1dfbff69..a9788ac6209 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -618,10 +615,12 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
/*
* Make sure we're not in a transaction block.
*
- * The reason is that repack_setup_logical_decoding() could deadlock
- * if there's an XID already assigned. It would be possible to run in
- * a transaction block if we had no XID, but this restriction is
- * simpler for users to understand and we don't lose anything.
+ * The reason is that repack_setup_logical_decoding() could wait
+ * indefinitely for our XID to complete. (The deadlock detector would
+ * not recognize it because we'd be waiting for ourselves, i.e. no
+ * real lock conflict.) It would be possible to run in a transaction
+ * block if we had no XID, but this restriction is simpler for users
+ * to understand and we don't lose anything.
*/
PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
@@ -1104,10 +1103,8 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose,
* Note that the worker has to wait for all transactions with XID
* already assigned to finish. If some of those transactions is
* waiting for a lock conflicting with ShareUpdateExclusiveLock on our
- * table (e.g. it runs CREATE INDEX), we can end up in a deadlock.
- * Not sure this risk is worth unlocking/locking the table (and its
- * clustering index) and checking again if it's still eligible for
- * REPACK CONCURRENTLY.
+ * table (e.g. it runs CREATE INDEX), it should encounter ERROR in the
+ * deadlock checking code.
*/
start_repack_decoding_worker(tableOid);
@@ -3766,9 +3763,11 @@ start_repack_decoding_worker(Oid relid)
/*
* The decoding setup must be done before the caller can have XID assigned
- * for any reason, otherwise the worker might end up in a deadlock,
- * waiting for the caller's transaction to end. Therefore wait here until
- * the worker indicates that it has the logical decoding initialized.
+ * for any reason, otherwise the worker might end up waiting for the
+ * caller's transaction to end. (Deadlock detector does not consider this
+ * a conflict because the worker is in the same locking group as the
+ * backend that launched it.) Therefore wait here until the worker
+ * indicates that it has the logical decoding initialized.
*/
ConditionVariablePrepareToSleep(&shared->cv);
for (;;)
[text/x-diff] nocfbot_comment_cluster_rel.diff (942B, ../../29614.1774524229@localhost/3-nocfbot_comment_cluster_rel.diff)
download | inline diff:
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index d5b1dfbff69..a9788ac6209 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -582,11 +582,8 @@ RepackLockLevel(bool concurrent)
* If indexOid is InvalidOid, the table will be rewritten in physical order
* instead of index order.
*
- * Note that, in the concurrent case, the function releases the lock at some
- * point, in order to get AccessExclusiveLock for the final steps (i.e. to
- * swap the relation files). To make things simpler, the caller should expect
- * OldHeap to be closed on return, regardless CLUOPT_CONCURRENT. (The
- * AccessExclusiveLock is kept till the end of the transaction.)
+ * On return, OldHeap is closed but locked with AccessExclusiveLock - the lock
+ * will be released at end of the transaction.
*
* 'cmd' indicates which command is being executed, to be used for error
* messages.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-26 09:51 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-26 23:14 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-26 23:14 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-26, Antonin Houska wrote:
> I haven't thought of it because I'm not familiar with the deadlock detector,
> but what you do seems consistent with the way blocking by autovacuum is
> handled.
>
> The only problem I noticed is that PROC_IN_CONCURRENT_REPACK is not cleared at
> the end of transaction. Perhaps it should be added to PROC_VACUUM_STATE_MASK
> (name of which is already misleading due to the presence of PROC_IN_SAFE_IC,
> but that's another problem).
Yeah, good observation -- we should absolutely clear it at transaction
end, and adding it to PROC_VACUUM_STATE_MASK seems the cleanest way to
make that happen. However, we should also clear it as soon as we've
acquired the AccessExclusiveLock on all rels. At that point we no
longer need it, and it makes more sense to have other transactions
resume the normal waiting behavior instead of having them fail right
away.
> Maybe just add a new permutation to repack.spec? I don't remembery if I
> created repack_toast.spec as a separate file just for better readability or if
> there was some other issue, but the deadlock test might fit into repack.spec.
Yeah, that was my first impulse too, but it's not clear to me that
things are better one way or the other. I haven't decided yet.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"[PostgreSQL] is a great group; in my opinion it is THE best open source
development communities in existence anywhere." (Lamar Owen)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-26 14:06 ` Antonin Houska <ah@cybertec.at>
2 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-26 14:06 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> (I omitted the last three patches in the series, and
> squashed my proposed changes into 0003, as announced in my previous
> posting.)
I've updated the comment about on-disk attributes in repack_store_change(),
but when verifying it, I hit an error when more than one UPDATEs (in separate
transactions) were executed during a single run of REPACK.
The problem is that reorderbuffer.c sets up an internal (sub)transaction
before replaying each decoded transaction. Therefore the tuple slot should not
be allocated in TopTransactionContext. I chose TopMemoryContext instead.
BTW, if you want to verify that the updated comment is correct, just add
elog(ERROR) next to it and run repack_toast.spec. The statement
UPDATE repack_test SET i=3 where i=1;
will then reach it.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] nocfbot_fix_repack_store_change.diff (1.3K, ../../43861.1774533989@localhost/2-nocfbot_fix_repack_store_change.diff)
download | inline diff:
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
index cc9ce615b18..5fe3115509e 100644
--- a/src/backend/replication/pgoutput_repack/pgoutput_repack.c
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -202,7 +202,11 @@ repack_store_change(LogicalDecodingContext *ctx, Relation relation,
/* Initialize the slot, if not done already */
if (dstate->slot == NULL)
{
- MemoryContextSwitchTo(oldcxt);
+ /*
+ * We are in the decoding worker, so no worries about polluting
+ * memory of the backend executing REPACK.
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
dstate->slot = MakeSingleTupleTableSlot(desc, &TTSOpsHeapTuple);
MemoryContextSwitchTo(dstate->change_cxt);
}
@@ -247,8 +251,8 @@ repack_store_change(LogicalDecodingContext *ctx, Relation relation,
* attributes (those actually should never appear on disk), so
* only TOASTed attribute can be seen here.
*
- * FIXME in what circumstances can an ONDISK attr appear? Why
- * aren't these written separately?
+ * We get here if the table has external values but only
+ * in-line values are being updated now.
*/
Assert(VARATT_IS_EXTERNAL_ONDISK(varlen));
}
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-31 17:37 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-31 18:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 2 replies; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-03-31 17:37 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi Alvaro,
On Thu, Mar 26, 2026 at 1:42 AM Alvaro Herrera <alvherre@alvh.no-ip.org>
wrote:
>
> As for lock upgrade, I wonder if the best way to handle this isn't to
> hack the deadlock detector so that it causes any *other* process to die,
> if they detect that they would block on REPACK. Arguably there's
> nothing that you can do to a table while its undergoing REPACK
> CONCURRENTLY; any alterations would have to wait until the repacking is
> compelted. We can implement that idea simply enough, as shown in this
> crude prototype.
>
After testing this, I observed that it solves the scenario where a query is
waiting
on REPACK. For example, if a DROP TABLE requests an AEL and queues
behind REPACK's ShareUpdateExclusiveLock, the deadlock detector comes
when REPACK tries to upgrade to AEL, killing the DROP to prevent the
circular
queue deadlock, But the case I originally mentioned [1] was the reverse:
what
happens if a transaction already holds a lock that conflicts with the
upcoming
AEL upgrade (e.g., an analytical SELECT or an idle-in-transaction holding
an AccessShareLock),
but isn't waiting on REPACK at all?
In this case, there's no circular wait. The deadlock detector never fires.
REPACK
simply queues behind the SELECT, eventually hits its lock_timeout, aborts
and
cleans up.Initially, I thought this cleanup was expected behavior. But
after seeing
your solution to protect REPACK from losing its transient table work, I
thought it's "not expected".
If the goal is to prevent REPACK's work from being wasted, should we error
out
the backend that is making REPACK wait during the final swap phase? I am
thinking
of something conceptually similar to
ResolveRecoveryConflictWithLock,actively
cancelling the conflicting session to allow the AEL upgrade to proceed.
Thoughts?
test scenario:
session 1:
postgres=# repack (concurrently) stress_victim;
had a breakpoint rebuild_relation_finish_concurrent->
LockRelationOid(old_table_oid, AccessExclusiveLock); just before getting
the exclusive lock.
with lock_timeout = 5s
session 2:
postgres=# BEGIN;
SELECT * FROM stress_victim LIMIT 1;
-- left it open
BEGIN
id | balance |
payload
-----+---------+---------------------------------
-------------------------------------------------
-------------------------------------------------
-------------------------------------------------
--------------
170 | 65 | d12f400c4d0d3c49818f88597e16cf29
d12f400c4d0d3c49818f88597e16cf29d12f400c4d0d3c498
18f88597e16cf29d12f400c4d0d3c49818f88597e16cf29d1
2f400c4d0d3c49818f88597e16cf29d12f400c4d0d3c49818
f88597e16cf29
(1 row)
-- this gets us a conflicting lock (AccessShareLock) on the same table,
REPACK (concurrently) is running on.
session 1:
release the breakpoint and now the backend waits for the conflicting lock
to be released.
in between if lock_timeout occurs then transaction aborts.
postgres=# repack (concurrently) stress_victim;
ERROR: canceling statement due to lock timeout
CONTEXT: waiting for AccessExclusiveLock on relation 16637 of database 5
[1] -
https://www.postgresql.org/message-id/CAFC%2Bb6pK9ogeSpMA8hg18XhC1eNPcsKWBwoC5OySXi4iTxwtRw%40mail.g...
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 17:37 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-31 18:22 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-31 18:22 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> On Thu, Mar 26, 2026 at 1:42 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> As for lock upgrade, I wonder if the best way to handle this isn't to
> hack the deadlock detector so that it causes any *other* process to die,
> if they detect that they would block on REPACK. Arguably there's
> nothing that you can do to a table while its undergoing REPACK
> CONCURRENTLY; any alterations would have to wait until the repacking is
> compelted. We can implement that idea simply enough, as shown in this
> crude prototype.
>
> After testing this, I observed that it solves the scenario where a query is waiting
> on REPACK. For example, if a DROP TABLE requests an AEL and queues
> behind REPACK's ShareUpdateExclusiveLock, the deadlock detector comes
> when REPACK tries to upgrade to AEL, killing the DROP to prevent the circular
> queue deadlock, But the case I originally mentioned [1] was the reverse: what
> happens if a transaction already holds a lock that conflicts with the upcoming
> AEL upgrade (e.g., an analytical SELECT or an idle-in-transaction holding an AccessShareLock),
> but isn't waiting on REPACK at all?
>
> In this case, there's no circular wait. The deadlock detector never fires. REPACK
> simply queues behind the SELECT, eventually hits its lock_timeout, aborts and
> cleans up.
Why should the user set non-zero lock_timeout before running REPACK?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 17:37 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-31 18:23 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 09:39 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-31 18:23 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-31, Srinath Reddy Sadipiralla wrote:
> In this case, there's no circular wait. The deadlock detector never
> fires. REPACK simply queues behind the SELECT, eventually hits its
> lock_timeout, aborts and cleans up.Initially, I thought this cleanup
> was expected behavior. But after seeing your solution to protect
> REPACK from losing its transient table work, I thought it's "not
> expected".
Yeah. Keep in mind that REPACK could have been running for many hours
or even days before it reaches the point of acquiring its AEL lock to do
the final swap; and it may well be critical work. We do not want to
lose it. So whatever is waiting to obtain a lock on the table, or
already has a lock on the table, has to yield.
> If the goal is to prevent REPACK's work from being wasted, should we
> error out the backend that is making REPACK wait during the final swap
> phase? I am thinking of something conceptually similar to
> ResolveRecoveryConflictWithLock, actively cancelling the conflicting
> session to allow the AEL upgrade to proceed.
Something like that might be appropriate, yeah.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"El miedo atento y previsor es la madre de la seguridad" (E. Burke)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 17:37 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-31 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-01 09:39 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 11:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-04-01 09:39 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, Mar 31, 2026 at 11:54 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Mar-31, Srinath Reddy Sadipiralla wrote:
>
> > In this case, there's no circular wait. The deadlock detector never
> > fires. REPACK simply queues behind the SELECT, eventually hits its
> > lock_timeout, aborts and cleans up.Initially, I thought this cleanup
> > was expected behavior. But after seeing your solution to protect
> > REPACK from losing its transient table work, I thought it's "not
> > expected".
>
> Yeah. Keep in mind that REPACK could have been running for many hours
> or even days before it reaches the point of acquiring its AEL lock to do
> the final swap; and it may well be critical work. We do not want to
> lose it. So whatever is waiting to obtain a lock on the table, or
> already has a lock on the table, has to yield.
>
> > If the goal is to prevent REPACK's work from being wasted, should we
> > error out the backend that is making REPACK wait during the final swap
> > phase? I am thinking of something conceptually similar to
> > ResolveRecoveryConflictWithLock, actively cancelling the conflicting
> > session to allow the AEL upgrade to proceed.
>
> Something like that might be appropriate, yeah.
>
What about if the blocking process is an autovacumm that is working to
prevent XID wraparound? I think we already avoid killing it in such
cases. BTW, one can say that cancelling a long-running report query
also wastes a lot of effort of the user generating such a report. Why
can't REPACK wait for such a select to finish instead of cancelling
it?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 17:37 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-31 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 09:39 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-01 11:38 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 12:21 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-01 11:38 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-01, Amit Kapila wrote:
> What about if the blocking process is an autovacumm that is working to
> prevent XID wraparound? I think we already avoid killing it in such
> cases.
If we just let REPACK finish, it will also renew the table's XID, so
autovacuum is not needed in that case. I mean, there's no reason to let
autovacuum process the contents of a table that is going to be replaced
completely.
One potentially problematic case would be that an emergency autovacuum
has been running for a long time and about to finish, and REPACK is
started. But in that case, autovacuum already has ShareUpdateExclusive,
so REPACK wouldn't be able to start at all, which means it won't kill
autovacuum. And in the case where autovacuum is doing emergency
vacuuming, then it won't commit suicide, so it will be able to complete
before repack starts.
> BTW, one can say that cancelling a long-running report query also
> wastes a lot of effort of the user generating such a report. Why can't
> REPACK wait for such a select to finish instead of cancelling it?
I don't understand exactly which scenario you're concerned about. Is
there a long-running query which, after spending a lot of time running a
report, tries to upgrade its lock level on the table? Keep in mind that
this check only runs when the affected session runs the deadlock
checker, which means it's been waiting to acquire a lock for
deadlock_timeout seconds. It's not repack that kills the query.
[ ... reflects ...] Oh, actually what Srinath proposed does exactly
that -- kill other queries. Hmm, yeah, I'm less sure about that
particular bit. Here I'm only talking about my proposal to have the
deadlock detector handle the case of somebody waiting to lock the table.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 17:37 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-31 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 09:39 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 11:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-01 12:21 ` Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Amit Kapila @ 2026-04-01 12:21 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, Apr 1, 2026 at 5:08 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Apr-01, Amit Kapila wrote:
>
> > What about if the blocking process is an autovacumm that is working to
> > prevent XID wraparound? I think we already avoid killing it in such
> > cases.
>
> If we just let REPACK finish, it will also renew the table's XID, so
> autovacuum is not needed in that case. I mean, there's no reason to let
> autovacuum process the contents of a table that is going to be replaced
> completely.
>
> One potentially problematic case would be that an emergency autovacuum
> has been running for a long time and about to finish, and REPACK is
> started. But in that case, autovacuum already has ShareUpdateExclusive,
> so REPACK wouldn't be able to start at all, which means it won't kill
> autovacuum. And in the case where autovacuum is doing emergency
> vacuuming, then it won't commit suicide, so it will be able to complete
> before repack starts.
>
> > BTW, one can say that cancelling a long-running report query also
> > wastes a lot of effort of the user generating such a report. Why can't
> > REPACK wait for such a select to finish instead of cancelling it?
>
> I don't understand exactly which scenario you're concerned about. Is
> there a long-running query which, after spending a lot of time running a
> report, tries to upgrade its lock level on the table? Keep in mind that
> this check only runs when the affected session runs the deadlock
> checker, which means it's been waiting to acquire a lock for
> deadlock_timeout seconds. It's not repack that kills the query.
>
> [ ... reflects ...] Oh, actually what Srinath proposed does exactly
> that -- kill other queries. Hmm, yeah, I'm less sure about that
> particular bit.
>
Yes, I was talking about Srinath's proposed solution. Do we need to do
anything about it?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-25 09:00 ` Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 16:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-26 17:28 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 3 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-25 09:00 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> - 0008 to 0010 are as posted by Antonin; they are unchanged, except for
> fixes for the problems pointed out by Mihail. Antonin, I would
> appreciate it if you want to change the "reform" bit in 0007 as
> discussed.
I've taken a look, but not sure if the tuple slots help here. In
heapam_relation_copy_for_cluster(), both table_scan_getnextslot() and
index_getnext_slot() call ExecStoreBufferHeapTuple() ->
tts_buffer_heap_store_tuple(), which AFAICS do not deform the tuple. Then
ExecFetchSlotHeapTuple() is used to retrieve the tuple, but again, the
underlying slot (TTSOpsBufferHeapTuple) handles it by copying rather than
deforming / forming. Thus I think the explicit "reforming" currently does not
add any performance overhead.
Of course, we can still use the slots, and do the following: 1) enforce tuple
deforming (by calling slot_getallattrs()), 2) set the dropped attributes to
NULL, 3) use ExecStoreVirtualTuple() to store the tuple into another slot and
4) get the heap tuple from the other slot. Should I do that? I'm asking
because I wasn't sure if you're concerned about performance or coding (or
both).
Whatever approach we take, I see two more opportunities for better
performance:
1. Do the "reforming" only if there are some dropped columns. (AFAICS even the
old CLUSTER / VACUUM FULL did not check this.)
2. Get rid of the values of dropped columns earlier, so that the dropped
values are not put into the tuplestore (likewise, I think that CLUSTER /
VACUUM FULL did not care.)
Besides that, I think that heap_form_tuple() should set the values of dropped
columns to NULL by default, or do I miss something? Anyway, this should be
addressed by a separate patch.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-25 15:52 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 15:59 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:15 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 2 replies; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-03-25 15:52 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
While reviewing/testing V44 patch set , i found that if we run REPACK
(CONCURRENTLY)
without a table name inside a transaction block throws the error "REPACK
CONCURRENTLY requires explicit
table name" instead of the expected transaction block error. This occurs
because ExecRepack() validates the parsed options and missing relation
before verifying the transaction state.
I attached a patch below to maintain consistency with other commands like
VACUUM, REINDEX , and more
and also not to confuse the user , because if user runs REPACK
(CONCURRENTLY)
without a table name inside a transaction block, if user gets "REPACK
CONCURRENTLY requires explicit
table name" and then to correct the mistake the user gives table and again
runs the in
transaction block , just to find out a new error "cannot run inside a
transaction block".
psql (19devel)
Type "help" for help.
postgres=# BEGIN;
SET TRANSACTION READ ONLY;
BEGIN
SET
postgres=*# REPACK (concurrently);
ERROR: REPACK CONCURRENTLY requires explicit table name
psql (19devel)
Type "help" for help.
postgres=# BEGIN;
SET TRANSACTION READ ONLY;
BEGIN
SET
postgres=*# REPACK (concurrently) stress_victim;
ERROR: REPACK (CONCURRENTLY) cannot run inside a transaction block
After the fix:
psql (19devel)
Type "help" for help.
postgres=# BEGIN;
SET TRANSACTION READ ONLY;
BEGIN
SET
postgres=*# REPACK (concurrently);
ERROR: REPACK (CONCURRENTLY) cannot run inside a transaction block
Thoughts?
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] 0001-Check-for-transaction-block-early-in-ExecRepack.patch (3.5K, ../../CAFC+b6q6EsNqZ4+QugmMugbGChuJfu4HorJ3hZ0f6cyxCAxFgQ@mail.gmail.com/3-0001-Check-for-transaction-block-early-in-ExecRepack.patch)
download | inline diff:
From 6e4d941cabbc16c60d778345558612208d78d881 Mon Sep 17 00:00:00 2001
From: Srinath Reddy Sadipiralla <srinath2133@gmail.com>
Date: Wed, 25 Mar 2026 20:53:09 +0530
Subject: [PATCH 1/1] Check for transaction block early in ExecRepack
Currently, executing REPACK (CONCURRENTLY) without a table name inside a
transaction block throws the error "REPACK CONCURRENTLY requires explicit
table name" instead of the expected transaction block error. This occurs
because ExecRepack() validates the parsed options and missing relation
before verifying the transaction state.
This behavior is inconsistent with other utility commands like VACUUM
,REINDEX, etc; which invoke PreventInTransactionBlock() at the very start
of their execution to properly reject execution inside user transactions
before validating targets.
Add PreventInTransactionBlock to the top of ExecRepack() to enforce the
transaction block restriction early. This prevents the user from fixing
a missing table error only to immediately hit a transaction block error,
and also ensures consistency with rest of the commands.
---
src/backend/commands/cluster.c | 39 +++++++++++++++++++---------------
1 file changed, 22 insertions(+), 17 deletions(-)
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 423cea26b0b..38c58f6df6e 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -369,6 +369,28 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
parser_errposition(pstate, opt->location));
}
+ if (params.options & CLUOPT_CONCURRENT)
+ {
+ /*
+ * Make sure we're not in a transaction block.
+ *
+ * The reason is that repack_setup_logical_decoding() could deadlock
+ * if there's an XID already assigned. It would be possible to run in
+ * a transaction block if we had no XID, but this restriction is
+ * simpler for users to understand and we don't lose anything.
+ */
+ PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
+ }
+ else
+ {
+ /*
+ * In order to avoid holding locks for too long, we want to process
+ * each table in its own transaction. This forces us to disallow
+ * running inside a user transaction block.
+ */
+ PreventInTransactionBlock(isTopLevel, RepackCommandAsString(stmt->command));
+ }
+
/* Determine the lock mode to use. */
lockmode = RepackLockLevel((params.options & CLUOPT_CONCURRENT) != 0);
@@ -413,13 +435,6 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
errmsg("REPACK CONCURRENTLY requires explicit table name"));
}
- /*
- * In order to avoid holding locks for too long, we want to process each
- * table in its own transaction. This forces us to disallow running
- * inside a user transaction block.
- */
- PreventInTransactionBlock(isTopLevel, RepackCommandAsString(stmt->command));
-
/* Also, we need a memory context to hold our list of relations */
repack_context = AllocSetContextCreate(PortalContext,
"Repack",
@@ -605,16 +620,6 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
*/
if (concurrent)
{
- /*
- * Make sure we're not in a transaction block.
- *
- * The reason is that repack_setup_logical_decoding() could deadlock
- * if there's an XID already assigned. It would be possible to run in
- * a transaction block if we had no XID, but this restriction is
- * simpler for users to understand and we don't lose anything.
- */
- PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
-
check_repack_concurrently_requirements(OldHeap, &ident_idx);
}
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-25 15:59 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-03-25 15:59 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Sorry my bad :( , I attached my patch with *.patch instead of *.something
or "nocfbot" prefix.
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-26 19:15 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-27 18:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-26 19:15 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-25, Srinath Reddy Sadipiralla wrote:
> Hello,
>
> While reviewing/testing V44 patch set , i found that if we run REPACK
> (CONCURRENTLY) without a table name inside a transaction block throws
> the error "REPACK CONCURRENTLY requires explicit table name" instead
> of the expected transaction block error. This occurs because
> ExecRepack() validates the parsed options and missing relation before
> verifying the transaction state.
>
> I attached a patch below to maintain consistency with other commands
> like VACUUM, REINDEX , and more and also not to confuse the user ,
> because if user runs REPACK (CONCURRENTLY) without a table name inside
> a transaction block, if user gets "REPACK CONCURRENTLY requires
> explicit table name" and then to correct the mistake the user gives
> table and again runs the in transaction block , just to find out a new
> error "cannot run inside a transaction block".
I don't disagree with changing this, but AFAICS the patch as presented
provokes multiple test failures.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:15 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-27 18:42 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-02 00:35 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-03-27 18:42 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi Alvaro,
On Fri, Mar 27, 2026 at 12:45 AM Alvaro Herrera <alvherre@alvh.no-ip.org>
wrote:
>
> I don't disagree with changing this, but AFAICS the patch as presented
> provokes multiple test failures.
>
Fixed with the attached patch.
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] nocfbot-v2-0001-Check-for-transaction-block-early-in-ExecRepack.patch (2.9K, ../../CAFC+b6oJkNoFEgDUNBOdspWkPGQbeaGohE=W179PKwEegp8YSw@mail.gmail.com/3-nocfbot-v2-0001-Check-for-transaction-block-early-in-ExecRepack.patch)
download | inline diff:
From cb3a14f8931d6be9135982bb6378e892abeb2d55 Mon Sep 17 00:00:00 2001
From: Srinath Reddy Sadipiralla <srinath2133@gmail.com>
Date: Fri, 27 Mar 2026 23:59:15 +0530
Subject: [PATCH v2 1/1] Check for transaction block early in ExecRepack
Currently, executing REPACK (CONCURRENTLY) without a table name inside a
transaction block throws the error "REPACK CONCURRENTLY requires explicit
table name" instead of the expected transaction block error. This occurs
because ExecRepack() validates the parsed options and missing relation
before verifying the transaction state.
This behavior is inconsistent with other utility commands like VACUUM
,REINDEX, etc; which invoke PreventInTransactionBlock() at the very start
of their execution to properly reject execution inside user transactions
before validating targets.
Add PreventInTransactionBlock to the top of ExecRepack() to enforce the
transaction block restriction early. This prevents the user from fixing
a missing table error only to immediately hit a transaction block error,
and also ensures consistency with rest of the commands.
---
src/backend/commands/cluster.c | 27 ++++++++++++++-------------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 77c206ff944..6d93b21df0a 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -384,6 +384,20 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
parser_errposition(pstate, opt->location));
}
+ if (params.options & CLUOPT_CONCURRENT)
+ {
+ /*
+ * Make sure we have no XID assigned, otherwise call of
+ * repack_setup_logical_decoding() can cause a deadlock.
+ *
+ * The existence of transaction block actually does not imply that XID
+ * was already assigned, but it very likely is. We might want to check
+ * the result of GetCurrentTransactionIdIfAny() instead, but that
+ * would be less clear from user's perspective.
+ */
+ PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
+ }
+
/*
* Determine the lock mode expected by cluster_rel().
*
@@ -608,20 +622,7 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
/* There are specific requirements on concurrent processing. */
if (concurrent)
- {
- /*
- * Make sure we have no XID assigned, otherwise call of
- * repack_setup_logical_decoding() can cause a deadlock.
- *
- * The existence of transaction block actually does not imply that XID
- * was already assigned, but it very likely is. We might want to check
- * the result of GetCurrentTransactionIdIfAny() instead, but that
- * would be less clear from user's perspective.
- */
- PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
-
check_repack_concurrently_requirements(OldHeap, &ident_idx);
- }
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:15 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-27 18:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-04-02 00:35 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-02 08:27 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-02 00:35 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Sat, Mar 28, 2026 at 12:12 AM Srinath Reddy Sadipiralla <
srinath2133@gmail.com> wrote:
> Hi Alvaro,
>
> On Fri, Mar 27, 2026 at 12:45 AM Alvaro Herrera <alvherre@alvh.no-ip.org>
> wrote:
>
>>
>> I don't disagree with changing this, but AFAICS the patch as presented
>> provokes multiple test failures.
>>
>
> Fixed with the attached patch.
>
Just want to remind about this patch.
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:15 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-27 18:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-02 00:35 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-04-02 08:27 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 15:58 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-02 08:27 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-02, Srinath Reddy Sadipiralla wrote:
> On Sat, Mar 28, 2026 at 12:12 AM Srinath Reddy Sadipiralla <
> srinath2133@gmail.com> wrote:
>
> > Hi Alvaro,
> >
> >> I don't disagree with changing this, but AFAICS the patch as presented
> >> provokes multiple test failures.
> >
> > Fixed with the attached patch.
>
> Just want to remind about this patch.
Yeah, it doesn't apply anymore. I rebased it some time ago, but it
still failed a few tests -- I'm guessing you don't have either TAP tests
or injection points enabled, which would explain why you don't see those
failures. So please rebase it against v49, but also look into making it
pass all tests (I suggest making it go through CI).
Thanks,
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"We’ve narrowed the problem down to the customer’s pants being in a situation
of vigorous combustion" (Robert Haas, Postgres expert extraordinaire)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:15 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-27 18:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-02 00:35 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-02 08:27 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-03 15:58 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-03 15:58 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Thu, Apr 2, 2026 at 1:57 PM Alvaro Herrera <alvherre@alvh.no-ip.org>
wrote:
>
> Yeah, it doesn't apply anymore. I rebased it some time ago, but it
> still failed a few tests -- I'm guessing you don't have either TAP tests
> or injection points enabled, which would explain why you don't see those
> failures. So please rebase it against v49, but also look into making it
> pass all tests (I suggest making it go through CI).
>
Rebased on V50 and check-world passed.
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] nocfbot-0001-Check-for-transaction-block-early-in-ExecRepack.patch (2.9K, ../../CAFC+b6q87-q0XETwMjBRMQk2Yvz8HtY6qPK6kG01HVLGtzjEZA@mail.gmail.com/3-nocfbot-0001-Check-for-transaction-block-early-in-ExecRepack.patch)
download | inline diff:
From 97ef5eca6ebf9d3932d624ed4e63d8387cc89c2a Mon Sep 17 00:00:00 2001
From: Srinath Reddy Sadipiralla <srinath2133@gmail.com>
Date: Fri, 3 Apr 2026 20:53:56 +0530
Subject: [PATCH 1/1] Check for transaction block early in ExecRepack
Currently, executing REPACK (CONCURRENTLY) without a table name inside a
transaction block throws the error "REPACK CONCURRENTLY requires explicit
table name" instead of the expected transaction block error. This occurs
because ExecRepack() validates the parsed options and missing relation
before verifying the transaction state.
This behavior is inconsistent with other utility commands like VACUUM
,REINDEX, etc; which invoke PreventInTransactionBlock() at the very start
of their execution to properly reject execution inside user transactions
before validating targets.
Add PreventInTransactionBlock to the top of ExecRepack() to enforce the
transaction block restriction early. This prevents the user from fixing
a missing table error only to immediately hit a transaction block error,
and also ensures consistency with rest of the commands.
---
src/backend/commands/repack.c | 26 ++++++++++++--------------
1 file changed, 12 insertions(+), 14 deletions(-)
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 2fc30008f59..d932d526235 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -292,6 +292,18 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
MyProc->statusFlags |= PROC_IN_CONCURRENT_REPACK;
ProcGlobal->statusFlags[MyProc->pgxactoff] = MyProc->statusFlags;
LWLockRelease(ProcArrayLock);
+
+ /*
+ * Make sure we're not in a transaction block.
+ *
+ * The reason is that repack_setup_logical_decoding() could wait
+ * indefinitely for our XID to complete. (The deadlock detector would
+ * not recognize it because we'd be waiting for ourselves, i.e. no
+ * real lock conflict.) It would be possible to run in a transaction
+ * block if we had no XID, but this restriction is simpler for users
+ * to understand and we don't lose anything.
+ */
+ PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
}
/*
@@ -523,21 +535,7 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
* replica index OID.
*/
if (concurrent)
- {
- /*
- * Make sure we're not in a transaction block.
- *
- * The reason is that repack_setup_logical_decoding() could wait
- * indefinitely for our XID to complete. (The deadlock detector would
- * not recognize it because we'd be waiting for ourselves, i.e. no
- * real lock conflict.) It would be possible to run in a transaction
- * block if we had no XID, but this restriction is simpler for users
- * to understand and we don't lose anything.
- */
- PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
-
check_repack_concurrently_requirements(OldHeap, &ident_idx);
- }
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-25 16:52 ` Antonin Houska <ah@cybertec.at>
2 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-25 16:52 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > - 0008 to 0010 are as posted by Antonin; they are unchanged, except for
> > fixes for the problems pointed out by Mihail. Antonin, I would
> > appreciate it if you want to change the "reform" bit in 0007 as
> > discussed.
>
> I've taken a look, but not sure if the tuple slots help here. In
> heapam_relation_copy_for_cluster(), both table_scan_getnextslot() and
> index_getnext_slot() call ExecStoreBufferHeapTuple() ->
> tts_buffer_heap_store_tuple(), which AFAICS do not deform the tuple. Then
> ExecFetchSlotHeapTuple() is used to retrieve the tuple, but again, the
> underlying slot (TTSOpsBufferHeapTuple) handles it by copying rather than
> deforming / forming. Thus I think the explicit "reforming" currently does not
> add any performance overhead.
Well, the deform / form steps do add some overhead of course, but these are
necessary to get rid of the values of the dropped columns. I wanted to say
that it wouldn't be cheaper with slots, because then we'd have to enforce the
deform / form steps too, although the coding would be different:
> Of course, we can still use the slots, and do the following: 1) enforce tuple
> deforming (by calling slot_getallattrs()), 2) set the dropped attributes to
> NULL, 3) use ExecStoreVirtualTuple() to store the tuple into another slot and
> 4) get the heap tuple from the other slot. Should I do that? I'm asking
> because I wasn't sure if you're concerned about performance or coding (or
> both).
> Whatever approach we take, I see two more opportunities for better
> performance:
>
> 1. Do the "reforming" only if there are some dropped columns. (AFAICS even the
> old CLUSTER / VACUUM FULL did not check this.)
I think this would need more work because CLUSTER / VACCUM FULL / REPACK do
not remove the dropped attributes from the tuple descriptor. So the
optimization would only work until the first column is dropped. All the
following runs would then do the reforming even if no other colmns were droped
since the previous run.
Perhaps we can teach REPACK to remove dropped columns from the tuple
descriptor in the future.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-26 17:28 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-26 17:28 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-25, Antonin Houska wrote:
> I've taken a look, but not sure if the tuple slots help here. In
> heapam_relation_copy_for_cluster(), both table_scan_getnextslot() and
> index_getnext_slot() call ExecStoreBufferHeapTuple() ->
> tts_buffer_heap_store_tuple(), which AFAICS do not deform the tuple.
> Then ExecFetchSlotHeapTuple() is used to retrieve the tuple, but
> again, the underlying slot (TTSOpsBufferHeapTuple) handles it by
> copying rather than deforming / forming. Thus I think the explicit
> "reforming" currently does not add any performance overhead.
Looking again, I think we should leave this alone for now. The existing
code (looking at heapam_relation_copy_for_cluster) is somewhat silly, in
that we do a bunch of operations with slots, then we heap-ify the tuple
by doing ExecFetchSlotHeapTuple(), then we do the reform_and_rewrite
dance, which must deform the tuple only to immediately form it back; and
after we've done that, we make it go through the rewriteheap.c motions,
which again deforms and forms back (in certain cases).
This code structure seems mostly a relic from the time when heapam.c was
all we had, and I think we should rewrite it from scratch to work using
only slots. But not for this patch, and not for pg19, because I think
that's going to take some nontrivial effort on its own. So for REPACK
in pg19 I think we should just do the simplest thing possible, and do
the minimum amount of deform/form dance necessary to make this whole
thing work. I suppose that that's what the submitted code (v44-0008)
does, so I'm going to leave it at that.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"In Europe they call me Niklaus Wirth; in the US they call me Nickel's worth.
That's because in Europe they call me by name, and in the US by value!"
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-03-26 08:23 ` Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-26 08:23 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hello, everyone!
>
> Some comments for v43:
Thanks!
> ------------------------------
>
> src/backend/catalog/index.c:766,1464-1469
>
> "bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;"
>
> AFAIU gin, hash and btree (at least) still just unconditionally write
> PROGRESS_CREATEIDX_* progress.
I think you're right, I'll check it.
> src/backend/catalog/toasting.c:334
>
> "INDEX_CREATE_IS_PRIMARY | INDEX_CREATE_REPORT_PROGRESS, 0,"
>
> Should we add the "progress" flag too here and move it from make_new_heap?
Good catch. I'd prefer removing this hunk from the patch because the index on
the TOAST relation is created while it's still empty.
(The other commens have already been addressed by Alvaro.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-27 16:12 ` Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-27 16:12 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > src/backend/catalog/index.c:766,1464-1469
> >
> > "bool progress = (flags & INDEX_CREATE_REPORT_PROGRESS) != 0;"
> >
> > AFAIU gin, hash and btree (at least) still just unconditionally write
> > PROGRESS_CREATEIDX_* progress.
>
> I think you're right, I'll check it.
I concluded this is a problem that exists for quite a while and should be
fixed separately. Currently I don't see conflicts of parameter indexes between
PROGRESS_COMMAND_REPACK and PROGRESS_COMMAND_CREATE_INDEX. There would be some
if the index was built with the CONCURRENTLY option, but REPACK uses normal
index build.
I tried to write a patch that allows progress tracking of two commands at the
same time (a "main command" and a "subcommand"), but regression tests revealed
that contrib/file_fdw is broken in a way that I could not fix easily: during
execution of a join, two COPY FROM commands are executed at the same time and
they overwrite the status of each other. Unlike the concept of a sub-command,
we cannot assume here that the command that started the reporting as the
second will stop as the first. Thus in pgstat_progress_end_command() we cannot
figure out which node is trying to stop the reporting.
It needs more work, I can get back to it after PG 19 feature freeze.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-27 17:01 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-29 22:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-27 17:01 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-27, Antonin Houska wrote:
> I tried to write a patch that allows progress tracking of two commands at the
> same time (a "main command" and a "subcommand"), but regression tests revealed
> that contrib/file_fdw is broken in a way that I could not fix easily: during
> execution of a join, two COPY FROM commands are executed at the same time and
> they overwrite the status of each other. Unlike the concept of a sub-command,
> we cannot assume here that the command that started the reporting as the
> second will stop as the first. Thus in pgstat_progress_end_command() we cannot
> figure out which node is trying to stop the reporting.
Yeah, I think we've discussed this kind of thing in the context of the
index creation by CLUSTER before, but no ideas have emerged on what the
best implementation is.
> It needs more work, I can get back to it after PG 19 feature freeze.
Yeah, clearly it's not getting in pg19. I'm not terribly upset about
that though; it would be nice to have, but it's not a dealbreaker.
Anyway, I've been changing how all the new code for REPACK is
distributed; here's patch series v45, and the breakdown and my
assessment is:
0001 is looking good. I don't commit it just yet because there's no
point in doing so until 0003 is also committable.
0002 continues to baffle me. I would much rather do without it.
0003 is the addition of CONCURRENTLY and all it needs. Regarding
structure, I think it's good to have the code that runs in a separate
worker have its own file, which I named commands/repack_worker.c.
Luckily it's all well contained. It shares data structures and whatnot
with the other parts of REPACK via include/commands/repack_internal.h.
Patch "Use BulkInsertState when copying data to the new heap." is now
0004 and I put it right after 0003, and I will probably be squashing
them into a single one.
0005 is new -- I renamed cluster.c to repack.c, as previously mentioned.
Then 0006 is "Fix a few problems in index build progress reporting",
which I'm likely to also squash in 0003 for v46.
0007 is "Error out any process that would block at REPACK" which is the
complete patch to change the deadlock detector to raise an error at any
process that competes with REPACK. This one I think I would commit
separately from 0003 -- if nothing else, because it introduces novel
behavior that could be contested, so I want to be able to easily revert
it without endangering the rest of CONCURRENTLY.
Lastly, 0008 is "Teach snapshot builder to skip transactions running
REPACK (CONCURRENTLY)" which I see as the least mature of the pack. I
would really like to be able to squash it with 0003, but I'm not yet
trusting it enough.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"No nos atrevemos a muchas cosas porque son difíciles,
pero son difíciles porque no nos atrevemos a hacerlas" (Séneca)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-29 22:02 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-29 22:02 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-27, Alvaro Herrera wrote:
> 0001 is looking good. I don't commit it just yet because there's no
> point in doing so until 0003 is also committable.
>
> 0002 continues to baffle me. I would much rather do without it.
>
> 0003 is the addition of CONCURRENTLY and all it needs. Regarding
> structure, I think it's good to have the code that runs in a separate
> worker have its own file, which I named commands/repack_worker.c.
> Luckily it's all well contained. It shares data structures and whatnot
> with the other parts of REPACK via include/commands/repack_internal.h.
I figured out that the right way to handle SET_VARSIZE() problem was to
use a union, giving sufficient space so that the compiler doesn't
complain. So I got rid of 0002 here. There are no other interesting
changes in this version compared to v45, and I didn't get around to
squashing anything.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-31 10:47 ` Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-03-31 10:47 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Fri, Mar 27, 2026 at 10:31 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Lastly, 0008 is "Teach snapshot builder to skip transactions running
> REPACK (CONCURRENTLY)" which I see as the least mature of the pack. I
> would really like to be able to squash it with 0003, but I'm not yet
> trusting it enough.
>
Few comments/questions by looking 0008 alone:
1.
+ TransactionId *xids_repack = NULL;
+ bool logical_decoding_enabled = IsLogicalDecodingEnabled();
Assert(!RecoveryInProgress());
...
...
/*
* Allocating space for maxProcs xids is usually overkill; numProcs would
* be sufficient. But it seems better to do the malloc while not holding
@@ -2663,11 +2672,14 @@ GetRunningTransactionData(void)
*/
if (CurrentRunningXacts->xids == NULL)
{
+ /* FIXME probably fails if logical decoding is enable on-the-fly */
+ int nrepack = logical_decoding_enabled ? MAX_REPACK_XIDS : 0;
This FIXME is important to fix for this patch, otherwise, we can't
rely on transactions remembered as repack_concurrently.
2.
*
+ /*
+ * TODO Consider a GUC to reserve certain amount of replication slots for
+ * REPACK (CONCURRENTLY) and using it here.
+ */
+#define MAX_REPACK_XIDS 16
+
This sounds a bit scary as reserving replication slots for REPACK
(CONCURRENTLY) may not be what users expect. But it is not clear why
replication slots need to be reserved for this.
IIUC, two reasons why logical decoding can ignore REPACK
(CONCURRENTLY) are (a) does not perform any catalog changes relevant
to logical decoding, (b) neither walsenders nor backends performing
logical decoding needs to care sending the WAL generated by REPACK
(CONCURRENTLY). Is that understanding correct? If so, we may want to
clarify why we want to ignore this command's WAL while sending changes
downstream in the commit message or give some reference of the patch
where the same is mentioned. This can help reviewing this patch
independently.
BTW, are we intending to commit this patch series for PG19?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-03-31 14:36 ` Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-31 14:36 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Fri, Mar 27, 2026 at 10:31 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > Lastly, 0008 is "Teach snapshot builder to skip transactions running
> > REPACK (CONCURRENTLY)" which I see as the least mature of the pack. I
> > would really like to be able to squash it with 0003, but I'm not yet
> > trusting it enough.
> >
>
> Few comments/questions by looking 0008 alone:
Thanks.
> 1.
> + TransactionId *xids_repack = NULL;
> + bool logical_decoding_enabled = IsLogicalDecodingEnabled();
>
> Assert(!RecoveryInProgress());
>
> ...
> ...
>
> /*
> * Allocating space for maxProcs xids is usually overkill; numProcs would
> * be sufficient. But it seems better to do the malloc while not holding
> @@ -2663,11 +2672,14 @@ GetRunningTransactionData(void)
> */
> if (CurrentRunningXacts->xids == NULL)
> {
> + /* FIXME probably fails if logical decoding is enable on-the-fly */
> + int nrepack = logical_decoding_enabled ? MAX_REPACK_XIDS : 0;
>
> This FIXME is important to fix for this patch, otherwise, we can't
> rely on transactions remembered as repack_concurrently.
Indeed.
> 2.
> *
> + /*
> + * TODO Consider a GUC to reserve certain amount of replication slots for
> + * REPACK (CONCURRENTLY) and using it here.
> + */
> +#define MAX_REPACK_XIDS 16
> +
>
> This sounds a bit scary as reserving replication slots for REPACK
> (CONCURRENTLY) may not be what users expect. But it is not clear why
> replication slots need to be reserved for this.
The point is that REPACK should not block replication [1]. Thus reserving
slots for non-REPACK users is probably more precise statement.
> IIUC, two reasons why logical decoding can ignore REPACK
> (CONCURRENTLY) are (a) does not perform any catalog changes relevant
> to logical decoding, (b) neither walsenders nor backends performing
> logical decoding needs to care sending the WAL generated by REPACK
> (CONCURRENTLY). Is that understanding correct? If so, we may want to
> clarify why we want to ignore this command's WAL while sending changes
> downstream in the commit message or give some reference of the patch
> where the same is mentioned. This can help reviewing this patch
> independently.
Correct, but in fact this diff only affects the setup of the logical decoding
rather than the decoding itself. On the other hand, if REPACK (CONCURRENTLY)
starts when the decoding backend's snapshot builder is already in the
SNAPBUILD_FULL_SNAPSHOT state, reorderbuffer.c processes the transaction
normally, and another part of the series (v46-0002) ensures that the table
rewriting is not decoded: REPACK simply tells heap_insert(), heap_update(),
heap_delete() not to put the extra (replication specific) information into the
corresponding WAL records. I suppose this is what you mean in (b).
Regarding (a), yes, the absence of catalog changes in the REPACK's transaction
is the reason that even the logical decoding setup does not have to wait for
the transaction to finish. AFAIU the reason the snapshot builder needs to wait
for completion of (non-REPACK) transaction started before
SNAPBUILD_FULL_SNAPSHOT was reached is exactly that the transaction might have
performed catalog changes before its decoding started, so we do not know for
sure if it contains catalog changes or not.
> BTW, are we intending to commit this patch series for PG19?
Yes, that's the current plan.
[1] https://www.postgresql.org/message-id/CABV9wwMQkN6wOxMnd1h95eqpC7wEqivBzsBzCp3VnxGFk%3DvDUw%40mail.g...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-31 15:35 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-31 15:35 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-31, Antonin Houska wrote:
> Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > On Fri, Mar 27, 2026 at 10:31 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > BTW, are we intending to commit this patch series for PG19?
>
> Yes, that's the current plan.
Yes -- though as I said upthread, this particular patch in the series is
the one I'm the least sure about. Also, the structure of the patches to
commit is not like the ones posted here or previously. I'll post a
committable one later on; see below for a breakdown. For now, I brought
the addition of options to table AM methods from [1] into this series,
in what I think is pretty much final form (0002); and I added a 0004
patch that's code review for the big patch, which I'll squash for the
next version, and is posted here separately just so that it's easy to
see.
My intention as to patches for final commit is:
- 0001 "Make index_concurrently_create_copy more general" same as here.
- 0002 "give options bitmask to table_delete/table_update" same as here,
with a real commit message.
- 0003 Rename cluster.c/h to repack.c/h (similar to 0006 here); no
essential change in contents.
- 0004 "Add CONCURRENTLY option to REPACK command". Squash of 0003,
0004, 0005 and 0007.
- 0005 "Error out any process that would block at REPACK", same as here.
I'm unsure on whether 0009 would be pushed or not.
[1] https://postgr.es/m/202603171606.kf6pmhscqbqz@alvherre.pgsql
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-31 19:41 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-03-31 19:41 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Mar-31, Alvaro Herrera wrote:
> My intention as to patches for final commit is:
>
> - 0001 "Make index_concurrently_create_copy more general" same as here.
> - 0002 "give options bitmask to table_delete/table_update" same as here,
> with a real commit message.
> - 0003 Rename cluster.c/h to repack.c/h (similar to 0006 here); no
> essential change in contents.
> - 0004 "Add CONCURRENTLY option to REPACK command". Squash of 0003,
> 0004, 0005 and 0007.
> - 0005 "Error out any process that would block at REPACK", same as here.
Here it is with that structure. The first three should be pretty much
in final form (0003 needs a commit message), but I still want to make
some more cosmetic adjustments to 0004.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
Thou shalt check the array bounds of all strings (indeed, all arrays), for
surely where thou typest "foo" someone someday shall type
"supercalifragilisticexpialidocious" (5th Commandment for C programmers)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-01 14:55 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-01 14:55 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
i was fuzz testing v48 , and found a crash when REPACK (concurrently)
itself errors out,
1) while running
create table test(id int);
REPACK (concurrently) test;
TBH i didn't knew this, sometimes it's better to not know "rules" ;)
[NOTE: maybe we should add that we can't run
REPACK (concurrently) on table without identity index or primary key into
repack.sgml]
ERROR: cannot process relation "test"
2026-04-01 19:06:31.211 IST [2495575] HINT: Relation "test" has no
identity index.
2026-04-01 19:06:31.211 IST [2495575] STATEMENT: repack (concurrently)
test;
TRAP: failed Assert("proc->statusFlags ==
ProcGlobal->statusFlags[proc->pgxactoff]"), File: "procarray.c", Line: 719,
PID: 2495575
postgres: srinath postgres [local]
REPACK(ExceptionalCondition+0x98)[0xaaaaad938d84]
postgres: srinath postgres [local]
REPACK(ProcArrayEndTransaction+0x1f0)[0xaaaaad6c15fc]
postgres: srinath postgres [local] REPACK(+0x210cf0)[0xaaaaad190cf0]
postgres: srinath postgres [local] REPACK(+0x2117e4)[0xaaaaad1917e4]
postgres: srinath postgres [local]
REPACK(AbortCurrentTransaction+0x10)[0xaaaaad191740]
postgres: srinath postgres [local]
REPACK(PostgresMain+0x568)[0xaaaaad7116e4]
postgres: srinath postgres [local] REPACK(+0x786ae0)[0xaaaaad706ae0]
postgres: srinath postgres [local]
REPACK(postmaster_child_launch+0x1f0)[0xaaaaad5d719c]
postgres: srinath postgres [local] REPACK(+0x65ea98)[0xaaaaad5dea98]
postgres: srinath postgres [local] REPACK(+0x65b650)[0xaaaaad5db650]
postgres: srinath postgres [local]
REPACK(PostmasterMain+0x1564)[0xaaaaad5dae1c]
postgres: srinath postgres [local] REPACK(main+0x3dc)[0xaaaaad466348]
/lib/aarch64-linux-gnu/libc.so.6(+0x284c4)[0xffffb40d84c4]
/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0x98)[0xffffb40d8598]
postgres: srinath postgres [local] REPACK(_start+0x30)[0xaaaaad06ddf0]
2026-04-01 19:06:31.800 IST [2494560] LOG: client backend (PID 2495575)
was terminated by signal 6: Aborted
2) And when running REPACK (concurrently) on the same table while already a
repack was running
on the same table ,just to verify the deadlock occurs and gets errored out
that
"could not wait for concurrent REPACK" but instead got the same crash.
ERROR: could not wait for concurrent REPACK
2026-04-01 12:55:39.481 IST [2397660] DETAIL: Process 2397660 waits for
REPACK running on process 2397307
2026-04-01 12:55:39.481 IST [2397660] CONTEXT: waiting for
ShareUpdateExclusiveLock on relation 16385 of database 5
2026-04-01 12:55:39.481 IST [2397660] STATEMENT: repack (concurrently)
stress_victim ;
2026-04-01 12:55:39.497 IST [2397151] LOG: checkpoint complete: time:
wrote 2056 buffers (12.5%), wrote 0 SLRU buffers; 0 WAL file(s) added, 0
removed, 0 recycled; write=206.804 s, sync=0.003 s, total=861.616 s; sync
files=17, longest=0.002 s, average=0.001 s; distance=318978 kB,
estimate=515341 kB; lsn=2/02810A60, redo lsn=2/02810910
TRAP: failed Assert("proc->statusFlags ==
ProcGlobal->statusFlags[proc->pgxactoff]"), File: "procarray.c", Line: 719,
PID: 2397660
postgres: srinath postgres [local]
REPACK(ExceptionalCondition+0x98)[0xaaaae7d58d84]
postgres: srinath postgres [local]
REPACK(ProcArrayEndTransaction+0x1f0)[0xaaaae7ae15fc]
postgres: srinath postgres [local] REPACK(+0x210cf0)[0xaaaae75b0cf0]
postgres: srinath postgres [local] REPACK(+0x2117e4)[0xaaaae75b17e4]
postgres: srinath postgres [local]
REPACK(AbortCurrentTransaction+0x10)[0xaaaae75b1740]
postgres: srinath postgres [local]
REPACK(PostgresMain+0x568)[0xaaaae7b316e4]
postgres: srinath postgres [local] REPACK(+0x786ae0)[0xaaaae7b26ae0]
postgres: srinath postgres [local]
REPACK(postmaster_child_launch+0x1f0)[0xaaaae79f719c]
postgres: srinath postgres [local] REPACK(+0x65ea98)[0xaaaae79fea98]
postgres: srinath postgres [local] REPACK(+0x65b650)[0xaaaae79fb650]
postgres: srinath postgres [local]
REPACK(PostmasterMain+0x1564)[0xaaaae79fae1c]
postgres: srinath postgres [local] REPACK(main+0x3dc)[0xaaaae7886348]
/lib/aarch64-linux-gnu/libc.so.6(+0x284c4)[0xffff9ec984c4]
/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0x98)[0xffff9ec98598]
postgres: srinath postgres [local] REPACK(_start+0x30)[0xaaaae748ddf0]
2026-04-01 12:58:18.198 IST [2397147] LOG: client backend (PID 2397660)
was terminated by signal 6: Aborted
the reason for this crash was ProcGlobal->statusFlags was not initialized
during the start of ExecRepack , earlier Abort before reaching the proper
initialization of ProcGlobal->statusFlags which was done in rebuild_relation
caused this assert failure in ProcArrayEndTransaction.
Here's the diff to solve this crash.
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 29ba49744eb..d44092a407a 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -284,7 +284,23 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool
isTopLevel)
* that others can conflict with.
*/
if ((params.options & CLUOPT_CONCURRENT) != 0)
+ {
+ /*
+ * Do not let other backends wait for our completion during their
+ * setup of logical replication. Unlike logical replication publisher,
+ * we will have XID assigned, so the other backends - whether
+ * walsenders involved in logical replication or regular backends
+ * executing also REPACK (CONCURRENTLY) - would have to wait for our
+ * completion before they can build their initial snapshot. It is o.k.
+ * for any decoding backend to ignore us because we do not change
+ * tuple descriptor of any table, and the data changes we write should
+ * not be decoded by other backends.
+ */
+ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
MyProc->statusFlags |= PROC_IN_CONCURRENT_REPACK;
+ ProcGlobal->statusFlags[MyProc->pgxactoff] = MyProc->statusFlags;
+ LWLockRelease(ProcArrayLock);
+ }
/*
* If a single relation is specified, process it and we're done ... unless
@@ -988,22 +1004,6 @@ rebuild_relation(Relation OldHeap, Relation index,
bool verbose,
if (concurrent)
{
- /*
- * Do not let other backends wait for our completion during their
- * setup of logical replication. Unlike logical replication publisher,
- * we will have XID assigned, so the other backends - whether
- * walsenders involved in logical replication or regular backends
- * executing also REPACK (CONCURRENTLY) - would have to wait for our
- * completion before they can build their initial snapshot. It is o.k.
- * for any decoding backend to ignore us because we do not change
- * tuple descriptor of any table, and the data changes we write should
- * not be decoded by other backends.
- */
- LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
- MyProc->statusFlags |= PROC_IN_CONCURRENT_REPACK;
- ProcGlobal->statusFlags[MyProc->pgxactoff] = MyProc->statusFlags;
- LWLockRelease(ProcArrayLock);
-
/*
* The worker needs to be member of the locking group we're the leader
* of. We ought to become the leader before the worker starts. The
Thoughts?
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-04-01 17:06 ` Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-02 00:19 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-01 17:06 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> i was fuzz testing v48 , and found a crash when REPACK (concurrently) itself errors out,
> 1) while running
>
> create table test(id int);
> REPACK (concurrently) test;
>
> TBH i didn't knew this, sometimes it's better to not know "rules" ;)
> [NOTE: maybe we should add that we can't run
> REPACK (concurrently) on table without identity index or primary key into repack.sgml]
>
> ERROR: cannot process relation "test"
> 2026-04-01 19:06:31.211 IST [2495575] HINT: Relation "test" has no identity index.
> 2026-04-01 19:06:31.211 IST [2495575] STATEMENT: repack (concurrently) test;
> TRAP: failed Assert("proc->statusFlags == ProcGlobal->statusFlags[proc->pgxactoff]"), File: "procarray.c", Line: 719, PID: 2495575
> Here's the diff to solve this crash.
Thanks. Attached here is v48-0006 fixed.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-01 21:52 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-01 21:52 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-01, Antonin Houska wrote:
> Thanks. Attached here is v48-0006 fixed.
Ah thanks, I incorporated this in the series and rebased, and here's
also a quick and simple additional patch that adds a GUC
max_repack_replication_slots which are especially there to support
REPACK.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-03 12:08 ` Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-03 12:08 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-01, Antonin Houska wrote:
>
> > Thanks. Attached here is v48-0006 fixed.
>
> Ah thanks, I incorporated this in the series and rebased, and here's
> also a quick and simple additional patch that adds a GUC
> max_repack_replication_slots which are especially there to support
> REPACK.
This is an alternative implementation of 0006, allowing one backend running
REPACK (CONCURRENTLY) in a database, instead of one backend in a cluster.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-03 14:59 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-03 14:59 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-03, Antonin Houska wrote:
> This is an alternative implementation of 0006, allowing one backend running
> REPACK (CONCURRENTLY) in a database, instead of one backend in a cluster.
Thanks! so I'm removing the previous one and taking this one. Here's a
v50:
- In testing, I noticed that we could sometimes request a Flush for a
WAL position that hasn't been written yet. This is due to my
replacing the original code that wrote a dummy xlog record that we
could flush, with a call to XLogGetInsertRecPtr(). So we'd get an
error like
LOG: request to flush past end of generated WAL; request 0/15CF0018, current position 0/15CF000
Antonin promptly noticed that this is because XLogGetInsertRecPtr()
gets the LSN past the segment header, which is 18 bytes wrong. So the
fix here is to use XLogGetInsertEndRecPtr() instead.
- My testing also uncovered a problem with exclusion constraints; tables
with them would fail to repack like
ERROR: exclusion constraint record missing for rel temporal_fk_mltrng2mltrng_pk_repacknew
Antonin sent a patch to create copies of the constraints on the
transient index, which seems like it fixes the problem. Those
constraints are obviously discarded together with the transient index.
- I polished the patch to reserve replication slots for REPACK. Given
the new implementation of 0006 that was submitted implies that we can
now run multiple repacks concurrently, I changed the default of 1 to 5.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Here's a general engineering tip: if the non-fun part is too complex for you
to figure out, that might indicate the fun part is too ambitious." (John Naylor)
https://postgr.es/m/CAFBsxsG4OWHBbSDM%3DsSeXrQGOtkPiOEOuME4yD7Ce41NtaAD9g%40mail.gmail.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-03 19:31 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 13:16 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-03 19:31 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-03, Alvaro Herrera wrote:
> - I polished the patch to reserve replication slots for REPACK. Given
> the new implementation of 0006 that was submitted implies that we can
> now run multiple repacks concurrently, I changed the default of 1 to 5.
Srinath let me know that this new part was causing CI failures on
Windows. This version v51 should be okay (or, at least, it passes for
me on CI). Additional changes worth mentioning:
- I think it's nicer for the index_create() API to get a bit to indicate
suppression of progress reporting; so existing callers don't need to
do anything. I guess this is mostly a matter of taste.
- I incorporated Srinath's fix for the PreventInTransactionBlock block.
- When CheckSlotRequirements() is to complain about
"max_replication_slots or max_repack_replication_slots", it seems
actually nicer to say exactly which one of these is the cause of the
problem. This is easy to change; patch 0010 does it; it requires
passing down a "repack" flag all the way from
CheckLogicalDecodingRequirements() and it needs to add an argument to
CreateInitDecodingContext(), which is perhaps not so great. On the
whole I'm inclined to do it anyway, but I'm about equally happy to
leave it alone. This is what would change:
original:
ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("replication slots can only be used if \"%s\" > 0 or \"%s\" > 0",
- "max_replication_slots", "max_repack_replication_slots")));
patched:
ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("replication slots can only be used if \"%s\" > 0",
+ repack ? "max_repack_replication_slots" : "max_replication_slots"));
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"This is a foot just waiting to be shot" (Andrew Dunstan)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-03 19:37 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-03 19:37 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
Sorry -- there's a doc build failure also in CI. Fixed here.
Regards
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Use it up, wear it out, make it do, or do without"
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-04 06:20 ` Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-04-04 06:20 UTC (permalink / raw)
To: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Dear Álvaro,
While testing REPACK CONCURRENTLY command with xid_wraparound module, noticed that
wraparound-autovac worker was terminated with an error like below.
``
ERROR: could not wait for concurrent REPACK
DETAIL: Process 41512 waits for REPACK running on process 41027
CONTEXT: waiting for ShareUpdateExclusiveLock on relation 16384 of database 5
automatic vacuum of table "postgres.public.test"
```
The behavior is different from other commands, like LOCK and maybe normal REPACK.
In these cases the autovac worker would wait till the command fails.
Is it an intentional behavior? If so, what is the advantage that we terminate the
failsafe vacuum?
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
@ 2026-04-04 10:19 ` 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: 'Alvaro Herrera' @ 2026-04-04 10:19 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
On 2026-Apr-04, Hayato Kuroda (Fujitsu) wrote:
> While testing REPACK CONCURRENTLY command with xid_wraparound module, noticed that
> wraparound-autovac worker was terminated with an error like below.
>
> ``
> ERROR: could not wait for concurrent REPACK
> DETAIL: Process 41512 waits for REPACK running on process 41027
> CONTEXT: waiting for ShareUpdateExclusiveLock on relation 16384 of database 5
> automatic vacuum of table "postgres.public.test"
> ```
>
> The behavior is different from other commands, like LOCK and maybe normal REPACK.
> In these cases the autovac worker would wait till the command fails.
>
> Is it an intentional behavior? If so, what is the advantage that we terminate the
> failsafe vacuum?
Hmm, this is intentional; see here:
https://postgr.es/m/202604011050.7ya3k4ccd3hg@alvherre.pgsql
Note that in order for this to happen, the autovacuum must be starting
when the repack is already underway. The theory behind this behavior is
that the autovacuum run is not useful anyway: its purpose is to advance
the freeze xmin/multixact, but the repack is also going to do that.
Once repack is done, autovacuum can assess again whether an emergency
vacuum is needed, and launch a worker in that case.
Do you think it would be better if we allowed autovacuum to continue?
I'm not 100% myself of this new behavior, and it would be very good to
give it some more thought.
I suppose it's unfortunate that autovacuum launcher is going to try
again and again to get workers to process that table, and they are going
to be killed over and over. Maybe it would be better to have autovac
ignore those tables.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Para tener más hay que desear menos"
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
@ 2026-04-06 05:24 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-04-06 05:24 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Sat, Apr 4, 2026 at 3:49 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Apr-04, Hayato Kuroda (Fujitsu) wrote:
>
> > While testing REPACK CONCURRENTLY command with xid_wraparound module, noticed that
> > wraparound-autovac worker was terminated with an error like below.
> >
> > ``
> > ERROR: could not wait for concurrent REPACK
> > DETAIL: Process 41512 waits for REPACK running on process 41027
> > CONTEXT: waiting for ShareUpdateExclusiveLock on relation 16384 of database 5
> > automatic vacuum of table "postgres.public.test"
> > ```
> >
> > The behavior is different from other commands, like LOCK and maybe normal REPACK.
> > In these cases the autovac worker would wait till the command fails.
> >
> > Is it an intentional behavior? If so, what is the advantage that we terminate the
> > failsafe vacuum?
>
> Hmm, this is intentional; see here:
> https://postgr.es/m/202604011050.7ya3k4ccd3hg@alvherre.pgsql
> Note that in order for this to happen, the autovacuum must be starting
> when the repack is already underway. The theory behind this behavior is
> that the autovacuum run is not useful anyway: its purpose is to advance
> the freeze xmin/multixact, but the repack is also going to do that.
> Once repack is done, autovacuum can assess again whether an emergency
> vacuum is needed, and launch a worker in that case.
>
But won't it delay in update of datfrozenxid/datminmxid? For example,
say repack errored out due to some reason, won't it further delay the
update to datfrozenxid/datminmxid which in turn can delay truncate of
clog. Is it possible that after repack errored out the launcher delays
in picking the same table again which further add to such a delay?
> Do you think it would be better if we allowed autovacuum to continue?
>
IIUC, the disadvantage of letting it continue is that if repack is
successful then we have unnecessarily occupied the worker which won't
do anything useful. If we want to not let autovacuum continue then we
should somehow deal with boundary cases which shouldn't lead to delay
in making progress to update frozen xids.
> I'm not 100% myself of this new behavior, and it would be very good to
> give it some more thought.
>
>
> I suppose it's unfortunate that autovacuum launcher is going to try
> again and again to get workers to process that table, and they are going
> to be killed over and over. Maybe it would be better to have autovac
> ignore those tables.
>
I feel that would be better at least when we know that the repack
concurrently command is already in progress. It can help avoid
launching workers again and again, especially when repack concurrently
command is going to take a long time.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-07 21:38 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 07:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-07 21:38 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-06, Amit Kapila wrote:
> On Sat, Apr 4, 2026 at 3:49 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > I suppose it's unfortunate that autovacuum launcher is going to try
> > again and again to get workers to process that table, and they are going
> > to be killed over and over. Maybe it would be better to have autovac
> > ignore those tables.
>
> I feel that would be better at least when we know that the repack
> concurrently command is already in progress. It can help avoid
> launching workers again and again, especially when repack concurrently
> command is going to take a long time.
Okay. I implemented that now, and here it is. 0001 is as before; 0002
creates shared memory for repack and has autovacuum recheck each table
there before processing it.
Having implemented it, I'm not sure the resulting behavior is all that
different from before. I mean, the only difference is that the worker
is going to see the table listed in repack shmem and skip it; as opposed
to trying to lock it and be terminated by the deadlock detector one
second later, at which point it continues with the next table on its
list. So it's some wasted work to set up the autovac work, but nothing
more.
However, there's also the point Andres just made in [1] which basically
kills this idea. So I'm withdrawing this proposal. Still, having
written it, I thought it'd be better to get it archived.
[1] https://postgr.es/m/4n4q3preb3lgyhpzstebhux7b2aojhsw7gik4ivaznyggiezrs@lrznutssxlh2
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
“Cuando no hay humildad las personas se degradan” (A. Christie)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-10 07:00 ` Alexander Lakhin <exclusion@gmail.com>
2026-04-10 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alexander Lakhin @ 2026-04-10 07:00 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello Alvaro,
08.04.2026 00:38, Alvaro Herrera wrote:
> Okay. I implemented that now, and here it is. ...
Could you please look at an assertion failure produced by the following
script, starting from 0d3dba38c:?
createdb db1
createdb db2
echo "
CREATE TABLE t0 (a text);
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
INSERT INTO t0 VALUES ('a');
SELECT pg_sleep(1);
" | psql db1 &
echo "
CREATE TABLE t1 (id int PRIMARY KEY);
CREATE TABLE t2 (id int PRIMARY KEY, a TEXT, FOREIGN KEY (id) REFERENCES t1);
SET min_parallel_table_scan_size = 1;
REPACK (CONCURRENTLY) t2;
" | psql db2
wait
It triggers for me:
TRAP: failed Assert("TransactionIdPrecedesOrEquals(TransactionXmin, RecentXmin)"), File: "procarray.c", Line: 2071, PID:
3529520
postgres: parallel worker for PID 3529517 (ExceptionalCondition+0x69)[0x62f724b19c4c]
postgres: parallel worker for PID 3529517 (+0x522456)[0x62f72498e456]
postgres: parallel worker for PID 3529517 (GetSnapshotData+0x6b)[0x62f72498f50c]
postgres: parallel worker for PID 3529517 (GetNonHistoricCatalogSnapshot+0x4b)[0x62f724b5bd90]
postgres: parallel worker for PID 3529517 (GetCatalogSnapshot+0x20)[0x62f724b5ce7b]
postgres: parallel worker for PID 3529517 (systable_beginscan+0x10b)[0x62f7245c31e7]
postgres: parallel worker for PID 3529517 (+0x69e1e7)[0x62f724b0a1e7]
postgres: parallel worker for PID 3529517 (+0x69e5a6)[0x62f724b0a5a6]
postgres: parallel worker for PID 3529517 (+0x6a4d86)[0x62f724b10d86]
postgres: parallel worker for PID 3529517 (RelationIdGetRelation+0x83)[0x62f724b11208]
postgres: parallel worker for PID 3529517 (relation_open+0x1e)[0x62f72456c235]
...
2026-04-10 05:59:51.471 UTC [3529495] LOG: background worker "parallel worker" (PID 3529520) was terminated by signal
6: Aborted
Best regards,
Alexander
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 07:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
@ 2026-04-10 10:57 ` Antonin Houska <ah@cybertec.at>
2026-04-12 14:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-10 10:57 UTC (permalink / raw)
To: Alexander Lakhin <exclusion@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alexander Lakhin <exclusion@gmail.com> wrote:
> Could you please look at an assertion failure produced by the following
> script, starting from 0d3dba38c:?
> createdb db1
> createdb db2
>
> echo "
> CREATE TABLE t0 (a text);
> BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
> INSERT INTO t0 VALUES ('a');
> SELECT pg_sleep(1);
> " | psql db1 &
>
> echo "
> CREATE TABLE t1 (id int PRIMARY KEY);
> CREATE TABLE t2 (id int PRIMARY KEY, a TEXT, FOREIGN KEY (id) REFERENCES t1);
> SET min_parallel_table_scan_size = 1;
> REPACK (CONCURRENTLY) t2;
> " | psql db2
> wait
>
> It triggers for me:
> TRAP: failed Assert("TransactionIdPrecedesOrEquals(TransactionXmin, RecentXmin)"), File: "procarray.c", Line: 2071, PID: 3529520
Attached is a fix that works for me.
Nevertheless, REPACK (CONCURRENTLY) in your test goes ahead only due to commit
0d3dba38c7, which will probably be reverted [1]. Then REPACK will wait for the
transaction in the other database (db1) to complete before it can actually
start.
Thanks for the report!
[1] https://www.postgresql.org/message-id/cdgw4sbbfcgk6du3iv54r2dgiy4tfywoklbotlmj4irxavdcr3@glxfw5jj277...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 07:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
2026-04-10 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-12 14:00 ` Alexander Lakhin <exclusion@gmail.com>
2026-04-13 09:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alexander Lakhin @ 2026-04-12 14:00 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello Antonin and Alvaro,
10.04.2026 13:57, Antonin Houska wrote:
> Attached is a fix that works for me.
>
> Nevertheless, REPACK (CONCURRENTLY) in your test goes ahead only due to commit
> 0d3dba38c7, which will probably be reverted [1]. Then REPACK will wait for the
> transaction in the other database (db1) to complete before it can actually
> start.
Thank you for the fix!
I've stumbled upon one more issue with this feature:
CREATE TABLE t (i int PRIMARY KEY);
REPACK (CONCURRENTLY) t;
fails for me with sanitizers enabled and
min_dynamic_shared_memory = '1GB'
in postgresql.conf as below:
2026-04-12 13:23:02.000 UTC [2733633] LOG: statement: REPACK (CONCURRENTLY) t;
repack.c:3373:15: runtime error: load of value 240, which is not a valid value for type '_Bool'
#0 0x6441f7eba454 in start_repack_decoding_worker .../src/backend/commands/repack.c:3373
#1 0x6441f7ebdaad in rebuild_relation .../src/backend/commands/repack.c:1010
#2 0x6441f7ebe9a2 in cluster_rel .../src/backend/commands/repack.c:656
#3 0x6441f7ebefea in process_single_relation .../src/backend/commands/repack.c:2359
#4 0x6441f7ebf870 in ExecRepack .../src/backend/commands/repack.c:296
#5 0x6441f886f20e in standard_ProcessUtility .../src/backend/tcop/utility.c:867
...
2026-04-12 13:23:03.620 UTC [2733620] LOG: client backend (PID 2733633) was terminated by signal 6: Aborted
2026-04-12 13:23:03.620 UTC [2733620] DETAIL: Failed process was running: REPACK (CONCURRENTLY) t;
Could you please have a look?
Best regards,
Alexander
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 07:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
2026-04-10 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-12 14:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
@ 2026-04-13 09:42 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-13 09:42 UTC (permalink / raw)
To: Alexander Lakhin <exclusion@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alexander Lakhin <exclusion@gmail.com> wrote:
> I've stumbled upon one more issue with this feature:
> CREATE TABLE t (i int PRIMARY KEY);
> REPACK (CONCURRENTLY) t;
>
> fails for me with sanitizers enabled and
> min_dynamic_shared_memory = '1GB'
> in postgresql.conf as below:
> 2026-04-12 13:23:02.000 UTC [2733633] LOG: statement: REPACK (CONCURRENTLY) t;
> repack.c:3373:15: runtime error: load of value 240, which is not a valid value for type '_Bool'
> #0 0x6441f7eba454 in start_repack_decoding_worker .../src/backend/commands/repack.c:3373
> #1 0x6441f7ebdaad in rebuild_relation .../src/backend/commands/repack.c:1010
> #2 0x6441f7ebe9a2 in cluster_rel .../src/backend/commands/repack.c:656
> #3 0x6441f7ebefea in process_single_relation .../src/backend/commands/repack.c:2359
> #4 0x6441f7ebf870 in ExecRepack .../src/backend/commands/repack.c:296
> #5 0x6441f886f20e in standard_ProcessUtility .../src/backend/tcop/utility.c:867
I could not reproduce the problem, but noticed that the field is not
initialized correctly. Please confirm that 0001 should fix that.
While working on it, I noticed that one field can be removed from
DecodingWorkerShared - 0002 removes that.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-10 10:53 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-21 07:24 ` Re: Adding REPACK [concurrently] Chao Li <li.evan.chao@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-04-10 10:53 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
When testing REPACK concurrently, I noticed that all WALs are retained from
the moment REPACK begins copying data to the new table until the command
finishes replaying concurrent changes on the new table and stops the repack
decoding worker.
I understand the reason: the REPACK command itself starts a long-running
transaction, and logical decoding does not advance restart_lsn beyond the
oldest running transaction's start position. As a result, slot.restart_lsn
remains unchanged, preventing the checkpointer from recycling WALs.
However, since REPACK can run for a long time (hours or even days), I'd like
to confirm whether this is expected behavior or if we plan to improve it
in the future ? And additionally, IIUC, REPACK without using concurrent option
does not have this issue.
Given that we do not restart a REPACK, I think the repack decoding worker
should be able to advance restart_lsn each time after writing changes
(similar to how a physical slot behaves). To illustrate this, I've written
a patch (attached) that implements this approach, and it works fine for me.
BTW, catalog_xmin also won't advance, but that seems not a big issue as
the REPACK transaction itself also holds a snapshot that retains catalog tuples,
so advancing catalog_xmin wouldn't change the situation anyway.
Thoughts ?
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v1-0001-Allow-old-WALs-to-be-removed-during-REPACK-CONCUR.patch (2.3K, ../../TYRPR01MB14195633567DA00ABD42570B794592@TYRPR01MB14195.jpnprd01.prod.outlook.com/2-v1-0001-Allow-old-WALs-to-be-removed-during-REPACK-CONCUR.patch)
download | inline diff:
From 3b4c9a890fdeb854949a36b6be8f1fc00a5edafe Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzj.fnst@fujitsu.com>
Date: Fri, 10 Apr 2026 16:24:55 +0800
Subject: [PATCH v1] Allow old WALs to be removed during REPACK CONCURRENTLY
---
src/backend/commands/repack_worker.c | 14 +++++++++++++-
src/backend/replication/logical/logical.c | 4 +++-
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c
index 5bd020e0184..50f000ca6df 100644
--- a/src/backend/commands/repack_worker.c
+++ b/src/backend/commands/repack_worker.c
@@ -394,12 +394,24 @@ decode_concurrent_changes(LogicalDecodingContext *ctx,
/*
* If WAL segment boundary has been crossed, inform the decoding
- * system that the catalog_xmin can advance.
+ * system that the slot can advance.
+ *
+ * Once REPACK begins copying data to the new table, the logical
+ * decoding machinery prevents the slot from advancing beyond the
+ * oldest running transaction (which is the REPACK transaction
+ * itself). As a result, restart_lsn and catalog_xmin can no longer
+ * advance automatically.
+ *
+ * To allow old WAL files to be recycled, we manually advance the
+ * slot each time a WAL segment boundary is crossed. We do not
+ * advance catalog_xmin here because the REPACK transaction anyway
+ * holds a snapshot that prevents catalog tuple removal.
*/
end_lsn = ctx->reader->EndRecPtr;
XLByteToSeg(end_lsn, segno_new, wal_segment_size);
if (segno_new != repack_current_segment)
{
+ LogicalIncreaseRestartDecodingForSlot(end_lsn, end_lsn);
LogicalConfirmReceivedLocation(end_lsn);
elog(DEBUG1, "REPACK: confirmed receive location %X/%X",
(uint32) (end_lsn >> 32), (uint32) end_lsn);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index d8e02c53558..baa639835f7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1913,8 +1913,10 @@ LogicalConfirmReceivedLocation(XLogRecPtr lsn)
SpinLockRelease(&MyReplicationSlot->mutex);
ReplicationSlotsComputeRequiredXmin(false);
- ReplicationSlotsComputeRequiredLSN();
}
+
+ if (updated_restart)
+ ReplicationSlotsComputeRequiredLSN();
}
else
{
--
2.53.0.windows.2
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
@ 2026-04-10 13:21 ` Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-10 13:21 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com> wrote:
> When testing REPACK concurrently, I noticed that all WALs are retained from
> the moment REPACK begins copying data to the new table until the command
> finishes replaying concurrent changes on the new table and stops the repack
> decoding worker.
>
> I understand the reason: the REPACK command itself starts a long-running
> transaction, and logical decoding does not advance restart_lsn beyond the
> oldest running transaction's start position. As a result, slot.restart_lsn
> remains unchanged, preventing the checkpointer from recycling WALs.
I think you're right, sorry for the omission.
> However, since REPACK can run for a long time (hours or even days), I'd like
> to confirm whether this is expected behavior or if we plan to improve it
> in the future ? And additionally,
Yes, it will be improved. I have a draft patch for it, will rebase and post it
soon. The plan is to:
1) preserve the original xmin/xmax of the tuples when we insert them into the
new heap. Thus, besides achieving MVCC safety, we won't need XID assigned for
most of the time.
2) do catalog changes in separate transactions - XID needed here, but these
transactions take very short time.
3) use a single snapshot only for limited number of tuples/pages. When more
data needs to be copied, a new snapshot is built, supposedly with higher
->xmin than the prevous one.
> IIUC, REPACK without using concurrent option does not have this issue.
It does not have the WAL recycling issue because it does not need to read
WAL. However it also runs in a long transaction. Even though it does not need
XID for the actual heap rewriting, it gets one at the moment it locks the
table using AccessExclusiveLock (which is at the very beginning).
> Given that we do not restart a REPACK, I think the repack decoding worker
> should be able to advance restart_lsn each time after writing changes
> (similar to how a physical slot behaves). To illustrate this, I've written
> a patch (attached) that implements this approach, and it works fine for me.
LGTM, thanks!
> BTW, catalog_xmin also won't advance, but that seems not a big issue as
> the REPACK transaction itself also holds a snapshot that retains catalog tuples,
> so advancing catalog_xmin wouldn't change the situation anyway.
The snapshot "resetting" (mentioned above) should fix this problem too.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-25 06:26 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-05-25 06:26 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
> -----Original Message-----
> From: Antonin Houska <ah@cybertec.at>
> Sent: Friday, April 10, 2026 9:22 PM
> To: Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
> Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila
> <amit.kapila16@gmail.com>; Kuroda, Hayato/黒田 隼人
> <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla
> <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>;
> Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers
> <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
> Subject: Re: Adding REPACK [concurrently]
>
> Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com> wrote:
>
> > When testing REPACK concurrently, I noticed that all WALs are retained
> > from the moment REPACK begins copying data to the new table until the
> > command finishes replaying concurrent changes on the new table and
> > stops the repack decoding worker.
> >
> > I understand the reason: the REPACK command itself starts a
> > long-running transaction, and logical decoding does not advance
> > restart_lsn beyond the oldest running transaction's start position. As
> > a result, slot.restart_lsn remains unchanged, preventing the checkpointer
> from recycling WALs.
>
> I think you're right, sorry for the omission.
>
> > IIUC, REPACK without using concurrent option does not have this issue.
>
> It does not have the WAL recycling issue because it does not need to read
> WAL. However it also runs in a long transaction. Even though it does not need
> XID for the actual heap rewriting, it gets one at the moment it locks the table
> using AccessExclusiveLock (which is at the very beginning).
>
> > Given that we do not restart a REPACK, I think the repack decoding
> > worker should be able to advance restart_lsn each time after writing
> > changes (similar to how a physical slot behaves). To illustrate this,
> > I've written a patch (attached) that implements this approach, and it works
> fine for me.
>
> LGTM, thanks!
>
Thanks for reviewing!
After listening to the REPACK talk at pgconf.dev this year, I understand that
WAL accumulation during REPACK CONCURRENTLY is not intended behavior. I think we
can consider fixing this in the current release. Attached is the rebased
patch, with comments adjusted based on Chao Li's comments.
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v2-0001-Allow-old-WAL-recycling-during-REPACK-CONCURRENTL.patch (3.0K, ../../TY4PR01MB17718B44164522D0798F8E898940A2@TY4PR01MB17718.jpnprd01.prod.outlook.com/2-v2-0001-Allow-old-WAL-recycling-during-REPACK-CONCURRENTL.patch)
download | inline diff:
From 87ba918b9020ce9fa0d4852f222221f521d8701f Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzj.fnst@fujitsu.com>
Date: Fri, 10 Apr 2026 16:24:55 +0800
Subject: [PATCH v2] Allow old WAL recycling during REPACK CONCURRENTLY
During REPACK CONCURRENTLY, logical decoding can keep replication
slot.restart_lsn pinned behind the oldest running transaction, which is often
the long-lived REPACK transaction itself. As a result, old WAL segments are
retained longer than necessary.
This commit advances the replication slot each time WAL insertion crosses a
segment boundary, so obsolete WAL files can be recycled while REPACK is still
running.
This change does not advance catalog_xmin. REPACK already holds a snapshot that
prevents catalog dead tuple removal, so catalog_xmin handling can be addressed
independently.
---
src/backend/commands/repack_worker.c | 14 +++++++++++++-
src/backend/replication/logical/logical.c | 8 +++++++-
2 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c
index b84041372b8..eed69d36508 100644
--- a/src/backend/commands/repack_worker.c
+++ b/src/backend/commands/repack_worker.c
@@ -397,12 +397,24 @@ decode_concurrent_changes(LogicalDecodingContext *ctx,
/*
* If WAL segment boundary has been crossed, inform the decoding
- * system that the catalog_xmin can advance.
+ * system that the slot can advance.
+ *
+ * Once REPACK begins copying data to the new table, the logical
+ * decoding machinery prevents the slot from advancing beyond the
+ * oldest running transaction (which is the REPACK transaction
+ * itself). As a result, restart_lsn and catalog_xmin can no
+ * longer advance automatically.
+ *
+ * To allow old WAL files to be recycled, we manually advance the
+ * slot each time a WAL segment boundary is crossed. We do not
+ * advance catalog_xmin here because the REPACK transaction anyway
+ * holds a snapshot that prevents catalog dead tuple removal.
*/
end_lsn = ctx->reader->EndRecPtr;
XLByteToSeg(end_lsn, segno_new, wal_segment_size);
if (segno_new != repack_current_segment)
{
+ LogicalIncreaseRestartDecodingForSlot(end_lsn, end_lsn);
LogicalConfirmReceivedLocation(end_lsn);
elog(DEBUG1, "REPACK: confirmed receive location %X/%X",
(uint32) (end_lsn >> 32), (uint32) end_lsn);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index a33a685dcc6..11b64de3d90 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1913,8 +1913,14 @@ LogicalConfirmReceivedLocation(XLogRecPtr lsn)
SpinLockRelease(&MyReplicationSlot->mutex);
ReplicationSlotsComputeRequiredXmin(false);
- ReplicationSlotsComputeRequiredLSN();
}
+
+ /*
+ * Now the new restart_lsn is safely on disk, recompute the global WAL
+ * retention requirement.
+ */
+ if (updated_restart)
+ ReplicationSlotsComputeRequiredLSN();
}
else
{
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
@ 2026-05-26 15:31 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-26 15:31 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-May-25, Zhijie Hou (Fujitsu) wrote:
> After listening to the REPACK talk at pgconf.dev this year, I understand that
> WAL accumulation during REPACK CONCURRENTLY is not intended behavior. I think we
> can consider fixing this in the current release. Attached is the rebased
> patch, with comments adjusted based on Chao Li's comments.
You're right, this is a thinko. I'll look at your patch hoping to get
it pushed shortly. I wonder if we should add a TAP test to verify that
WAL files are actually removed? Sounds a bit excessive TBH, but maybe
it isn't really.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"I think my standards have lowered enough that now I think 'good design'
is when the page doesn't irritate the living f*ck out of me." (JWZ)
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-05-27 08:08 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-28 00:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-29 22:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-05-27 08:08 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tuesday, May 26, 2026 11:32 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-May-25, Zhijie Hou (Fujitsu) wrote:
>
> > After listening to the REPACK talk at pgconf.dev this year, I
> > understand that WAL accumulation during REPACK CONCURRENTLY is not
> > intended behavior. I think we can consider fixing this in the current
> > release. Attached is the rebased patch, with comments adjusted based on
> Chao Li's comments.
>
> You're right, this is a thinko. I'll look at your patch hoping to get it pushed
> shortly. I wonder if we should add a TAP test to verify that WAL files are
> actually removed? Sounds a bit excessive TBH, but maybe it isn't really.
I tried a bit, and the test complexity and speed (< 1s) appear to be within
acceptable limits. I'm attaching 0002 as a reference test.
0001 remains unchanged.
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v3-0001-Allow-old-WAL-recycling-during-REPACK-CONCURRENTL.patch (3.0K, ../../TY4PR01MB177189B910069E99897E4323D94082@TY4PR01MB17718.jpnprd01.prod.outlook.com/2-v3-0001-Allow-old-WAL-recycling-during-REPACK-CONCURRENTL.patch)
download | inline diff:
From 698d6e2e475669ffce16e6b4de04b836130e0ed4 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzj.fnst@fujitsu.com>
Date: Fri, 10 Apr 2026 16:24:55 +0800
Subject: [PATCH v3 1/2] Allow old WAL recycling during REPACK CONCURRENTLY
During REPACK CONCURRENTLY, logical decoding can keep replication
slot.restart_lsn pinned behind the oldest running transaction, which is often
the long-lived REPACK transaction itself. As a result, old WAL segments are
retained longer than necessary.
This commit advances the replication slot each time WAL insertion crosses a
segment boundary, so obsolete WAL files can be recycled while REPACK is still
running.
This change does not advance catalog_xmin. REPACK already holds a snapshot that
prevents catalog dead tuple removal, so catalog_xmin handling can be addressed
independently.
---
src/backend/commands/repack_worker.c | 14 +++++++++++++-
src/backend/replication/logical/logical.c | 8 +++++++-
2 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c
index b84041372b8..eed69d36508 100644
--- a/src/backend/commands/repack_worker.c
+++ b/src/backend/commands/repack_worker.c
@@ -397,12 +397,24 @@ decode_concurrent_changes(LogicalDecodingContext *ctx,
/*
* If WAL segment boundary has been crossed, inform the decoding
- * system that the catalog_xmin can advance.
+ * system that the slot can advance.
+ *
+ * Once REPACK begins copying data to the new table, the logical
+ * decoding machinery prevents the slot from advancing beyond the
+ * oldest running transaction (which is the REPACK transaction
+ * itself). As a result, restart_lsn and catalog_xmin can no
+ * longer advance automatically.
+ *
+ * To allow old WAL files to be recycled, we manually advance the
+ * slot each time a WAL segment boundary is crossed. We do not
+ * advance catalog_xmin here because the REPACK transaction anyway
+ * holds a snapshot that prevents catalog dead tuple removal.
*/
end_lsn = ctx->reader->EndRecPtr;
XLByteToSeg(end_lsn, segno_new, wal_segment_size);
if (segno_new != repack_current_segment)
{
+ LogicalIncreaseRestartDecodingForSlot(end_lsn, end_lsn);
LogicalConfirmReceivedLocation(end_lsn);
elog(DEBUG1, "REPACK: confirmed receive location %X/%X",
(uint32) (end_lsn >> 32), (uint32) end_lsn);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index b969caae72e..8b8095bd5d8 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1910,8 +1910,14 @@ LogicalConfirmReceivedLocation(XLogRecPtr lsn)
SpinLockRelease(&MyReplicationSlot->mutex);
ReplicationSlotsComputeRequiredXmin(false);
- ReplicationSlotsComputeRequiredLSN();
}
+
+ /*
+ * Now the new restart_lsn is safely on disk, recompute the global WAL
+ * retention requirement.
+ */
+ if (updated_restart)
+ ReplicationSlotsComputeRequiredLSN();
}
else
{
--
2.43.0
[application/octet-stream] v3-0002-Add-a-test-for-repack-concurrently.patch (3.2K, ../../TY4PR01MB177189B910069E99897E4323D94082@TY4PR01MB17718.jpnprd01.prod.outlook.com/3-v3-0002-Add-a-test-for-repack-concurrently.patch)
download | inline diff:
From d97df7c2ecf815907b7e14322f6f8fde09951b9b Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzj.fnst@fujitsu.com>
Date: Wed, 27 May 2026 15:58:05 +0800
Subject: [PATCH v3 2/2] Add a test for repack concurrently
---
.../recovery/t/046_checkpoint_logical_slot.pl | 74 +++++++++++++++++++
1 file changed, 74 insertions(+)
diff --git a/src/test/recovery/t/046_checkpoint_logical_slot.pl b/src/test/recovery/t/046_checkpoint_logical_slot.pl
index 66761bf56c1..aa32859dd15 100644
--- a/src/test/recovery/t/046_checkpoint_logical_slot.pl
+++ b/src/test/recovery/t/046_checkpoint_logical_slot.pl
@@ -226,4 +226,78 @@ is( $standby->safe_psql(
"t",
'logical slot is not invalidated');
+# Verify that the REPACK slot's restart_lsn can advance while REPACK
+# CONCURRENTLY is still running, allowing WAL files to be recycled during this
+# period.
+
+# Create the table to be repacked and populate it with some data.
+$node->safe_psql(
+ 'postgres',
+ q{
+CREATE TABLE repack_test(i int PRIMARY KEY, t text);
+INSERT INTO repack_test
+SELECT g, md5(g::text)
+FROM generate_series(1, 100) g;
+});
+
+# Pause the REPACK command in the middle of its execution so that the decoding
+# worker continues running, allowing us to test slot restart_lsn advancement
+# later.
+$node->safe_psql('postgres',
+ q(select injection_points_attach('repack-concurrently-before-lock','wait'))
+);
+
+my $repack = $node->background_psql('postgres');
+$repack->query_until(
+ qr/repack_started/,
+ q(
+\echo repack_started
+REPACK (CONCURRENTLY) repack_test;
+\q
+));
+
+# Wait until REPACK reaches the injection point.
+$node->wait_for_event('client backend', 'repack-concurrently-before-lock');
+
+my $restart_lsn_before = $node->safe_psql('postgres',
+ "SELECT restart_lsn FROM pg_replication_slots WHERE slot_name ~ '^repack_[0-9]+' AND slot_type = 'logical' AND temporary;");
+
+# Verify that the replication slot created by the subscription exists and has a
+# valid restart_lsn.
+ok(defined($restart_lsn_before) && $restart_lsn_before ne '',
+ 'REPACK slot has restart_lsn');
+
+my $segment_before = $node->safe_psql('postgres',
+ "SELECT pg_walfile_name('$restart_lsn_before')");
+my $segment_before_path = $node->data_dir . "/pg_wal/$segment_before";
+ok(-f $segment_before_path,
+ "segment for initial restart_lsn exists: $segment_before");
+
+# Switch WAL file on the primary while REPACK is still running and then force
+# WAL removal/recycling with a checkpoint.
+$node->advance_wal(1);
+
+# Wait until the REPACK slot's restart_lsn advances
+ok( $node->poll_query_until(
+ 'postgres', qq[
+ SELECT count(*) > 0
+ FROM pg_replication_slots
+ WHERE slot_name ~ '^repack_[0-9]+'
+ AND slot_type = 'logical'
+ AND temporary
+ AND restart_lsn IS NOT NULL
+ AND restart_lsn <> '$restart_lsn_before'::pg_lsn]),
+ 'REPACK slot restart_lsn advances while command is still running');
+
+$node->safe_psql('postgres', 'CHECKPOINT');
+
+# Test that the old WAL segment was recycled
+ok(!-f $segment_before_path,
+ 'old WAL segment was recycled while REPACK CONCURRENTLY was running');
+
+$node->safe_psql('postgres',
+ "SELECT injection_points_wakeup('repack-concurrently-before-lock')");
+
+$repack->quit;
+
done_testing();
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
@ 2026-05-28 00:31 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-28 03:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-28 00:31 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, May 27, 2026 at 1:08 AM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>
> 0001 remains unchanged.
>
Few minor comments:
=================
*
+ * To allow old WAL files to be recycled, we manually advance the
+ * slot each time a WAL segment boundary is crossed.
This is safe only because REPACK creates a temporary slot that is
dropped if REPACK fails — there's no scenario where this slot needs to
restart decoding from
an earlier position while still alive. I feel that is worth a mention.
*
@@ -1910,8 +1910,14 @@ LogicalConfirmReceivedLocation(XLogRecPtr lsn)
SpinLockRelease(&MyReplicationSlot->mutex);
ReplicationSlotsComputeRequiredXmin(false);
- ReplicationSlotsComputeRequiredLSN();
}
+
+ /*
+ * Now the new restart_lsn is safely on disk, recompute the global WAL
+ * retention requirement.
+ */
+ if (updated_restart)
+ ReplicationSlotsComputeRequiredLSN();
This change is not related to this patch, rather we need it even
without this patch, is it worth mentioning in the commit message?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-28 00:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-28 03:34 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-28 05:18 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-28 03:34 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, May 27, 2026 at 5:31 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, May 27, 2026 at 1:08 AM Zhijie Hou (Fujitsu)
> <houzj.fnst@fujitsu.com> wrote:
> >
> > 0001 remains unchanged.
> >
>
> Few minor comments:
> =================
Commit message says: "This change does not advance catalog_xmin.
REPACK already holds a snapshot that prevents catalog dead tuple
removal, so catalog_xmin handling can be addressed independently.".
Isn't it equally important to advance this, otherwise, for long
running REPACKs dead tuples will be accumulated needlessly? If so, do
we have any ideas to avoid this?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-28 00:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-28 03:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-28 05:18 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-29 15:39 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-05-28 05:18 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Thursday, May 28, 2026 11:34 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Wed, May 27, 2026 at 5:31 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> >
> > On Wed, May 27, 2026 at 1:08 AM Zhijie Hou (Fujitsu)
> > <houzj.fnst@fujitsu.com> wrote:
> > >
> > > 0001 remains unchanged.
> > >
> >
> > Few minor comments:
> > =================
>
> Commit message says: "This change does not advance catalog_xmin.
> REPACK already holds a snapshot that prevents catalog dead tuple removal,
> so catalog_xmin handling can be addressed independently.".
> Isn't it equally important to advance this, otherwise, for long running REPACKs
> dead tuples will be accumulated needlessly? If so, do we have any ideas to
> avoid this?
My understanding is that dead tuple accumulation is common to all long-running
commands (including CLUSTER, VACUUM FULL, and REPACK without CONCURRENTLY). As
long as a command holds a snapshot for a long time while scanning and copying
data, the backend xmin will cause similar accumulation. So, this doesn't seem
like a new issue to me, and given that catalog_xmin only affect tuples in system
catalog which is less harmful, I thought it could be handled independently.
There was a proposal to improve this case in [1]. Sorry if I've missed something.
Attaching the v4 patch which improved the comments and commit message as
suggested.
[1] https://www.postgresql.org/message-id/125085.1775827305%40localhost
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v4-0001-Allow-old-WAL-recycling-during-REPACK-CONCURRENTL.patch (3.6K, ../../TY4PR01MB177181DF3B3DA853AA2298D8D94092@TY4PR01MB17718.jpnprd01.prod.outlook.com/2-v4-0001-Allow-old-WAL-recycling-during-REPACK-CONCURRENTL.patch)
download | inline diff:
From 74881e1bf03da8a4772b3bc5a24542b6dfb51042 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzj.fnst@fujitsu.com>
Date: Fri, 10 Apr 2026 16:24:55 +0800
Subject: [PATCH v4 1/2] Allow old WAL recycling during REPACK CONCURRENTLY
During REPACK CONCURRENTLY, logical decoding can keep replication
slot.restart_lsn pinned behind the oldest running transaction, which is often
the long-lived REPACK transaction itself. As a result, old WAL segments are
retained longer than necessary.
This commit advances the replication slot each time WAL insertion crosses a
segment boundary, so obsolete WAL files can be recycled while REPACK is still
running.
This change does not advance catalog_xmin. REPACK already holds a snapshot that
prevents catalog dead tuple removal, so catalog_xmin handling can be addressed
independently.
Additionally, this commit improves LogicalConfirmReceivedLocation to compute the
oldest restart LSN whenever slot.restart_lsn is updated. Previously, this
function performed the computation only when catalog_xmin was updated, which was
less problematic because catalog_xmin typically advances in most replication
cases, but not for REPACK.
---
src/backend/commands/repack_worker.c | 20 +++++++++++++++++++-
src/backend/replication/logical/logical.c | 8 +++++++-
2 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c
index 4f82eb46bec..56974cbc1f5 100644
--- a/src/backend/commands/repack_worker.c
+++ b/src/backend/commands/repack_worker.c
@@ -397,12 +397,30 @@ decode_concurrent_changes(LogicalDecodingContext *ctx,
/*
* If WAL segment boundary has been crossed, inform the decoding
- * system that the catalog_xmin can advance.
+ * system that the slot can advance.
+ *
+ * Once REPACK begins copying data to the new table, the logical
+ * decoding machinery prevents the slot from advancing beyond the
+ * oldest running transaction (which is the REPACK transaction
+ * itself). As a result, restart_lsn and catalog_xmin can no
+ * longer advance automatically.
+ *
+ * To allow old WAL files to be recycled, we manually advance the
+ * slot each time a WAL segment boundary is crossed. This is safe
+ * because the REPACK slot is temporary and will be dropped
+ * automatically if the REPACK command fails. There is no scenario
+ * where this slot needs to restart decoding from an earlier
+ * position while still alive.
+ *
+ * We do not advance catalog_xmin here because the REPACK
+ * transaction anyway holds a snapshot that prevents catalog dead
+ * tuple removal.
*/
end_lsn = ctx->reader->EndRecPtr;
XLByteToSeg(end_lsn, segno_new, wal_segment_size);
if (segno_new != repack_current_segment)
{
+ LogicalIncreaseRestartDecodingForSlot(end_lsn, end_lsn);
LogicalConfirmReceivedLocation(end_lsn);
elog(DEBUG1, "REPACK: confirmed receive location %X/%X",
(uint32) (end_lsn >> 32), (uint32) end_lsn);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index b969caae72e..8b8095bd5d8 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1910,8 +1910,14 @@ LogicalConfirmReceivedLocation(XLogRecPtr lsn)
SpinLockRelease(&MyReplicationSlot->mutex);
ReplicationSlotsComputeRequiredXmin(false);
- ReplicationSlotsComputeRequiredLSN();
}
+
+ /*
+ * Now the new restart_lsn is safely on disk, recompute the global WAL
+ * retention requirement.
+ */
+ if (updated_restart)
+ ReplicationSlotsComputeRequiredLSN();
}
else
{
--
2.43.0
[application/octet-stream] v4-0002-Add-a-test-for-repack-concurrently.patch (3.2K, ../../TY4PR01MB177181DF3B3DA853AA2298D8D94092@TY4PR01MB17718.jpnprd01.prod.outlook.com/3-v4-0002-Add-a-test-for-repack-concurrently.patch)
download | inline diff:
From 3a82ef6b68280efb1ed008cf436b5a33eed1488f Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzj.fnst@fujitsu.com>
Date: Wed, 27 May 2026 15:58:05 +0800
Subject: [PATCH v4 2/2] Add a test for repack concurrently
---
.../recovery/t/046_checkpoint_logical_slot.pl | 74 +++++++++++++++++++
1 file changed, 74 insertions(+)
diff --git a/src/test/recovery/t/046_checkpoint_logical_slot.pl b/src/test/recovery/t/046_checkpoint_logical_slot.pl
index 66761bf56c1..aa32859dd15 100644
--- a/src/test/recovery/t/046_checkpoint_logical_slot.pl
+++ b/src/test/recovery/t/046_checkpoint_logical_slot.pl
@@ -226,4 +226,78 @@ is( $standby->safe_psql(
"t",
'logical slot is not invalidated');
+# Verify that the REPACK slot's restart_lsn can advance while REPACK
+# CONCURRENTLY is still running, allowing WAL files to be recycled during this
+# period.
+
+# Create the table to be repacked and populate it with some data.
+$node->safe_psql(
+ 'postgres',
+ q{
+CREATE TABLE repack_test(i int PRIMARY KEY, t text);
+INSERT INTO repack_test
+SELECT g, md5(g::text)
+FROM generate_series(1, 100) g;
+});
+
+# Pause the REPACK command in the middle of its execution so that the decoding
+# worker continues running, allowing us to test slot restart_lsn advancement
+# later.
+$node->safe_psql('postgres',
+ q(select injection_points_attach('repack-concurrently-before-lock','wait'))
+);
+
+my $repack = $node->background_psql('postgres');
+$repack->query_until(
+ qr/repack_started/,
+ q(
+\echo repack_started
+REPACK (CONCURRENTLY) repack_test;
+\q
+));
+
+# Wait until REPACK reaches the injection point.
+$node->wait_for_event('client backend', 'repack-concurrently-before-lock');
+
+my $restart_lsn_before = $node->safe_psql('postgres',
+ "SELECT restart_lsn FROM pg_replication_slots WHERE slot_name ~ '^repack_[0-9]+' AND slot_type = 'logical' AND temporary;");
+
+# Verify that the replication slot created by the subscription exists and has a
+# valid restart_lsn.
+ok(defined($restart_lsn_before) && $restart_lsn_before ne '',
+ 'REPACK slot has restart_lsn');
+
+my $segment_before = $node->safe_psql('postgres',
+ "SELECT pg_walfile_name('$restart_lsn_before')");
+my $segment_before_path = $node->data_dir . "/pg_wal/$segment_before";
+ok(-f $segment_before_path,
+ "segment for initial restart_lsn exists: $segment_before");
+
+# Switch WAL file on the primary while REPACK is still running and then force
+# WAL removal/recycling with a checkpoint.
+$node->advance_wal(1);
+
+# Wait until the REPACK slot's restart_lsn advances
+ok( $node->poll_query_until(
+ 'postgres', qq[
+ SELECT count(*) > 0
+ FROM pg_replication_slots
+ WHERE slot_name ~ '^repack_[0-9]+'
+ AND slot_type = 'logical'
+ AND temporary
+ AND restart_lsn IS NOT NULL
+ AND restart_lsn <> '$restart_lsn_before'::pg_lsn]),
+ 'REPACK slot restart_lsn advances while command is still running');
+
+$node->safe_psql('postgres', 'CHECKPOINT');
+
+# Test that the old WAL segment was recycled
+ok(!-f $segment_before_path,
+ 'old WAL segment was recycled while REPACK CONCURRENTLY was running');
+
+$node->safe_psql('postgres',
+ "SELECT injection_points_wakeup('repack-concurrently-before-lock')");
+
+$repack->quit;
+
done_testing();
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-28 00:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-28 03:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-28 05:18 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
@ 2026-05-29 15:39 ` Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Amit Kapila @ 2026-05-29 15:39 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, May 27, 2026 at 10:18 PM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>
> On Thursday, May 28, 2026 11:34 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > On Wed, May 27, 2026 at 5:31 PM Amit Kapila <amit.kapila16@gmail.com>
> > wrote:
> > >
> > > On Wed, May 27, 2026 at 1:08 AM Zhijie Hou (Fujitsu)
> > > <houzj.fnst@fujitsu.com> wrote:
> > > >
> > > > 0001 remains unchanged.
> > > >
> > >
> > > Few minor comments:
> > > =================
> >
> > Commit message says: "This change does not advance catalog_xmin.
> > REPACK already holds a snapshot that prevents catalog dead tuple removal,
> > so catalog_xmin handling can be addressed independently.".
> > Isn't it equally important to advance this, otherwise, for long running REPACKs
> > dead tuples will be accumulated needlessly? If so, do we have any ideas to
> > avoid this?
>
> My understanding is that dead tuple accumulation is common to all long-running
> commands (including CLUSTER, VACUUM FULL, and REPACK without CONCURRENTLY). As
> long as a command holds a snapshot for a long time while scanning and copying
> data, the backend xmin will cause similar accumulation. So, this doesn't seem
> like a new issue to me, and given that catalog_xmin only affect tuples in system
> catalog which is less harmful, I thought it could be handled independently.
> There was a proposal to improve this case in [1].
>
Fair enough. It makes sense to deal with catalog_xmin separately.
> Attaching the v4 patch which improved the comments and commit message as
> suggested.
>
I haven't tested it but otherwise the code changes looks good to me.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
@ 2026-05-29 22:25 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-01 02:25 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-29 22:25 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-May-27, Zhijie Hou (Fujitsu) wrote:
> I tried a bit, and the test complexity and speed (< 1s) appear to be within
> acceptable limits. I'm attaching 0002 as a reference test.
>
> 0001 remains unchanged.
Pushed 0001, in two parts. I rewrote the comment in
decode_concurrent_changes() though, hope it ended up okay.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Los cuentos de hadas no dan al niño su primera idea sobre los monstruos.
Lo que le dan es su primera idea de la posible derrota del monstruo."
(G. K. Chesterton)
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-29 22:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-06-01 02:25 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
0 siblings, 0 replies; 416+ messages in thread
From: Zhijie Hou (Fujitsu) @ 2026-06-01 02:25 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Saturday, May 30, 2026 6:25 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-May-27, Zhijie Hou (Fujitsu) wrote:
>
> > I tried a bit, and the test complexity and speed (< 1s) appear to be
> > within acceptable limits. I'm attaching 0002 as a reference test.
> >
> > 0001 remains unchanged.
>
> Pushed 0001, in two parts. I rewrote the comment in
> decode_concurrent_changes() though, hope it ended up okay.
Thank you for pushing the patches ! The comments look good to me.
In the original 0002 test patch, I realized that I missed to drop existing slots
from previous tests (I did locally but missed to merge into the patch), which
also prevented WAL removal, sorry for that. Attached is the fixed test patch.
Best Regards,
Hou zj
Attachments:
[application/octet-stream] v5-0001-Add-TAP-test-for-WAL-recycling-during-REPACK-CONC.patch (3.7K, ../../TY4PR01MB1771838FAEA8614A388EF138E94152@TY4PR01MB17718.jpnprd01.prod.outlook.com/2-v5-0001-Add-TAP-test-for-WAL-recycling-during-REPACK-CONC.patch)
download | inline diff:
From 92537befabccbbd8e5e16f14f7cf3f25b4562399 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzj.fnst@fujitsu.com>
Date: Wed, 27 May 2026 15:58:05 +0800
Subject: [PATCH v5] Add TAP test for WAL recycling during REPACK CONCURRENTLY
Following 45b0298 which allows old WAL recycling during REPACK CONCURRENTLY,
this commit adds a TAP test to verify the intended behavior and helps prevent
future regressions in WAL retention during long-running REPACK CONCURRENTLY
operations.
---
.../recovery/t/046_checkpoint_logical_slot.pl | 83 +++++++++++++++++++
1 file changed, 83 insertions(+)
diff --git a/src/test/recovery/t/046_checkpoint_logical_slot.pl b/src/test/recovery/t/046_checkpoint_logical_slot.pl
index 66761bf56c1..778bdf42406 100644
--- a/src/test/recovery/t/046_checkpoint_logical_slot.pl
+++ b/src/test/recovery/t/046_checkpoint_logical_slot.pl
@@ -226,4 +226,87 @@ is( $standby->safe_psql(
"t",
'logical slot is not invalidated');
+# Clean up old replication slots that might interfere with later WAL removal
+# tests.
+$standby->stop;
+$primary->safe_psql('postgres',
+ q{SELECT pg_drop_replication_slot('slot_logical');
+ SELECT pg_drop_replication_slot('slot_physical');
+ SELECT pg_drop_replication_slot('failover_slot');
+ SELECT pg_drop_replication_slot('phys_slot');});
+
+# Verify that the REPACK slot's restart_lsn can advance while REPACK
+# CONCURRENTLY is still running, allowing WAL files to be recycled during this
+# period.
+
+# Create the table to be repacked and populate it with some data.
+$node->safe_psql(
+ 'postgres',
+ q{
+CREATE TABLE repack_test(i int PRIMARY KEY, t text);
+INSERT INTO repack_test
+SELECT g, md5(g::text)
+FROM generate_series(1, 100) g;
+});
+
+# Pause the REPACK command in the middle of its execution so that the decoding
+# worker continues running, allowing us to test slot restart_lsn advancement
+# later.
+$node->safe_psql('postgres',
+ q(select injection_points_attach('repack-concurrently-before-lock','wait'))
+);
+
+my $repack = $node->background_psql('postgres');
+$repack->query_until(
+ qr/repack_started/,
+ q(
+\echo repack_started
+REPACK (CONCURRENTLY) repack_test;
+\q
+));
+
+# Wait until REPACK reaches the injection point.
+$node->wait_for_event('client backend', 'repack-concurrently-before-lock');
+
+my $restart_lsn_before = $node->safe_psql('postgres',
+ "SELECT restart_lsn FROM pg_replication_slots WHERE slot_name ~ '^repack_[0-9]+' AND slot_type = 'logical' AND temporary;");
+
+# Verify that the replication slot created by the subscription exists and has a
+# valid restart_lsn.
+ok(defined($restart_lsn_before) && $restart_lsn_before ne '',
+ 'REPACK slot has restart_lsn');
+
+my $segment_before = $node->safe_psql('postgres',
+ "SELECT pg_walfile_name('$restart_lsn_before')");
+my $segment_before_path = $node->data_dir . "/pg_wal/$segment_before";
+ok(-f $segment_before_path,
+ "segment for initial restart_lsn exists: $segment_before");
+
+# Switch WAL file on the primary while REPACK is still running and then force
+# WAL removal/recycling with a checkpoint.
+$node->advance_wal(1);
+
+# Wait until the REPACK slot's restart_lsn advances
+ok( $node->poll_query_until(
+ 'postgres', qq[
+ SELECT count(*) > 0
+ FROM pg_replication_slots
+ WHERE slot_name ~ '^repack_[0-9]+'
+ AND slot_type = 'logical'
+ AND temporary
+ AND restart_lsn IS NOT NULL
+ AND restart_lsn <> '$restart_lsn_before'::pg_lsn]),
+ 'REPACK slot restart_lsn advances while command is still running');
+
+$node->safe_psql('postgres', 'CHECKPOINT');
+
+# Test that the old WAL segment was recycled
+ok(!-f $segment_before_path,
+ 'old WAL segment was recycled while REPACK CONCURRENTLY was running');
+
+$node->safe_psql('postgres',
+ "SELECT injection_points_wakeup('repack-concurrently-before-lock')");
+
+$repack->quit;
+
done_testing();
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
@ 2026-04-21 07:24 ` Chao Li <li.evan.chao@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Chao Li @ 2026-04-21 07:24 UTC (permalink / raw)
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
> On Apr 10, 2026, at 18:53, Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com> wrote:
>
> Hi,
>
> When testing REPACK concurrently, I noticed that all WALs are retained from
> the moment REPACK begins copying data to the new table until the command
> finishes replaying concurrent changes on the new table and stops the repack
> decoding worker.
>
> I understand the reason: the REPACK command itself starts a long-running
> transaction, and logical decoding does not advance restart_lsn beyond the
> oldest running transaction's start position. As a result, slot.restart_lsn
> remains unchanged, preventing the checkpointer from recycling WALs.
>
> However, since REPACK can run for a long time (hours or even days), I'd like
> to confirm whether this is expected behavior or if we plan to improve it
> in the future ? And additionally, IIUC, REPACK without using concurrent option
> does not have this issue.
>
> Given that we do not restart a REPACK, I think the repack decoding worker
> should be able to advance restart_lsn each time after writing changes
> (similar to how a physical slot behaves). To illustrate this, I've written
> a patch (attached) that implements this approach, and it works fine for me.
>
> BTW, catalog_xmin also won't advance, but that seems not a big issue as
> the REPACK transaction itself also holds a snapshot that retains catalog tuples,
> so advancing catalog_xmin wouldn't change the situation anyway.
>
> Thoughts ?
>
> Best Regards,
> Hou zj
> <v1-0001-Allow-old-WALs-to-be-removed-during-REPACK-CONCUR.patch>
I found the same problem with LogicalConfirmReceivedLocation and posted a fix in a separate thread [1]. So I would withdraw my patch.
Looking at this patch, the change is exactly the same as what I did in [1], but I think the code comment should be updated as well. For the comment change, please see my patch in [1].
[1] https://www.postgresql.org/message-id/D8D9F770-DAA2-482C-A7E0-F87E5104C13E%40gmail.com
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-04 13:16 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-04 13:55 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-04 13:16 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi Alvaro,
On Sat, Apr 4, 2026 at 1:01 AM Alvaro Herrera <alvherre@alvh.no-ip.org>
wrote:
> On 2026-Apr-03, Alvaro Herrera wrote:
>
> > - I polished the patch to reserve replication slots for REPACK. Given
> > the new implementation of 0006 that was submitted implies that we can
> > now run multiple repacks concurrently, I changed the default of 1 to 5.
>
> Srinath let me know that this new part was causing CI failures on
> Windows. This version v51 should be okay (or, at least, it passes for
> me on CI).
yes indeed , TotalMaxReplicationSlots was the culprit , as the window
after getting repack worked was forked , i think this was set back to 0.
Thanks for fixing this with "repack" flag , initially i thought why can't we
go with a macro
#define TotalMaxReplicationSlots (max_replication_slots +
max_repack_replication_slots)
but i thought maybe in future we might need this "repack" flag to add
more slot requirements.
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 13:16 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-04-04 13:55 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-04 13:55 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-04, Srinath Reddy Sadipiralla wrote:
> yes indeed , TotalMaxReplicationSlots was the culprit , as the window
> after getting repack worked was forked , i think this was set back to 0.
> Thanks for fixing this with "repack" flag , initially i thought why can't we
> go with a macro
> #define TotalMaxReplicationSlots (max_replication_slots +
> max_repack_replication_slots)
> but i thought maybe in future we might need this "repack" flag to add
> more slot requirements.
Yeah, the other option would have been to add the variable to
restore_backend_variables() so that it's restored after the fork+exec,
but that didn't seem better.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Niemand ist mehr Sklave, als der sich für frei hält, ohne es zu sein."
Nadie está tan esclavizado como el que se cree libre no siéndolo
(Johann Wolfgang von Goethe)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-03 17:24 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-03 17:24 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-03, Antonin Houska wrote:
> diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c
> index 00b21ede481..c25dbeadff3 100644
> --- a/src/backend/commands/repack_worker.c
> +++ b/src/backend/commands/repack_worker.c
> @@ -233,6 +234,13 @@ repack_setup_logical_decoding(Oid relid)
>
> EnsureLogicalDecodingEnabled();
>
> + /*
> + * By declaring that our output plugin does not need shared catalogs, we
> + * avoid waiting for completion of transactions running in other databases
> + * than the one we're connected to.
> + */
> + accessSharedCatalogsInDecoding = false;
> +
> /*
> * Neither prepare_write nor do_write callback nor update_progress is
> * useful for us.
I find this reliance on a global variable for this a bit icky. Would it
work to instead change the CreateInitDecodingContext() signature, so
that instead of "bool need_full_snapshot" it has a three-valued boolean
to distinguish the two cases from the original plus this new one? I
think the value could be stored in LogicalDecodingContext, from where
standby_decode() could obtain it.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"The important things in the world are problems with society that we don't
understand at all. The machines will become more complicated but they won't
be more complicated than the societies that run them." (Freeman Dyson)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-04 08:50 ` Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-04 08:50 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-03, Antonin Houska wrote:
>
> > diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c
> > index 00b21ede481..c25dbeadff3 100644
> > --- a/src/backend/commands/repack_worker.c
> > +++ b/src/backend/commands/repack_worker.c
>
> > @@ -233,6 +234,13 @@ repack_setup_logical_decoding(Oid relid)
> >
> > EnsureLogicalDecodingEnabled();
> >
> > + /*
> > + * By declaring that our output plugin does not need shared catalogs, we
> > + * avoid waiting for completion of transactions running in other databases
> > + * than the one we're connected to.
> > + */
> > + accessSharedCatalogsInDecoding = false;
> > +
> > /*
> > * Neither prepare_write nor do_write callback nor update_progress is
> > * useful for us.
>
> I find this reliance on a global variable for this a bit icky. Would it
> work to instead change the CreateInitDecodingContext() signature, so
> that instead of "bool need_full_snapshot" it has a three-valued boolean
> to distinguish the two cases from the original plus this new one? I
> think the value could be stored in LogicalDecodingContext, from where
> standby_decode() could obtain it.
I agree that the global variable is not handy, but instead of modifying
CreateInitDecodingContext(), how about adding a boolean returning callback to
OutputPluginCallbacks? The point is that whether shared catalogs are needed
during the decoding or not is actually property of the plugin.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-04 09:48 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-04 09:48 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-04, Antonin Houska wrote:
> I agree that the global variable is not handy, but instead of modifying
> CreateInitDecodingContext(), how about adding a boolean returning callback to
> OutputPluginCallbacks? The point is that whether shared catalogs are needed
> during the decoding or not is actually property of the plugin.
Oh, yeah, that sounds good to me.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"After a quick R of TFM, all I can say is HOLY CR** THAT IS COOL! PostgreSQL was
amazing when I first started using it at 7.2, and I'm continually astounded by
learning new features and techniques made available by the continuing work of
the development team."
Berend Tober, http://archives.postgresql.org/pgsql-hackers/2007-08/msg01009.php
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-04 15:29 ` Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-04 15:29 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-04, Antonin Houska wrote:
>
> > I agree that the global variable is not handy, but instead of modifying
> > CreateInitDecodingContext(), how about adding a boolean returning callback to
> > OutputPluginCallbacks? The point is that whether shared catalogs are needed
> > during the decoding or not is actually property of the plugin.
>
> Oh, yeah, that sounds good to me.
This is it. New callback was actually not needed, I just added a new flag to
the OutputPluginOptions structure.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-04 23:53 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-04 23:53 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-04, Antonin Houska wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > On 2026-Apr-04, Antonin Houska wrote:
> >
> > > I agree that the global variable is not handy, but instead of modifying
> > > CreateInitDecodingContext(), how about adding a boolean returning callback to
> > > OutputPluginCallbacks? The point is that whether shared catalogs are needed
> > > during the decoding or not is actually property of the plugin.
> >
> > Oh, yeah, that sounds good to me.
>
> This is it. New callback was actually not needed, I just added a new flag to
> the OutputPluginOptions structure.
Thank you, I removed the previous one and picked up this one (it's 0001
here.) The only potentially troublesome thing I see with it is this change:
/*
* Update range of interesting xids based on the running xacts
* information. We don't increase ->xmax using it, because once we are in
* a consistent state we can do that ourselves and much more efficiently
* so, because we only need to do it for catalog transactions since we
* only ever look at those.
*
* NB: We only increase xmax when a catalog modifying transaction commits
* (see SnapBuildCommitTxn). Because of this, xmax can be lower than
* xmin, which looks odd but is correct and actually more efficient, since
* we hit fast paths in heapam_visibility.c.
+ *
+ * If database specific transaction info was used during startup, the info
+ * for the whole cluster can make xmin go backwards. That would be bad
+ * because we might no longer have older XIDs in ->committed.
*/
- builder->xmin = running->oldestRunningXid;
+ if (NormalTransactionIdFollows(running->oldestRunningXid, builder->xmin))
+ builder->xmin = running->oldestRunningXid;
I can't see any problem with advancing the ->xmin only when it goes
forward, but I wonder if it's possible to introduce any bugs this way.
This bit looks funny though:
/*
* Advance the xmin limit for the current replication slot, to allow
* vacuum to clean up the tuples this slot has been protecting.
*
* The reorderbuffer might have an xmin among the currently running
* snapshots; use it if so. If not, we need only consider the snapshots
* we'll produce later, which can't be less than the oldest running xid in
* the record we're reading now.
*/
xmin = ReorderBufferGetOldestXmin(builder->reorder);
- if (xmin == InvalidTransactionId)
+ /*
+ * Like above, do not let slot xmin go backwards.
+ */
+ if (xmin == InvalidTransactionId && !db_specific)
xmin = running->oldestRunningXid;
I probably need some sleep, but this doesn't make sense to me.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"No me acuerdo, pero no es cierto. No es cierto, y si fuera cierto,
no me acuerdo." (Augusto Pinochet a una corte de justicia)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-05 07:54 ` Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-05 07:54 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-04, Antonin Houska wrote:
>
> > Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > > On 2026-Apr-04, Antonin Houska wrote:
> > >
> > > > I agree that the global variable is not handy, but instead of modifying
> > > > CreateInitDecodingContext(), how about adding a boolean returning callback to
> > > > OutputPluginCallbacks? The point is that whether shared catalogs are needed
> > > > during the decoding or not is actually property of the plugin.
> > >
> > > Oh, yeah, that sounds good to me.
> >
> > This is it. New callback was actually not needed, I just added a new flag to
> > the OutputPluginOptions structure.
>
> Thank you, I removed the previous one and picked up this one (it's 0001
> here.) The only potentially troublesome thing I see with it is this change:
>
> /*
> * Update range of interesting xids based on the running xacts
> * information. We don't increase ->xmax using it, because once we are in
> * a consistent state we can do that ourselves and much more efficiently
> * so, because we only need to do it for catalog transactions since we
> * only ever look at those.
> *
> * NB: We only increase xmax when a catalog modifying transaction commits
> * (see SnapBuildCommitTxn). Because of this, xmax can be lower than
> * xmin, which looks odd but is correct and actually more efficient, since
> * we hit fast paths in heapam_visibility.c.
> + *
> + * If database specific transaction info was used during startup, the info
> + * for the whole cluster can make xmin go backwards. That would be bad
> + * because we might no longer have older XIDs in ->committed.
> */
> - builder->xmin = running->oldestRunningXid;
> + if (NormalTransactionIdFollows(running->oldestRunningXid, builder->xmin))
> + builder->xmin = running->oldestRunningXid;
>
>
> I can't see any problem with advancing the ->xmin only when it goes
> forward, but I wonder if it's possible to introduce any bugs this way.
>
>
> This bit looks funny though:
>
> /*
> * Advance the xmin limit for the current replication slot, to allow
> * vacuum to clean up the tuples this slot has been protecting.
> *
> * The reorderbuffer might have an xmin among the currently running
> * snapshots; use it if so. If not, we need only consider the snapshots
> * we'll produce later, which can't be less than the oldest running xid in
> * the record we're reading now.
> */
> xmin = ReorderBufferGetOldestXmin(builder->reorder);
> - if (xmin == InvalidTransactionId)
> + /*
> + * Like above, do not let slot xmin go backwards.
> + */
> + if (xmin == InvalidTransactionId && !db_specific)
> xmin = running->oldestRunningXid;
>
> I probably need some sleep, but this doesn't make sense to me.
ok, maybe just skip the whole cleanup in that special case.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-05 11:50 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-05 11:50 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-05, Antonin Houska wrote:
> ok, maybe just skip the whole cleanup in that special case.
Hmm, should we make this test only in the db_specific case? Doing it
unconditionally makes me a bit nervous (maybe because I don't fully
understand historic snapshot building).
Anyway I just pushed the addition of a progress-suppression bit to
index_create() interface. Here's the rest of the series.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Ninguna manada de bestias tiene una voz tan horrible como la humana" (Orual)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-05 16:30 ` Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-05 16:30 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-05, Antonin Houska wrote:
>
> > ok, maybe just skip the whole cleanup in that special case.
>
> Hmm, should we make this test only in the db_specific case? Doing it
> unconditionally makes me a bit nervous (maybe because I don't fully
> understand historic snapshot building).
I thought about adding Assert(db_specific) in front of the new return
statement. So what you suggest makes sense to me.
As far as I understand, the xl_running_xacts record is not directly involved
in the snapshot build. Rather, the list of XIDs for snapshots is created and
updated by processing COMMIT and ABORT records.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-05 18:56 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-05 18:56 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
So I've been trying to understand the "Introduce an option to make
logical replication database specific." patch and I have to confess I
just cannot.
As far as I can read, the point is that if we reach
SnapBuildProcessRunningXacts() when db_specific is true (which means
standby_decode is called in an output plugin that has set
need_shared_catalogs to false), _and_ we've not reached consistent state
yet, then we'll call LogStandbySnapshot with our DB oid to emit a new
xl_running_xacts message.
So the WAL-decoding process emits WAL. I don't know if in normal
conditions logical decoding processes emit WAL. If this is exceptional,
I think we should add a comment.
Now, this additional WAL message will be processed by all other
processes decoding WAL. Perhaps it will ignored by most of them. But
most importantly, it will also reach back to ourselves, at which point
we can hopefully use it to see that we might have reached consistent
state within our database. Then we know our snapshot is ready to be
used.
Is this correct?
I think the reason it's safe to skip a lot of the processing caused by
this additional process, is that xl_running_xacts messages are also
emitted in other places in a non-database specific manner. So all the
other placecs that are emitting that message continue to exist and
cause logical-decoders operate in the same way as before.
I think we should sprinkle lots of comments in several places about
this. For example, I propose that standby_redo() should have something
like
* If 'dbid' is valid, only gather transactions running in that database.
+ * Such records should not be the only ones emitted, because this has
+ * potentially dangerous side-effects which makes some places ignore them:
+ *
+ * 1. SnapBuildProcessRunningXacts will skip computing the xmin and restart
+ * point from its input record if the record's xmin is older that the
+ * snapbuilder's current xmin; this should normally be fine because that
+ * information will be updated from other xl_running_xacts records.
+ * 2. standby_redo will likewise skip processing such a record
*
(are there other things that should be mentioned?)
Also, LogStandbySnapshot() should have a comment explaining that passing
a valid dboid is a weird corner case which is to be used with care, and
that functions X Y and Z are going to ignore snapshots carrying a valid
dbid.
Why do we call SnapBuildFindSnapshot() to do this, instead of doing it
directly in SnapBuildProcessRunningXacts? Seems like it would be more
straightforward.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-05 20:41 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-05 20:41 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
So here's a v55 version of the base REPACK patches that I'm feeling
comfortable calling very close to committable. I'm going to give an
additional read tomorrow and maybe make cosmetic adjustments, but there
should be nothing substantial. Of course, the subsequent additions in
the other patches of v54 are still in the cards, and they are most
likely essential.
Changes compared to v54:
- changed reform_tuple() to not deform the tuple if no attributes are
going to be touched. We can simply make a copy instead, which I
suspect is considerably cheaper (but I didn't measure).
- cleaned up worker shmem shutdown callback. I think it's how it is
because it copied parallel worker code, but that has a weird structure
for --as far as I can see-- no good reason (we oughta change it too)
- renamed the worker from pgoutput_repack to pgrepack.
(Note that this is an internal name that users don't face.)
- reverted some unnecessary changes to master
Thanks
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 09:15 ` vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: vignesh C @ 2026-04-06 09:15 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, 6 Apr 2026 at 02:12, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Hi,
>
> So here's a v55 version of the base REPACK patches that I'm feeling
> comfortable calling very close to committable. I'm going to give an
> additional read tomorrow and maybe make cosmetic adjustments, but there
> should be nothing substantial. Of course, the subsequent additions in
> the other patches of v54 are still in the cards, and they are most
> likely essential.
Few comments:
1) Can we add a comment why we should error out here, as repack
concurrently requires this whereas repack does not require this check.
Even if it is required for decoding can't it be handled by replica
identity full:
+ /*
+ * If the identity index is not set due to replica identity being, PK
+ * might exist.
+ */
+ ident_idx = RelationGetReplicaIndex(rel);
+ if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
+ ident_idx = rel->rd_pkindex;
+ if (!OidIsValid(ident_idx))
+ ereport(ERROR,
+
errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot process relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("Relation \"%s\" has no
identity index.",
+ RelationGetRelationName(rel)));
2) Do you think it will be good to add a test to simulate a case where
one of the swap_replation_files is successful and a failure after
that. We can verify that the oid should still point to old oids:
+ /*
+ * Even ShareUpdateExclusiveLock should have prevented others from
+ * creating / dropping indexes (even using the CONCURRENTLY
option), so we
+ * do not need to check whether the lists match.
+ */
+ forboth(lc, ind_oids_old, lc2, ind_oids_new)
+ {
+ Oid ind_old = lfirst_oid(lc);
+ Oid ind_new = lfirst_oid(lc2);
+ Oid mapped_tables[4] = {0};
+
+ swap_relation_files(ind_old, ind_new,
+ (old_table_oid
== RelationRelationId),
+ false, /*
swap_toast_by_content */
+ true,
+ InvalidTransactionId,
+ InvalidMultiXactId,
+ mapped_tables);
3) I'm not sure if this change should be part of this patch:
diff --git a/src/backend/storage/lmgr/generate-lwlocknames.pl
b/src/backend/storage/lmgr/generate-lwlocknames.pl
index b49007167b0..2e7f1054e62 100644
--- a/src/backend/storage/lmgr/generate-lwlocknames.pl
+++ b/src/backend/storage/lmgr/generate-lwlocknames.pl
@@ -162,7 +162,7 @@ while (<$lwlocklist>)
die
"$wait_event_lwlocks[$lwlock_count] defined in wait_event_names.txt but "
- . " missing from lwlocklist.h"
+ . "missing from lwlocklist.h"
if $lwlock_count < scalar @wait_event_lwlocks;
4) Can we add an example for concurrently in documentation
5) Typos
5.a) "jsut" should be "just":
+ * operation to avoid any lock-upgrade hazards. In the concurrent case, we
+ * grab ShareUpdateExclusiveLock (jsut like VACUUM) for most of the
5.b In commit message "intial" should be "initial"
While the "concurrent data" changes are applied at specific stages (we cannot
do that until the intial copy is finished and indexes are built), a background
6) This includes are not required in repack.c, for me it could compile
without it:
+#include "access/detoast.h"
+#include "access/xloginsert.h"
+#include "catalog/pg_control.h"
7) Can you check if the copyright year mentioned for the new files are
correct, as different files mention different years like:
/*-------------------------------------------------------------------------
*
* pgrepack.c
* Logical Replication output plugin for REPACK command
*
* Copyright (c) 2012-2026, PostgreSQL Global Development Group
*
* IDENTIFICATION
* src/backend/replication/pgrepack/pgrepack.c
*
*-------------------------------------------------------------------------
*/
/*-------------------------------------------------------------------------
*
* repack_worker.c
* Implementation of the background worker for ad-hoc logical decoding
* during REPACK (CONCURRENTLY).
*
*
* Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
* Portions Copyright (c) 1994-5, Regents of the University of California
*
*
* IDENTIFICATION
* src/backend/commands/repack_worker.c
Meson.build file:
# Copyright (c) 2022-2026, PostgreSQL Global Development Group
Regards,
Vignesh
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
@ 2026-04-06 09:38 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:01 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 11:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-06 11:56 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 10:20 ` Re: Adding REPACK [concurrently] Tomas Vondra <tomas@vondra.me>
0 siblings, 5 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-06 09:38 UTC (permalink / raw)
To: vignesh C <vignesh21@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-06, vignesh C wrote:
> On Mon, 6 Apr 2026 at 02:12, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> Few comments:
Thanks for reviewing this patch!
> 1) Can we add a comment why we should error out here, as repack
> concurrently requires this whereas repack does not require this check.
> Even if it is required for decoding can't it be handled by replica
> identity full:
> + /*
> + * If the identity index is not set due to replica identity being, PK
> + * might exist.
> + */
> + ident_idx = RelationGetReplicaIndex(rel);
> + if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
> + ident_idx = rel->rd_pkindex;
> + if (!OidIsValid(ident_idx))
> + ereport(ERROR,
> +
> errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> + errmsg("cannot process relation \"%s\"",
> + RelationGetRelationName(rel)),
> + errhint("Relation \"%s\" has no
> identity index.",
> + RelationGetRelationName(rel)));
Ah, I was just rewriting that comment moments ago. I think we could
make it work with replica identity full in theory, but I'm guessing it
would be useless. You really need an index that lets you locate the
affected tuples; otherwise the replay would take forever. If the table
is big, that would make the whole thing unworkable. If the table isn't
big, then you can probably just use straight repack without too much
disruption.
/*
* Obtain the replica identity index -- either one that has been set
* explicitly, or the primary key. If none of these cases apply, the
* table cannot be repacked concurrently. It might be possible to have
* repack work with a FULL replica identity; however that requires more
* work and is not implemented yet.
*/
Now, it's possible to have a non-unique index on some columns (not good
enough to be replica identity) which gives you a list of candidate
tuples, and then the implementation chooses one based on the other
columns which are present in the FULL replica identity. (This seems a
bit dangerous, because there might be multiple matching tuples; and how
do you choose?) Now let's further suppose you can narrow down to a
single tuple; in that case, using FULL would work. However, as I said,
this requires more implementation effort. We could entertain a patch
for this during the pg20 cycle, though I'm doubtful that it would really
be worth your while.
> 2) Do you think it will be good to add a test to simulate a case where
> one of the swap_replation_files is successful and a failure after
> that. We can verify that the oid should still point to old oids:
Hmm, it's not clear to me in which cases this can happen. Are you
thinking that the first swap_replation_files call dies because of
out-of-memory?
Note that the really weird cases, like pg_class or mapped relations, are
directly rejected. So we don't get into the branch with
!RelFileNumberIsValid, and so on.
I mean -- I'm not opposed to adding a test case for it. But I suspect
it's going to be somewhat annoying to write.
> 3) I'm not sure if this change should be part of this patch:
> diff --git a/src/backend/storage/lmgr/generate-lwlocknames.pl
> b/src/backend/storage/lmgr/generate-lwlocknames.pl
> index b49007167b0..2e7f1054e62 100644
> --- a/src/backend/storage/lmgr/generate-lwlocknames.pl
> +++ b/src/backend/storage/lmgr/generate-lwlocknames.pl
> @@ -162,7 +162,7 @@ while (<$lwlocklist>)
>
> die
> "$wait_event_lwlocks[$lwlock_count] defined in wait_event_names.txt but "
> - . " missing from lwlocklist.h"
> + . "missing from lwlocklist.h"
> if $lwlock_count < scalar @wait_event_lwlocks;
Yeah, will remove.
> 4) Can we add an example for concurrently in documentation
> 5) Typos
Sure.
> 6) This includes are not required in repack.c, for me it could compile
> without it:
> +#include "access/detoast.h"
> +#include "access/xloginsert.h"
> +#include "catalog/pg_control.h"
> 7) Can you check if the copyright year mentioned for the new files are
> correct, as different files mention different years like:
I'll look into these, thanks for pointing it out.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Linux transformó mi computadora, de una `máquina para hacer cosas',
en un aparato realmente entretenido, sobre el cual cada día aprendo
algo nuevo" (Jaime Salinas)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 10:01 ` vignesh C <vignesh21@gmail.com>
4 siblings, 0 replies; 416+ messages in thread
From: vignesh C @ 2026-04-06 10:01 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, 6 Apr 2026 at 15:08, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Apr-06, vignesh C wrote:
>
> > On Mon, 6 Apr 2026 at 02:12, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > Few comments:
>
> Thanks for reviewing this patch!
>
> > 1) Can we add a comment why we should error out here, as repack
> > concurrently requires this whereas repack does not require this check.
> > Even if it is required for decoding can't it be handled by replica
> > identity full:
> > + /*
> > + * If the identity index is not set due to replica identity being, PK
> > + * might exist.
> > + */
> > + ident_idx = RelationGetReplicaIndex(rel);
> > + if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
> > + ident_idx = rel->rd_pkindex;
> > + if (!OidIsValid(ident_idx))
> > + ereport(ERROR,
> > +
> > errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> > + errmsg("cannot process relation \"%s\"",
> > + RelationGetRelationName(rel)),
> > + errhint("Relation \"%s\" has no
> > identity index.",
> > + RelationGetRelationName(rel)));
>
> Ah, I was just rewriting that comment moments ago. I think we could
> make it work with replica identity full in theory, but I'm guessing it
> would be useless. You really need an index that lets you locate the
> affected tuples; otherwise the replay would take forever. If the table
> is big, that would make the whole thing unworkable. If the table isn't
> big, then you can probably just use straight repack without too much
> disruption.
>
> /*
> * Obtain the replica identity index -- either one that has been set
> * explicitly, or the primary key. If none of these cases apply, the
> * table cannot be repacked concurrently. It might be possible to have
> * repack work with a FULL replica identity; however that requires more
> * work and is not implemented yet.
> */
Should this be mentioned in XXX comment:
It might be possible to have repack work with a FULL replica identity;
however that requires more work and is not implemented yet.
> > 2) Do you think it will be good to add a test to simulate a case where
> > one of the swap_replation_files is successful and a failure after
> > that. We can verify that the oid should still point to old oids:
>
> Hmm, it's not clear to me in which cases this can happen. Are you
> thinking that the first swap_replation_files call dies because of
> out-of-memory?
Yes, I was thinking of that.
> Note that the really weird cases, like pg_class or mapped relations, are
> directly rejected. So we don't get into the branch with
> !RelFileNumberIsValid, and so on.
>
> I mean -- I'm not opposed to adding a test case for it. But I suspect
> it's going to be somewhat annoying to write.
I will verify this scenario through debugger.
Regards,
Vignesh
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 11:23 ` Antonin Houska <ah@cybertec.at>
4 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-06 11:23 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: vignesh C <vignesh21@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-06, vignesh C wrote:
> > 2) Do you think it will be good to add a test to simulate a case where
> > one of the swap_replation_files is successful and a failure after
> > that. We can verify that the oid should still point to old oids:
>
> Hmm, it's not clear to me in which cases this can happen. Are you
> thinking that the first swap_replation_files call dies because of
> out-of-memory?
>
> Note that the really weird cases, like pg_class or mapped relations, are
> directly rejected. So we don't get into the branch with
> !RelFileNumberIsValid, and so on.
>
> I mean -- I'm not opposed to adding a test case for it. But I suspect
> it's going to be somewhat annoying to write.
After all, I think we'd end up testing whether transaction abort works
correctly.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 11:56 ` Amit Kapila <amit.kapila16@gmail.com>
4 siblings, 0 replies; 416+ messages in thread
From: Amit Kapila @ 2026-04-06 11:56 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, Apr 6, 2026 at 3:08 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Apr-06, vignesh C wrote:
>
> > On Mon, 6 Apr 2026 at 02:12, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > Few comments:
>
> Thanks for reviewing this patch!
>
> > 1) Can we add a comment why we should error out here, as repack
> > concurrently requires this whereas repack does not require this check.
> > Even if it is required for decoding can't it be handled by replica
> > identity full:
> > + /*
> > + * If the identity index is not set due to replica identity being, PK
> > + * might exist.
> > + */
> > + ident_idx = RelationGetReplicaIndex(rel);
> > + if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
> > + ident_idx = rel->rd_pkindex;
> > + if (!OidIsValid(ident_idx))
> > + ereport(ERROR,
> > +
> > errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> > + errmsg("cannot process relation \"%s\"",
> > + RelationGetRelationName(rel)),
> > + errhint("Relation \"%s\" has no
> > identity index.",
> > + RelationGetRelationName(rel)));
>
> Ah, I was just rewriting that comment moments ago. I think we could
> make it work with replica identity full in theory, but I'm guessing it
> would be useless. You really need an index that lets you locate the
> affected tuples; otherwise the replay would take forever. If the table
> is big, that would make the whole thing unworkable. If the table isn't
> big, then you can probably just use straight repack without too much
> disruption.
>
> /*
> * Obtain the replica identity index -- either one that has been set
> * explicitly, or the primary key. If none of these cases apply, the
> * table cannot be repacked concurrently. It might be possible to have
> * repack work with a FULL replica identity; however that requires more
> * work and is not implemented yet.
> */
>
> Now, it's possible to have a non-unique index on some columns (not good
> enough to be replica identity) which gives you a list of candidate
> tuples, and then the implementation chooses one based on the other
> columns which are present in the FULL replica identity. (This seems a
> bit dangerous, because there might be multiple matching tuples; and how
> do you choose?) Now let's further suppose you can narrow down to a
> single tuple; in that case, using FULL would work.
>
I think this is already working for apply worker where we compare
tuples if the index is non_uniuqe, see FindReplTupleInLocalRel.
>
However, as I said,
> this requires more implementation effort. We could entertain a patch
> for this during the pg20 cycle, though I'm doubtful that it would really
> be worth your while.
>
It is fine to support it later in pg20. In general, I wonder if we
have evaluated whether some of the apply-worker infrastructure could
have been reused here?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 21:11 ` Andres Freund <andres@anarazel.de>
2026-04-06 21:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
4 siblings, 2 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-06 21:11 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; Noah Misch <noah@leadboat.com>; +Cc: vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
I just saw this got committed and wanted to briefly play with it. It works
nicely!
Except that at first I tried this in a debugging build, and was briefly rather
dismayed by the performance. It was really slow. But it's not really related
to repack / the patches here.
Turns out that it is to a good part due to
heap_insert()
->CacheInvalidateHeapTuple()
->CacheInvalidateHeapTupleCommon()
->AssertCouldGetRelation()
not being cheap and running a *lot*.
Admittedly it's way worse if you build with -O0, which I tend to do to make
debugging easier.
In that config, the assert single-handled increases the time for a repack by
35% or so.
Noah, is there any reason we need to do the AssertCouldGetRelation() before
the !IsCatalogRelation(relation)? Given that the goal is to make
RelationGetRelid() safe, it doesn't seem there is?
It's totally valid to not have done so initially, this is a quite complicated
feature:
I saw this is using individual heap_insert()s during the
heapam_relation_copy_for_cluster(). Doing individual WAL logged inserts isn't
exactly cheap or efficient from a WAL volume perspective...
Is there anything other than round tuits preventing us from using
multi_insert?
That actually would also reduce the cost in the REPACK decoding worker, due to
having to parse far fewer WAL records.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-06 21:59 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 09:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-06-19 06:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-06 21:59 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-06, Andres Freund wrote:
> I just saw this got committed and wanted to briefly play with it. It works
> nicely!
Yeah, I have to say that Antonin did a great job here.
> Except that at first I tried this in a debugging build, and was briefly rather
> dismayed by the performance. It was really slow. But it's not really related
> to repack / the patches here.
>
> In that config, the assert single-handled increases the time for a repack by
> 35% or so.
Yeah, I saw it was kinda sluggish, but wow, I didn't see *that* much
overhead.
> It's totally valid to not have done so initially, this is a quite complicated
> feature:
>
> I saw this is using individual heap_insert()s during the
> heapam_relation_copy_for_cluster(). Doing individual WAL logged inserts isn't
> exactly cheap or efficient from a WAL volume perspective...
>
> Is there anything other than round tuits preventing us from using
> multi_insert?
>
> That actually would also reduce the cost in the REPACK decoding worker, due to
> having to parse far fewer WAL records.
Nope, not really ... but I don't have any :-(
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-06 21:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 09:39 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 09:39 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-06, Andres Freund wrote:
>
> > I just saw this got committed and wanted to briefly play with it. It works
> > nicely!
>
> Yeah, I have to say that Antonin did a great job here.
Likewise, I appreciate all your improvements! And of course, all the reviews
and testing done by other developers.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-06 21:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-06-19 06:48 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-06-19 06:48 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
This is a diff to remove a comment that was only valid in earlier versions of
the patch - I failed to notice that so far. Another comment in
copy_table_data() explains why rd_toastoid is not set in the CONCURRENTLY
mode:
* This would not work with CONCURRENTLY because we may need to delete
* TOASTed tuples from the new heap. With this hack, we'd delete them
* from the old heap.
*/
NewHeap->rd_toastoid = OldHeap->rd_rel->reltoastrelid;
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] repack_comment.diff (506B, ../../5998.1781851731@localhost/2-repack_comment.diff)
download | inline diff:
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..195d85c323b 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1453,8 +1453,6 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
/*
* Reset rd_toastoid just to be tidy --- it shouldn't be looked at again.
- * In the CONCURRENTLY case, we need to set it again before applying the
- * concurrent changes.
*/
NewHeap->rd_toastoid = InvalidOid;
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-07 01:10 ` Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 416+ messages in thread
From: Noah Misch @ 2026-04-07 01:10 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, Apr 06, 2026 at 05:11:30PM -0400, Andres Freund wrote:
> heap_insert()
> ->CacheInvalidateHeapTuple()
> ->CacheInvalidateHeapTupleCommon()
> ->AssertCouldGetRelation()
> not being cheap and running a *lot*.
>
> Admittedly it's way worse if you build with -O0, which I tend to do to make
> debugging easier.
>
> In that config, the assert single-handled increases the time for a repack by
> 35% or so.
>
>
> Noah, is there any reason we need to do the AssertCouldGetRelation() before
> the !IsCatalogRelation(relation)? Given that the goal is to make
> RelationGetRelid() safe, it doesn't seem there is?
By running AssertCouldGetRelation() during every INSERT statement, this
detects cases that would be unsafe when the target of the INSERT happens to be
a system catalog. Little of our INSERT/UPDATE coverage targets a system
catalog. Hence, the current position is better for detection.
I wonder if this got slower in v19. In v14-v18, the assert's cost is
proportional to the number of held lwlocks, often 0 or 1. In v19, it's
proportional to PrivateRefCountHash cardinality.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
@ 2026-04-07 01:58 ` Andres Freund <andres@anarazel.de>
2026-04-07 02:42 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-07 01:58 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-06 18:10:56 -0700, Noah Misch wrote:
> On Mon, Apr 06, 2026 at 05:11:30PM -0400, Andres Freund wrote:
> > heap_insert()
> > ->CacheInvalidateHeapTuple()
> > ->CacheInvalidateHeapTupleCommon()
> > ->AssertCouldGetRelation()
> > not being cheap and running a *lot*.
> >
> > Admittedly it's way worse if you build with -O0, which I tend to do to make
> > debugging easier.
> >
> > In that config, the assert single-handled increases the time for a repack by
> > 35% or so.
> >
> >
> > Noah, is there any reason we need to do the AssertCouldGetRelation() before
> > the !IsCatalogRelation(relation)? Given that the goal is to make
> > RelationGetRelid() safe, it doesn't seem there is?
>
> By running AssertCouldGetRelation() during every INSERT statement, this
> detects cases that would be unsafe when the target of the INSERT happens to be
> a system catalog.
I see.
> Little of our INSERT/UPDATE coverage targets a system catalog.
Sure. We do have plenty DML doing heap_insert/update however.
> Hence, the current position is better for detection.
What if we returned early in AssertBufferLocksPermitCatalogRead() if
InterruptHoldoffCount == 0? That'd only fail if some code manually did a
RESUME_INTERRUPTS() to balance the one acquired as part of the content lock?
> I wonder if this got slower in v19. In v14-v18, the assert's cost is
> proportional to the number of held lwlocks, often 0 or 1. In v19, it's
> proportional to PrivateRefCountHash cardinality.
Yea, plausible. It will only scan PrivateRefCountHash if
PrivateRefCountOverflowed overflowed, but it did overflow in the case I was
testing...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-07 02:42 ` Noah Misch <noah@leadboat.com>
1 sibling, 0 replies; 416+ messages in thread
From: Noah Misch @ 2026-04-07 02:42 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Mon, Apr 06, 2026 at 09:58:19PM -0400, Andres Freund wrote:
> On 2026-04-06 18:10:56 -0700, Noah Misch wrote:
> > Hence, the current position is better for detection.
>
> What if we returned early in AssertBufferLocksPermitCatalogRead() if
> InterruptHoldoffCount == 0? That'd only fail if some code manually did a
> RESUME_INTERRUPTS() to balance the one acquired as part of the content lock?
Sounds good.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-07 04:44 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:40 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Tom Lane @ 2026-04-07 04:44 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Noah Misch <noah@leadboat.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Maybe you saw this already, but BF member skink is failing on
src/test/modules/injection_points/specs/repack.spec:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2026-04-06%2022%3A50%3A41
regards, tom lane
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-04-07 08:40 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-07 08:40 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-07, Tom Lane wrote:
> Maybe you saw this already, but BF member skink is failing on
> src/test/modules/injection_points/specs/repack.spec:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2026-04-06%2022%3A50%3A41
Thanks, I hadn't, and I cannot reproduce it locally. Anyway, the
valgrind report is
==1617044== VALGRINDERROR-BEGIN
==1617044== Syscall param pwrite64(buf) points to uninitialised byte(s)
==1617044== at 0x6704003: pwrite (pwrite64.c:25)
==1617044== by 0x44AAC72: pg_pwritev (pg_iovec.h:101)
==1617044== by 0x44AC6B5: FileWriteV (fd.c:2280)
==1617044== by 0x44A8EC4: FileWrite (fd.h:245)
==1617044== by 0x44A8EC4: BufFileDumpBuffer (buffile.c:538)
==1617044== by 0x44A9034: BufFileFlush (buffile.c:724)
==1617044== by 0x44A9661: BufFileClose (buffile.c:418)
==1617044== by 0x426C4DE: export_initial_snapshot (repack_worker.c:346)
==1617044== by 0x426CA17: RepackWorkerMain (repack_worker.c:145)
==1617044== by 0x441D3AD: BackgroundWorkerMain (bgworker.c:865)
==1617044== by 0x44219D9: postmaster_child_launch (launch_backend.c:265)
==1617044== Address 0x12d745e2 is 106 bytes inside a block of size 8,272 client-defined
==1617044== at 0x4661CFE: palloc (mcxt.c:1411)
==1617044== by 0x44A8C54: makeBufFileCommon (buffile.c:121)
==1617044== by 0x44A933F: BufFileCreateFileSet (buffile.c:272)
==1617044== by 0x426C4A5: export_initial_snapshot (repack_worker.c:341)
==1617044== by 0x426CA17: RepackWorkerMain (repack_worker.c:145)
==1617044== by 0x441D3AD: BackgroundWorkerMain (bgworker.c:865)
==1617044== by 0x44219D9: postmaster_child_launch (launch_backend.c:265)
==1617044== by 0x4423B6D: StartBackgroundWorker (postmaster.c:4197)
==1617044== by 0x4423DB8: maybe_start_bgworkers (postmaster.c:4362)
==1617044== by 0x4425119: LaunchMissingBackgroundProcesses (postmaster.c:3437)
==1617044== by 0x4426A75: ServerLoop (postmaster.c:1737)
==1617044== by 0x44280DC: PostmasterMain (postmaster.c:1412)
==1617044== Uninitialised value was created by a stack allocation
==1617044== at 0x4674D39: SerializeSnapshot (snapmgr.c:1737)
==1617044==
==1617044== VALGRINDERROR-END
and obviously BufFileCreateFileSet() is being called by existing code
already and it doesn't fail, so I *think* the problem might be that
SerializedSnapshotData has some padding bytes that are being written.
Maybe using palloc0() is enough? I'll try that.
If that doesn't silence skink, I guess my next step is to reproduce
skink's environment more precisely ...
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Uno puede defenderse de los ataques; contra los elogios se esta indefenso"
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-04-07 08:42 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-07 08:42 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi Tom,
On Tue, Apr 7, 2026 at 10:14 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Maybe you saw this already, but BF member skink is failing on
> src/test/modules/injection_points/specs/repack.spec:
>
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2026-04-06%2022%3A50%3A41
>
i looked into this , it seems like valgrind catches the uninitialised
padding bytes, which
repack worker is writing using BufFileWrite, it seems this fix solved the
problem.
diff --git a/src/backend/utils/time/snapmgr.c
b/src/backend/utils/time/snapmgr.c
index 2e6197f5f35..f5682b87626 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1739,6 +1739,8 @@ SerializeSnapshot(Snapshot snapshot, char
*start_address)
Assert(snapshot->subxcnt >= 0);
+ MemSet(&serialized_snapshot, 0, sizeof(SerializedSnapshotData));
+
/* Copy all required fields */
serialized_snapshot.xmin = snapshot->xmin;
serialized_snapshot.xmax = snapshot->xmax;
thoughts?
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-04-07 08:57 ` Antonin Houska <ah@cybertec.at>
2026-04-07 09:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 09:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 11:19 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 3 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 08:57 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> Hi Tom,
>
> On Tue, Apr 7, 2026 at 10:14 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Maybe you saw this already, but BF member skink is failing on
> src/test/modules/injection_points/specs/repack.spec:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2026-04-06%2022%3A50%3A41
>
> i looked into this , it seems like valgrind catches the uninitialised padding bytes, which
> repack worker is writing using BufFileWrite, it seems this fix solved the problem.
>
> diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
> index 2e6197f5f35..f5682b87626 100644
> --- a/src/backend/utils/time/snapmgr.c
> +++ b/src/backend/utils/time/snapmgr.c
> @@ -1739,6 +1739,8 @@ SerializeSnapshot(Snapshot snapshot, char *start_address)
>
> Assert(snapshot->subxcnt >= 0);
>
> + MemSet(&serialized_snapshot, 0, sizeof(SerializedSnapshotData));
> +
> /* Copy all required fields */
> serialized_snapshot.xmin = snapshot->xmin;
> serialized_snapshot.xmax = snapshot->xmax;
>
> thoughts?
Could you reproduce the failure in your environment?
I haven't thought of this explanation because BufFileWrite() only copies the
data to a buffer in the BufFile structure and BufFileDumpBuffer() writes the
buffer. Maybe valgrind is able to track the copying?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-07 09:22 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 10:08 ` Re: Adding REPACK [concurrently] John Naylor <johncnaylorls@gmail.com>
2 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-07 09:22 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-07, Antonin Houska wrote:
> > @@ -1739,6 +1739,8 @@ SerializeSnapshot(Snapshot snapshot, char *start_address)
> >
> > Assert(snapshot->subxcnt >= 0);
> >
> > + MemSet(&serialized_snapshot, 0, sizeof(SerializedSnapshotData));
> > +
> > /* Copy all required fields */
> > serialized_snapshot.xmin = snapshot->xmin;
> > serialized_snapshot.xmax = snapshot->xmax;
> >
> > thoughts?
>
> Could you reproduce the failure in your environment?
Yeah, he did.
> I haven't thought of this explanation because BufFileWrite() only copies the
> data to a buffer in the BufFile structure and BufFileDumpBuffer() writes the
> buffer. Maybe valgrind is able to track the copying?
Yeah, apparently it keeps track of tainted bytes somehow. Clever.
The change to palloc0() that I was proposing did not fix the problem,
because the stack allocated struct overwrote those zeroes with the
uninitialized padding bytes.
I ended up with an equivalent fix to Srinath's -- zero-initializing
the stack-allocated struct, so that the bytes that end up copied by
memcpy() are all defined. Srinath confirmed that in his environment the
valgrind failure goes away, so I think we're good.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"No tengo por qué estar de acuerdo con lo que pienso"
(Carlos Caszeli)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 09:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 09:35 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 09:35 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-07, Antonin Houska wrote:
>
> > I haven't thought of this explanation because BufFileWrite() only copies the
> > data to a buffer in the BufFile structure and BufFileDumpBuffer() writes the
> > buffer. Maybe valgrind is able to track the copying?
>
> Yeah, apparently it keeps track of tainted bytes somehow. Clever.
>
> The change to palloc0() that I was proposing did not fix the problem,
> because the stack allocated struct overwrote those zeroes with the
> uninitialized padding bytes.
>
> I ended up with an equivalent fix to Srinath's -- zero-initializing
> the stack-allocated struct, so that the bytes that end up copied by
> memcpy() are all defined. Srinath confirmed that in his environment the
> valgrind failure goes away, so I think we're good.
Thanks!
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 09:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 10:08 ` John Naylor <johncnaylorls@gmail.com>
2026-04-07 10:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: John Naylor @ 2026-04-07 10:08 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
mamba is failing with
repack.c: In function 'initialize_change_context':
repack.c:2946:7: error: cast from pointer to integer of different size
[-Werror=pointer-to-int-cast]
2946 | (Datum) NULL);
| ^
cc1: all warnings being treated as errors
--
John Naylor
Amazon Web Services
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 09:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 10:08 ` Re: Adding REPACK [concurrently] John Naylor <johncnaylorls@gmail.com>
@ 2026-04-07 10:30 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-07 10:30 UTC (permalink / raw)
To: John Naylor <johncnaylorls@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-Apr-07, John Naylor wrote:
> mamba is failing with
>
> repack.c: In function 'initialize_change_context':
> repack.c:2946:7: error: cast from pointer to integer of different size
> [-Werror=pointer-to-int-cast]
> 2946 | (Datum) NULL);
> | ^
> cc1: all warnings being treated as errors
Hmm, yeah that should have been "(Datum) 0". Pushed.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"The Postgresql hackers have what I call a "NASA space shot" mentality.
Quite refreshing in a world of "weekend drag racer" developers."
(Scott Marlowe)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-07 09:29 ` Antonin Houska <ah@cybertec.at>
2026-04-07 09:41 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 09:29 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
>
> > i looked into this , it seems like valgrind catches the uninitialised padding bytes, which
> > repack worker is writing using BufFileWrite, it seems this fix solved the problem.
> >
> > diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
> > index 2e6197f5f35..f5682b87626 100644
> > --- a/src/backend/utils/time/snapmgr.c
> > +++ b/src/backend/utils/time/snapmgr.c
> > @@ -1739,6 +1739,8 @@ SerializeSnapshot(Snapshot snapshot, char *start_address)
> >
> > Assert(snapshot->subxcnt >= 0);
> >
> > + MemSet(&serialized_snapshot, 0, sizeof(SerializedSnapshotData));
> > +
> > /* Copy all required fields */
> > serialized_snapshot.xmin = snapshot->xmin;
> > serialized_snapshot.xmax = snapshot->xmax;
> >
> > thoughts?
>
> Could you reproduce the failure in your environment?
>
> I haven't thought of this explanation because BufFileWrite() only copies the
> data to a buffer in the BufFile structure and BufFileDumpBuffer() writes the
> buffer. Maybe valgrind is able to track the copying?
Given this message, you may be right:
==1617044== Address 0x12d745e2 is 106 bytes inside a block of size 8,272 client-defined
In my environment, the 'buffer' field starts at offset 80 into the BufFile
structure. We first write 8 bytes into it
BufFileWrite(file, &snap_size, sizeof(snap_size));
followed by the snapshot. Since sizeof(SerializedSnapshotData) is 24, the
offset 106 should be the padding following the 'takenDuringRecovery' field.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 09:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-07 09:41 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-07 09:41 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, Apr 7, 2026 at 2:59 PM Antonin Houska <ah@cybertec.at> wrote:
> Antonin Houska <ah@cybertec.at> wrote:
>
> > Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> >
> > > i looked into this , it seems like valgrind catches the uninitialised
> padding bytes, which
> > > repack worker is writing using BufFileWrite, it seems this fix solved
> the problem.
> > >
> > > diff --git a/src/backend/utils/time/snapmgr.c
> b/src/backend/utils/time/snapmgr.c
> > > index 2e6197f5f35..f5682b87626 100644
> > > --- a/src/backend/utils/time/snapmgr.c
> > > +++ b/src/backend/utils/time/snapmgr.c
> > > @@ -1739,6 +1739,8 @@ SerializeSnapshot(Snapshot snapshot, char
> *start_address)
> > >
> > > Assert(snapshot->subxcnt >= 0);
> > >
> > > + MemSet(&serialized_snapshot, 0, sizeof(SerializedSnapshotData));
> > > +
> > > /* Copy all required fields */
> > > serialized_snapshot.xmin = snapshot->xmin;
> > > serialized_snapshot.xmax = snapshot->xmax;
> > >
> > > thoughts?
> >
> > Could you reproduce the failure in your environment?
> >
> > I haven't thought of this explanation because BufFileWrite() only copies
> the
> > data to a buffer in the BufFile structure and BufFileDumpBuffer() writes
> the
> > buffer. Maybe valgrind is able to track the copying?
>
> Given this message, you may be right:
>
> ==1617044== Address 0x12d745e2 is 106 bytes inside a block of size 8,272
> client-defined
>
> In my environment, the 'buffer' field starts at offset 80 into the BufFile
> structure. We first write 8 bytes into it
>
> BufFileWrite(file, &snap_size, sizeof(snap_size));
>
> followed by the snapshot. Since sizeof(SerializedSnapshotData) is 24, the
> offset 106 should be the padding following the 'takenDuringRecovery' field.
>
yeah , same in my environment aslo
==00:00:00:06.569 3479320== Syscall param pwrite64(buf) points to
uninitialised byte(s)
==00:00:00:06.569 3479320== at 0x55D6D38: pwrite (pwrite64.c:25)
==00:00:00:06.569 3479320== by 0x842EE7: pg_pwritev (pg_iovec.h:101)
==00:00:00:06.569 3479320== by 0x845F67: FileWriteV (fd.c:2280)
==00:00:00:06.569 3479320== by 0x840877: FileWrite (fd.h:245)
==00:00:00:06.569 3479320== by 0x841407: BufFileDumpBuffer
(buffile.c:538)
==00:00:00:06.569 3479320== by 0x841A67: BufFileFlush (buffile.c:724)
==00:00:00:06.569 3479320== by 0x8410B7: BufFileClose (buffile.c:418)
==00:00:00:06.569 3479320== by 0x4BFBBB: export_initial_snapshot
(repack_worker.c:346)
==00:00:00:06.569 3479320== by 0x4BF703: RepackWorkerMain
(repack_worker.c:145)
==00:00:00:06.569 3479320== by 0x765D3F: BackgroundWorkerMain
(bgworker.c:865)
==00:00:00:06.569 3479320== by 0x76C703: postmaster_child_launch
(launch_backend.c:265)
==00:00:00:06.569 3479320== by 0x774F87: StartBackgroundWorker
(postmaster.c:4197)
==00:00:00:06.569 3479320== Address 0x7b055f2 is 106 bytes inside a block
of size 8,272 client-defined
==00:00:00:06.569 3479320== at 0xB234F4: palloc (mcxt.c:1411)
==00:00:00:06.569 3479320== by 0x8408BF: makeBufFileCommon
(buffile.c:121)
==00:00:00:06.569 3479320== by 0x840C2F: BufFileCreateFileSet
(buffile.c:272)
==00:00:00:06.569 3479320== by 0x4BFB7F: export_initial_snapshot
(repack_worker.c:341)
==00:00:00:06.569 3479320== by 0x4BF703: RepackWorkerMain
(repack_worker.c:145)
==00:00:00:06.569 3479320== by 0x765D3F: BackgroundWorkerMain
(bgworker.c:865)
==00:00:00:06.569 3479320== by 0x76C703: postmaster_child_launch
(launch_backend.c:265)
==00:00:00:06.569 3479320== by 0x774F87: StartBackgroundWorker
(postmaster.c:4197)
==00:00:00:06.569 3479320== by 0x775277: maybe_start_bgworkers
(postmaster.c:4362)
==00:00:00:06.569 3479320== by 0x7739F7:
LaunchMissingBackgroundProcesses (postmaster.c:3437)
==00:00:00:06.569 3479320== by 0x770C2B: ServerLoop (postmaster.c:1737)
==00:00:00:06.569 3479320== by 0x77037B: PostmasterMain
(postmaster.c:1412)
==00:00:00:06.569 3479320== Uninitialised value was created by a stack
allocation
==00:00:00:06.569 3479320== at 0xB43008: SerializeSnapshot
(snapmgr.c:1737)
==00:00:00:06.569 3479320==
and you are spot on here with the explanation with offsets
Size snap_size; 8 bytes;
TransactionId xmin; 4 bytes
TransactionId xmax; 4 bytes
uint32 xcnt; 4 bytes
int32 subxcnt; 4 bytes
bool suboverflowed; 1 byte
bool takenDuringRecovery; 1 byte
/* 2 bytes of padding */
CommandId curcid; 4 bytes
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-07 11:19 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-07 11:19 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
There's one more failure in thorntail
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thorntail&dt=2026-04-07%2006%3A31%3A03
because it runs with wal_level=minimal. I think we can make it work by
using a temp-config argument to run the tests, as in the attached.
I didn't actually try to run the buildfarm client though ...
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 11:19 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 12:17 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 12:17 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Andres Freund <andres@anarazel.de>; Noah Misch <noah@leadboat.com>; vignesh C <vignesh21@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> There's one more failure in thorntail
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thorntail&dt=2026-04-07%2006%3A31%3A03
> because it runs with wal_level=minimal. I think we can make it work by
> using a temp-config argument to run the tests, as in the attached.
LGTM. We had almost exactly this in earlier versions of the patch [1], before
commit 67c20979ce.
[1] https://www.postgresql.org/message-id/137668.1768235610%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-08 10:20 ` Tomas Vondra <tomas@vondra.me>
2026-04-08 10:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
4 siblings, 1 reply; 416+ messages in thread
From: Tomas Vondra @ 2026-04-08 10:20 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
while building on a rpi5 with a 32-bit system, I'm getting these warnings:
config.status: linking src/makefiles/Makefile.linux to src/Makefile.port
In file included from ../../../src/include/access/tupmacs.h:20,
from ../../../src/include/access/htup_details.h:20,
from ../../../src/include/access/relscan.h:17,
from ../../../src/include/access/heapam.h:19,
from repack.c:36:
In function ‘VARSIZE_ANY’,
inlined from ‘restore_tuple’ at repack.c:2731:15:
../../../src/include/varatt.h:243:51: warning: array subscript
‘varattrib_4b[0]’ is partly outside array bounds of ‘union
<anonymous>[1]’ [-Warray-bounds=]
243 | ((((const varattrib_4b *) (PTR))->va_4byte.va_header >>
2) & 0x3FFFFFFF)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
../../../src/include/varatt.h:467:24: note: in expansion of macro
‘VARSIZE_4B’
467 | return VARSIZE_4B(PTR);
| ^~~~~~~~~~
repack.c: In function ‘restore_tuple’:
repack.c:2717:49: note: object ‘chunk_header’ of size 4
2717 | } chunk_header;
| ^~~~~~~~~~~~
In function ‘VARSIZE_ANY’,
inlined from ‘restore_tuple’ at repack.c:2734:4:
../../../src/include/varatt.h:243:51: warning: array subscript
‘varattrib_4b[0]’ is partly outside array bounds of ‘union
<anonymous>[1]’ [-Warray-bounds=]
243 | ((((const varattrib_4b *) (PTR))->va_4byte.va_header >>
2) & 0x3FFFFFFF)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
../../../src/include/varatt.h:467:24: note: in expansion of macro
‘VARSIZE_4B’
467 | return VARSIZE_4B(PTR);
| ^~~~~~~~~~
repack.c: In function ‘restore_tuple’:
repack.c:2717:49: note: object ‘chunk_header’ of size 4
2717 | } chunk_header;
| ^~~~~~~~~~~~
I'm not sure if it's just the compiler (gcc 14.2) being pesky, or if
it's an actual issue. The repack tests seem to pass fine.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-08 10:20 ` Re: Adding REPACK [concurrently] Tomas Vondra <tomas@vondra.me>
@ 2026-04-08 10:46 ` Antonin Houska <ah@cybertec.at>
2026-04-08 16:47 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-08 10:46 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Tomas Vondra <tomas@vondra.me> wrote:
> while building on a rpi5 with a 32-bit system, I'm getting these warnings:
>
> config.status: linking src/makefiles/Makefile.linux to src/Makefile.port
> In file included from ../../../src/include/access/tupmacs.h:20,
> from ../../../src/include/access/htup_details.h:20,
> from ../../../src/include/access/relscan.h:17,
> from ../../../src/include/access/heapam.h:19,
> from repack.c:36:
> In function ‘VARSIZE_ANY’,
> inlined from ‘restore_tuple’ at repack.c:2731:15:
> ../../../src/include/varatt.h:243:51: warning: array subscript
> ‘varattrib_4b[0]’ is partly outside array bounds of ‘union
> <anonymous>[1]’ [-Warray-bounds=]
> 243 | ((((const varattrib_4b *) (PTR))->va_4byte.va_header >>
> 2) & 0x3FFFFFFF)
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> ../../../src/include/varatt.h:467:24: note: in expansion of macro
> ‘VARSIZE_4B’
> 467 | return VARSIZE_4B(PTR);
> | ^~~~~~~~~~
> repack.c: In function ‘restore_tuple’:
> repack.c:2717:49: note: object ‘chunk_header’ of size 4
> 2717 | } chunk_header;
> | ^~~~~~~~~~~~
> In function ‘VARSIZE_ANY’,
> inlined from ‘restore_tuple’ at repack.c:2734:4:
> ../../../src/include/varatt.h:243:51: warning: array subscript
> ‘varattrib_4b[0]’ is partly outside array bounds of ‘union
> <anonymous>[1]’ [-Warray-bounds=]
> 243 | ((((const varattrib_4b *) (PTR))->va_4byte.va_header >>
> 2) & 0x3FFFFFFF)
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> ../../../src/include/varatt.h:467:24: note: in expansion of macro
> ‘VARSIZE_4B’
> 467 | return VARSIZE_4B(PTR);
> | ^~~~~~~~~~
> repack.c: In function ‘restore_tuple’:
> repack.c:2717:49: note: object ‘chunk_header’ of size 4
> 2717 | } chunk_header;
> | ^~~~~~~~~~~~
> I'm not sure if it's just the compiler (gcc 14.2) being pesky, or if
> it's an actual issue. The repack tests seem to pass fine.
We already introduced this definition above in the function to suppress this
kind of warning
union
{
alignas(int32) varlena hdr;
char data[sizeof(void *)];
} chunk_header;
The problem on a 32-bit system probably is that sizeof(void *) is 4. We need
some other constant. Maybe (sizeof(varlena) + 1) ...
Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-08 10:20 ` Re: Adding REPACK [concurrently] Tomas Vondra <tomas@vondra.me>
2026-04-08 10:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-08 16:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-04-09 06:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Tom Lane @ 2026-04-08 16:47 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Tomas Vondra <tomas@vondra.me>; Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> writes:
> Tomas Vondra <tomas@vondra.me> wrote:
>> while building on a rpi5 with a 32-bit system, I'm getting these warnings:
>> ...
>> I'm not sure if it's just the compiler (gcc 14.2) being pesky, or if
>> it's an actual issue. The repack tests seem to pass fine.
Several 32-bit BF animals are showing these too (so far: dodo,
grison, and turaco).
> We already introduced this definition above in the function to suppress this
> kind of warning
> union
> {
> alignas(int32) varlena hdr;
> char data[sizeof(void *)];
> } chunk_header;
> The problem on a 32-bit system probably is that sizeof(void *) is 4. We need
> some other constant. Maybe (sizeof(varlena) + 1) ...
This seems unnecessarily Rube Goldberg-ish already. Why not just
uint64 chunk_header;
It will not hurt anything if the variable has more-than-required
alignment. And it'd be better if it were the same size everywhere.
regards, tom lane
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-08 10:20 ` Re: Adding REPACK [concurrently] Tomas Vondra <tomas@vondra.me>
2026-04-08 10:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-08 16:47 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-04-09 06:59 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-09 06:59 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tomas Vondra <tomas@vondra.me>; Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Antonin Houska <ah@cybertec.at> writes:
> > We already introduced this definition above in the function to suppress this
> > kind of warning
>
> > union
> > {
> > alignas(int32) varlena hdr;
> > char data[sizeof(void *)];
> > } chunk_header;
>
> > The problem on a 32-bit system probably is that sizeof(void *) is 4. We need
> > some other constant. Maybe (sizeof(varlena) + 1) ...
>
> This seems unnecessarily Rube Goldberg-ish already.
Indeed.
> Why not just
>
> uint64 chunk_header;
>
> It will not hurt anything if the variable has more-than-required
> alignment. And it'd be better if it were the same size everywhere.
That's certainly better. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 10:14 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-06 10:14 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Sun, Apr 5, 2026 at 10:41 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> So here's a v55 version of the base REPACK patches that I'm feeling
> comfortable calling very close to committable.
Some comments for v55.
repack.c:2725
if (!VARATT_IS_EXTERNAL(varlen))
continue;
I think it should be VARATT_IS_EXTERNAL_INDIRECT - the same as in pgrepack:244.
Also, after
natt_ext--;
I think it worth to add
Assert(natt_ext >= 0);
or Assert(natt_ext == 0); but in another place.
Or exit early with
for (int i = 0; i < desc->natts && natt_ext > 0; i++)
----------------------------
repack.c:2587
table_tuple_insert(rel, slot, GetCurrentCommandId(true),
HEAP_INSERT_NO_LOGICAL, NULL);
More idiomatic to use TABLE_INSERT_NO_LOGICAL instead.
----------------------------
repack.c:2696
ExecForceStoreHeapTuple(tup, slot, false);
AFAIU there is a memory leak here. Memory allocated above (for tuple)
is not freed in any way, because shouldFree == false.
Also, ExecClearTuple (tts_virtual_clear for virtual tuples) requires
TTS_SHOULDFREE to be set to free anything.
-------------------------
grab ShareUpdateExclusiveLock (jsut like VACUUM
typo in "just"
--------------------------
"If the identity index is not set due to replica identity being, PK"
Missing "FULL" after "being"?
-------------------------
Commit message:
"intial copy" -> "initial copy"
"backed performing REPACK" -> "backend performing REPACK"
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-06 10:48 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-06 10:48 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-06, Mihail Nikalayeu wrote:
> repack.c:2725
>
> if (!VARATT_IS_EXTERNAL(varlen))
> continue;
>
> I think it should be VARATT_IS_EXTERNAL_INDIRECT - the same as in pgrepack:244.
Right.
> Also, after
>
> natt_ext--;
>
> I think it worth to add
>
> Assert(natt_ext >= 0);
>
> or Assert(natt_ext == 0); but in another place.
>
> Or exit early with
> for (int i = 0; i < desc->natts && natt_ext > 0; i++)
Hmm, how about something like this?
natt_ext--;
if (natt_ext < 0)
ereport(ERROR,
errcode(ERRCODE_DATA_CORRUPTED),
errmsg("insufficient number of attributes stored separately"));
I'd like to give more details, such as the tuple's identity, but that
seems hard ...
> ----------------------------
> repack.c:2587
>
> table_tuple_insert(rel, slot, GetCurrentCommandId(true),
> HEAP_INSERT_NO_LOGICAL, NULL);
>
> More idiomatic to use TABLE_INSERT_NO_LOGICAL instead.
Ah right.
> ----------------------------
> repack.c:2696
>
> ExecForceStoreHeapTuple(tup, slot, false);
>
> AFAIU there is a memory leak here. Memory allocated above (for tuple)
> is not freed in any way, because shouldFree == false.
> Also, ExecClearTuple (tts_virtual_clear for virtual tuples) requires
> TTS_SHOULDFREE to be set to free anything.
Yeah but I don't want the virtual tuple to be materialized (which would
happen in tts_virtual_materialize if I set shouldFree=true). The memory
should be freed in
ResetPerTupleExprContext(chgcxt->cc_estate);
anyway, right? Maybe deserves a comment.
> -------------------------
> grab ShareUpdateExclusiveLock (jsut like VACUUM
>
> typo in "just"
Right.
> --------------------------
>
> "If the identity index is not set due to replica identity being, PK"
>
> Missing "FULL" after "being"?
Ah yeah, I rewrote this.
> -------------------------
>
> Commit message:
>
> "intial copy" -> "initial copy"
> "backed performing REPACK" -> "backend performing REPACK"
Thanks!
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Sallah, I said NO camels! That's FIVE camels; can't you count?"
(Indiana Jones)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 11:21 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-06 11:21 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi!
On Mon, Apr 6, 2026 at 12:48 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> Hmm, how about something like this?
>
> natt_ext--;
> if (natt_ext < 0)
> ereport(ERROR,
> errcode(ERRCODE_DATA_CORRUPTED),
> errmsg("insufficient number of attributes stored separately"));
I think it is ok.
> Yeah but I don't want the virtual tuple to be materialized (which would
> happen in tts_virtual_materialize if I set shouldFree=true). The memory
> should be freed in
> ResetPerTupleExprContext(chgcxt->cc_estate);
> anyway, right? Maybe deserves a comment.
Not sure, ResetPerTupleExprContext resets just "ExecutorState".
But slots are created in another memory context.
Also, we can't reset slot->tts_mcxt itself - it will free the slot also.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-06 22:22 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 23:53 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:32 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-07 19:43 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 5 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-06 22:22 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
On 2026-Apr-06, Mihail Nikalayeu wrote:
> > Yeah but I don't want the virtual tuple to be materialized (which would
> > happen in tts_virtual_materialize if I set shouldFree=true). The memory
> > should be freed in
> > ResetPerTupleExprContext(chgcxt->cc_estate);
> > anyway, right? Maybe deserves a comment.
>
> Not sure, ResetPerTupleExprContext resets just "ExecutorState".
> But slots are created in another memory context.
>
> Also, we can't reset slot->tts_mcxt itself - it will free the slot also.
So what I ended up doing, is to just not change to the slot's context in
restore_tuple. That was just dumb (mea culpa). So all the memory we
allocate for the slot lives in the executor context, and goes away on
the ResetPerTupleExprContext call at the bottom of the loop. BTW I
don't understand why you say that function only resets ExecutorState --
as far as I can tell, it does this
#define ResetPerTupleExprContext(estate) \
do { \
if ((estate)->es_per_tuple_exprcontext) \
ResetExprContext((estate)->es_per_tuple_exprcontext); \
} while (0)
which in turn does
#define ResetExprContext(econtext) \
MemoryContextReset((econtext)->ecxt_per_tuple_memory)
which AFAICT is exactly what we want.
Anyway, here's the three missing parts. I have not yet edited the
deadlock-checker one to protect autovacuum from processing tables under
repack.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 23:53 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
4 siblings, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-06 23:53 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi!
On Tue, Apr 7, 2026 at 12:22 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> So what I ended up doing, is to just not change to the slot's context in
> restore_tuple.
Yes, it should work.
> BTW I
> don't understand why you say that function only resets ExecutorState
My bad, I confused "ExprContext" with "ExecutorState" (names passed to
AllocSetContextCreate) while tracing.
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 12:15 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
4 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-04-07 12:15 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, Apr 7, 2026 at 3:52 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Anyway, here's the three missing parts. I have not yet edited the
> deadlock-checker one to protect autovacuum from processing tables under
> repack.
>
I have a question based on 0001's commit message: "This patch adds a
new option to logical replication output plugin, to declare that it
does not use shared catalogs (i.e. catalogs that can be changed by
transactions running in other databases in the cluster).". In which
cases, currently plugin needs to access multi-database transactions or
transactions that need to access shared catalogs and on what basis a
plugin can decide that the changes it requires won't need any such
access.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-07 12:33 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-07 12:33 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-07, Amit Kapila wrote:
> I have a question based on 0001's commit message: "This patch adds a
> new option to logical replication output plugin, to declare that it
> does not use shared catalogs (i.e. catalogs that can be changed by
> transactions running in other databases in the cluster).". In which
> cases, currently plugin needs to access multi-database transactions or
> transactions that need to access shared catalogs and on what basis a
> plugin can decide that the changes it requires won't need any such
> access.
I don't think any plugin needs "multi-database" access as such, but
needing access to shared catalogs is likely normal. Repack knows it
won't access any shared catalogs, so it can set the flag at ease.
There's a cross-check added in the commit that tests for access to
shared catalogs if the flag is set to false. I guess you could set it
to false and see what breaks :-)
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 14:10 ` Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Andres Freund @ 2026-04-07 14:10 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-07 14:33:50 +0200, Alvaro Herrera wrote:
> On 2026-Apr-07, Amit Kapila wrote:
>
> > I have a question based on 0001's commit message: "This patch adds a
> > new option to logical replication output plugin, to declare that it
> > does not use shared catalogs (i.e. catalogs that can be changed by
> > transactions running in other databases in the cluster).". In which
> > cases, currently plugin needs to access multi-database transactions or
> > transactions that need to access shared catalogs and on what basis a
> > plugin can decide that the changes it requires won't need any such
> > access.
>
> I don't think any plugin needs "multi-database" access as such, but
> needing access to shared catalogs is likely normal. Repack knows it
> won't access any shared catalogs, so it can set the flag at ease.
>
> There's a cross-check added in the commit that tests for access to
> shared catalogs if the flag is set to false. I guess you could set it
> to false and see what breaks :-)
I think this has a quite high chance of indirect breakages. You just need some
cache invalidation processing / building accessing shared catalogs to violate
the rule, and whether that happens very heavily depends on what cache entries
are present and whether something registers relcache callbacks or such.
This can be triggered by an output function during logical decoding or such,
so you don't really have control over it.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-07 15:38 ` Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 15:38 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Andres Freund <andres@anarazel.de> wrote:
> On 2026-04-07 14:33:50 +0200, Alvaro Herrera wrote:
> > On 2026-Apr-07, Amit Kapila wrote:
> >
> > > I have a question based on 0001's commit message: "This patch adds a
> > > new option to logical replication output plugin, to declare that it
> > > does not use shared catalogs (i.e. catalogs that can be changed by
> > > transactions running in other databases in the cluster).". In which
> > > cases, currently plugin needs to access multi-database transactions or
> > > transactions that need to access shared catalogs and on what basis a
> > > plugin can decide that the changes it requires won't need any such
> > > access.
> >
> > I don't think any plugin needs "multi-database" access as such, but
> > needing access to shared catalogs is likely normal. Repack knows it
> > won't access any shared catalogs, so it can set the flag at ease.
> >
> > There's a cross-check added in the commit that tests for access to
> > shared catalogs if the flag is set to false. I guess you could set it
> > to false and see what breaks :-)
>
> I think this has a quite high chance of indirect breakages. You just need some
> cache invalidation processing / building accessing shared catalogs to violate
> the rule, and whether that happens very heavily depends on what cache entries
> are present and whether something registers relcache callbacks or such.
>
> This can be triggered by an output function during logical decoding or such,
> so you don't really have control over it.
The REPACK plugin only deforms tuples and writes them to a file, so I think
that things like this should not happen. However, I admit that an option that
allows the plugin developer to declare "I don't need shared catalogs" may be
considered deceptive.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-07 15:48 ` Andres Freund <andres@anarazel.de>
2026-04-07 16:01 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-07 15:48 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-07 17:38:24 +0200, Antonin Houska wrote:
> Andres Freund <andres@anarazel.de> wrote:
>
> > On 2026-04-07 14:33:50 +0200, Alvaro Herrera wrote:
> > > On 2026-Apr-07, Amit Kapila wrote:
> > >
> > > > I have a question based on 0001's commit message: "This patch adds a
> > > > new option to logical replication output plugin, to declare that it
> > > > does not use shared catalogs (i.e. catalogs that can be changed by
> > > > transactions running in other databases in the cluster).". In which
> > > > cases, currently plugin needs to access multi-database transactions or
> > > > transactions that need to access shared catalogs and on what basis a
> > > > plugin can decide that the changes it requires won't need any such
> > > > access.
> > >
> > > I don't think any plugin needs "multi-database" access as such, but
> > > needing access to shared catalogs is likely normal. Repack knows it
> > > won't access any shared catalogs, so it can set the flag at ease.
> > >
> > > There's a cross-check added in the commit that tests for access to
> > > shared catalogs if the flag is set to false. I guess you could set it
> > > to false and see what breaks :-)
> >
> > I think this has a quite high chance of indirect breakages. You just need some
> > cache invalidation processing / building accessing shared catalogs to violate
> > the rule, and whether that happens very heavily depends on what cache entries
> > are present and whether something registers relcache callbacks or such.
> >
> > This can be triggered by an output function during logical decoding or such,
> > so you don't really have control over it.
>
> The REPACK plugin only deforms tuples and writes them to a file, so I think
> that things like this should not happen.
You don't need to do it yourself. It just requires a shared_preload_library
extension to register a relcache invalidation callback that accesses shared
catalog.
It's only kind of an accident that we don't have a case today that accesses
shared catcaches during a relcache build in core (I'm not even sure there's
nothing). You'd just need somebody to add e.g. relcache caching for
publications for that to change. Or look up information about a reloption in
pg_parameter_acl. Or lookup tablespace configuration.
> However, I admit that an option that allows the plugin developer to declare
> "I don't need shared catalogs" may be considered deceptive.
At the very least it would need to be a runtime check rather than just an
assert. This would much more likely to be hit in production because otherwise
it's probably hard to hit the case where shared invalidations happen in the
wrong moment. And the consequences are corrupted caches, which could cause
all kinds of havoc.
But I think this may need more infrastructure / deeper analysis than what we
can do right now.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-07 16:01 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 16:01 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Andres Freund <andres@anarazel.de> wrote:
> On 2026-04-07 17:38:24 +0200, Antonin Houska wrote:
> > The REPACK plugin only deforms tuples and writes them to a file, so I think
> > that things like this should not happen.
>
> You don't need to do it yourself. It just requires a shared_preload_library
> extension to register a relcache invalidation callback that accesses shared
> catalog.
>
> It's only kind of an accident that we don't have a case today that accesses
> shared catcaches during a relcache build in core (I'm not even sure there's
> nothing). You'd just need somebody to add e.g. relcache caching for
> publications for that to change. Or look up information about a reloption in
> pg_parameter_acl. Or lookup tablespace configuration.
>
>
> > However, I admit that an option that allows the plugin developer to declare
> > "I don't need shared catalogs" may be considered deceptive.
>
> At the very least it would need to be a runtime check rather than just an
> assert. This would much more likely to be hit in production because otherwise
> it's probably hard to hit the case where shared invalidations happen in the
> wrong moment. And the consequences are corrupted caches, which could cause
> all kinds of havoc.
>
>
> But I think this may need more infrastructure / deeper analysis than what we
> can do right now.
ok, thanks a lot for having looked at it.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-27 04:25 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-27 04:46 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Amit Kapila @ 2026-04-27 04:25 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, Apr 7, 2026 at 9:18 PM Andres Freund <andres@anarazel.de> wrote:
>
> On 2026-04-07 17:38:24 +0200, Antonin Houska wrote:
> > Andres Freund <andres@anarazel.de> wrote:
> >
> > > On 2026-04-07 14:33:50 +0200, Alvaro Herrera wrote:
> > > > On 2026-Apr-07, Amit Kapila wrote:
> > > >
> > > > > I have a question based on 0001's commit message: "This patch adds a
> > > > > new option to logical replication output plugin, to declare that it
> > > > > does not use shared catalogs (i.e. catalogs that can be changed by
> > > > > transactions running in other databases in the cluster).". In which
> > > > > cases, currently plugin needs to access multi-database transactions or
> > > > > transactions that need to access shared catalogs and on what basis a
> > > > > plugin can decide that the changes it requires won't need any such
> > > > > access.
> > > >
> > > > I don't think any plugin needs "multi-database" access as such, but
> > > > needing access to shared catalogs is likely normal. Repack knows it
> > > > won't access any shared catalogs, so it can set the flag at ease.
> > > >
> > > > There's a cross-check added in the commit that tests for access to
> > > > shared catalogs if the flag is set to false. I guess you could set it
> > > > to false and see what breaks :-)
> > >
> > > I think this has a quite high chance of indirect breakages. You just need some
> > > cache invalidation processing / building accessing shared catalogs to violate
> > > the rule, and whether that happens very heavily depends on what cache entries
> > > are present and whether something registers relcache callbacks or such.
> > >
> > > This can be triggered by an output function during logical decoding or such,
> > > so you don't really have control over it.
> >
> > The REPACK plugin only deforms tuples and writes them to a file, so I think
> > that things like this should not happen.
>
> You don't need to do it yourself. It just requires a shared_preload_library
> extension to register a relcache invalidation callback that accesses shared
> catalog.
>
> It's only kind of an accident that we don't have a case today that accesses
> shared catcaches during a relcache build in core (I'm not even sure there's
> nothing). You'd just need somebody to add e.g. relcache caching for
> publications for that to change. Or look up information about a reloption in
> pg_parameter_acl. Or lookup tablespace configuration.
>
>
> > However, I admit that an option that allows the plugin developer to declare
> > "I don't need shared catalogs" may be considered deceptive.
>
> At the very least it would need to be a runtime check rather than just an
> assert. This would much more likely to be hit in production because otherwise
> it's probably hard to hit the case where shared invalidations happen in the
> wrong moment. And the consequences are corrupted caches, which could cause
> all kinds of havoc.
>
>
> But I think this may need more infrastructure / deeper analysis than what we
> can do right now.
>
We still have not decided on this point. The code corresponding to
this part [1] has an open bug as well [2]. Based on the discussion
here, I don't see we have a consensus with the current approach and
even if we want to go with the current approach it seems that we need
more work which needs further analysis. Alvaro, others, what is your
take on this?
[1]: commit 0d3dba38c777384a9dd7dffe924355c9683a6b71: Allow logical
replication snapshots to be database-specific
[2]: https://www.postgresql.org/message-id/CAHg%2BQDcQak4jx_6X2_Ws98rzG%3DxBARLjqm_%3D56wTRUtNsY4DZQ%40ma...
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-27 04:46 ` Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
1 sibling, 0 replies; 416+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-04-27 04:46 UTC (permalink / raw)
To: 'Amit Kapila' <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; +Cc: Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Dear Hackers,
>
> We still have not decided on this point. The code corresponding to
> this part [1] has an open bug as well [2]. Based on the discussion
> here, I don't see we have a consensus with the current approach and
> even if we want to go with the current approach it seems that we need
> more work which needs further analysis. Alvaro, others, what is your
> take on this?
>
FYI, the point I raised [1] [2] seemed not to be fixed yet. Not sure it's the
last point we missed.
[1]: https://www.postgresql.org/message-id/225003.1775571560%40localhost
[2]: https://www.postgresql.org/message-id/CAA4eK1KWDbBk4FgbbWdivQLrPPzR4zgvfnHK4WjWC78rbuRVbg%40mail.gma...
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-30 11:24 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 15:04 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-30 11:24 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Mon, Apr 27, 2026 at 6:25 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> Alvaro, others, what is your take on this?
I agree with you here - we should AT LEAST make that an ERROR instead
of an assert and also check it during cache access (not only during
the scan because of cache misses).
But I think it will still be fragile in case of some extensions installed.
Anyway... We also have an issue with correctness right now.
I took the old stress test from [0] (the first two) and it fails now,
even with the fix from [1] ("Possible premature SNAPBUILD_CONSISTENT
with DB-specific running_xacts").
It looks like [1] fixes 008_repack_concurrently.pl, but
007_repack_concurrently.pl fails anyway, including
pgbench: error: client 1 script 0 aborted in command 10 query 0:
ERROR: could not create unique index "tbl_pkey_repacknew"
# DETAIL: Key (i)=(383) is duplicated.
and
'pgbench: error: pgbench:client 23 script 0 aborted in command 31
query 0: ERROR: division by zero
Last one is not MVCC-related; you can see from the logs that it
performs something like SELECT (509063) / 0 when the table sum
changes.
Setting need_shared_catalogs = true make them pass, so something is
wrong with its correctness.
P.S.
I think it is good idea to add these stress tests to the source tree,
perhaps with some kind PG_TEST_EXTRA=stress (as done in [1]).
[0]: https://www.postgresql.org/message-id/flat/CADzfLwUitd5J17O9FUxNGrZBurOpL6n%2BtnS6dgArXi-i9DNxhg%40m...
[1]: https://www.postgresql.org/message-id/flat/CAHg%2BQDcQak4jx_6X2_Ws98rzG%3DxBARLjqm_%3D56wTRUtNsY4DZQ...
[2]: https://www.postgresql.org/message-id/flat/CADzfLwWC%2BKxYWb-2QotWaz-q1LK8koLNVUR1Q8obAtt%2BR_sORA%4...
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-05-04 13:24 ` Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-05-04 13:24 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> On Mon, Apr 27, 2026 at 6:25 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > Alvaro, others, what is your take on this?
>
> I agree with you here - we should AT LEAST make that an ERROR instead
> of an assert and also check it during cache access (not only during
> the scan because of cache misses).
> But I think it will still be fragile in case of some extensions installed.
>
> Anyway... We also have an issue with correctness right now.
>
> I took the old stress test from [0] (the first two) and it fails now,
> even with the fix from [1] ("Possible premature SNAPBUILD_CONSISTENT
> with DB-specific running_xacts").
>
> It looks like [1] fixes 008_repack_concurrently.pl, but
> 007_repack_concurrently.pl fails anyway, including
>
> pgbench: error: client 1 script 0 aborted in command 10 query 0:
> ERROR: could not create unique index "tbl_pkey_repacknew"
> # DETAIL: Key (i)=(383) is duplicated.
> and
> 'pgbench: error: pgbench:client 23 script 0 aborted in command 31
> query 0: ERROR: division by zero
>
> Last one is not MVCC-related; you can see from the logs that it
> performs something like SELECT (509063) / 0 when the table sum
> changes.
>
> Setting need_shared_catalogs = true make them pass, so something is
> wrong with its correctness.
Thanks for testing again. Whether we keep the "database specific slots" or
not, it'd be good to know what exactly the reason of these errors is. I wonder
if the feature just exposes a problem that remains shadowed otherwise, due to
the contention on replication slot. I'm going to investigate.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-05 12:47 ` Antonin Houska <ah@cybertec.at>
2026-05-05 15:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-05-05 12:47 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
>
> > On Mon, Apr 27, 2026 at 6:25 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > Alvaro, others, what is your take on this?
> >
> > I agree with you here - we should AT LEAST make that an ERROR instead
> > of an assert and also check it during cache access (not only during
> > the scan because of cache misses).
> > But I think it will still be fragile in case of some extensions installed.
> >
> > Anyway... We also have an issue with correctness right now.
> >
> > I took the old stress test from [0] (the first two) and it fails now,
> > even with the fix from [1] ("Possible premature SNAPBUILD_CONSISTENT
> > with DB-specific running_xacts").
> >
> > It looks like [1] fixes 008_repack_concurrently.pl, but
> > 007_repack_concurrently.pl fails anyway, including
> >
> > pgbench: error: client 1 script 0 aborted in command 10 query 0:
> > ERROR: could not create unique index "tbl_pkey_repacknew"
> > # DETAIL: Key (i)=(383) is duplicated.
> > and
> > 'pgbench: error: pgbench:client 23 script 0 aborted in command 31
> > query 0: ERROR: division by zero
> >
> > Last one is not MVCC-related; you can see from the logs that it
> > performs something like SELECT (509063) / 0 when the table sum
> > changes.
> >
> > Setting need_shared_catalogs = true make them pass, so something is
> > wrong with its correctness.
>
> Thanks for testing again. Whether we keep the "database specific slots" or
> not, it'd be good to know what exactly the reason of these errors is. I wonder
> if the feature just exposes a problem that remains shadowed otherwise, due to
> the contention on replication slot. I'm going to investigate.
I think the problem is that with database-specific snapshot,
SnapBuildProcessRunningXacts() returns early, w/o adjusting builder->xmin
/*
* Database specific transaction info may exist to reach CONSISTENT state
* faster, however the code below makes no use of it. Moreover, such
* record might cause problems because the following normal (cluster-wide)
* record can have lower value of oldestRunningXid. In that case, let's
* wait with the cleanup for the next regular cluster-wide record.
*/
if (OidIsValid(running->dbid))
return;
and thus some transactions whose XID is below running->oldestRunningXid may
continue to be incorrectly considered running.
I originally thought that this should not happen because such transactions
will be added to the builder's array of committed transactions by
SnapBuildCommitTxn() anyway. However, I failed to notice that COMMIT record of
a transaction listed in the xl_running_xacts WAL record is not guaranteed to
follow the xl_running_xacts record in WAL. In other words, even if
xl_running_xacts is created before a COMMIT record of the contained
transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
might not take place.
I've got no good idea how to fix that. Not sure I'm able to pursue the
"database-specific snapshots" feature now.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-05 15:02 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-06 08:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-05 15:02 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-May-05, Antonin Houska wrote:
> However, I failed to notice that COMMIT record of
> a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> follow the xl_running_xacts record in WAL. In other words, even if
> xl_running_xacts is created before a COMMIT record of the contained
> transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> might not take place.
That's pretty bad news.
> I've got no good idea how to fix that. Not sure I'm able to pursue the
> "database-specific snapshots" feature now.
It appears that the only reasonable course of action at this point is to
revert 0d3dba38c777.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 15:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-05-06 08:25 ` Antonin Houska <ah@cybertec.at>
2026-05-08 12:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-05-06 08:25 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-May-05, Antonin Houska wrote:
>
> > However, I failed to notice that COMMIT record of
> > a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> > follow the xl_running_xacts record in WAL. In other words, even if
> > xl_running_xacts is created before a COMMIT record of the contained
> > transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> > might not take place.
>
> That's pretty bad news.
>
> > I've got no good idea how to fix that.
One idea occurred to me yet, effectively it's just a cleanup. Part of it was
already proposed [1].
[1] https://www.postgresql.org/message-id/flat/CAHg%2BQDcQak4jx_6X2_Ws98rzG%3DxBARLjqm_%3D56wTRUtNsY4DZQ...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 15:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-06 08:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-08 12:25 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-08 13:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-08 12:25 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, May 6, 2026 at 1:55 PM Antonin Houska <ah@cybertec.at> wrote:
>
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > On 2026-May-05, Antonin Houska wrote:
> >
> > > However, I failed to notice that COMMIT record of
> > > a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> > > follow the xl_running_xacts record in WAL. In other words, even if
> > > xl_running_xacts is created before a COMMIT record of the contained
> > > transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> > > might not take place.
> >
> > That's pretty bad news.
> >
> > > I've got no good idea how to fix that.
>
> One idea occurred to me yet, effectively it's just a cleanup. Part of it was
> already proposed [1].
>
Some issues/inefficiencies regarding this fix and base code related to
db-specific snapshots built during decoding:
*
After fix, whenever a db-specific decoder sees a cluster-wide
xl_running_xacts record, it unconditionally calls
LogStandbySnapshot(MyDatabaseId) and returns. This triggers for every
cluster-wide record the decoder encounters (including post snapbuild's
CONSISTENT state) , for the entire duration of the decoding session.
LogStandbySnapshot acquires ProcArrayLock + XidGenLock, calls
GetRunningTransactionData, and writes WAL. With N active db-specific
decoding sessions, each cluster-wide record now triggers N additional
WAL writes.
Additionally, LogStandbySnapshot also logs AccessExclusiveLocks before
the running_xacts record. Physical standbys skip db-specific
XLOG_RUNNING_XACTS records via standby_redo(), but they do process the
preceding XLOG_STANDBY_LOCK records. The same locks may already have
been logged in the most recent cluster-wide snapshot. Physical
standbys could end up processing these lock records twice which may
not be harmful because I think we avoid re-acquiring the lock but
still it is a new overhead in the system.
*
When a cluster-wide running_xacts record arrives:
SnapBuildProcessRunningXacts calls LogStandbySnapshot and returns
early. ReorderBufferAbortOld is called, but with the cluster-wide
oldestRunningXid, which could lag far behind the db-specific value
(due to a long-running transaction in another database).
When a db-specific record arrives: SnapBuildProcessRunningXacts
processes it and advances builder->xmin with the db-specific (more
current) oldestRunningXid. But ReorderBufferAbortOld is NOT called for
db-specific records. This means the reorder buffer is cleaned up using
a conservative, potentially very old, cluster-wide oldestRunningXid,
even though builder->xmin has already advanced much further. The
reorder buffer holds stale entries longer than necessary, increasing
memory pressure.
*
I also see a design level problem with plugins that have
need_shared_catalogs=false and use failover slots. IIUC, the
db-specific optimization was designed around a live decoding session
on the primary which can emit and immediately read its own db-specific
records in the WAL stream to reach consistent state. The
LogicalSlotAdvanceAndCheckSnapState path used by slotsync has a
bounded WAL window and cannot exploit the feedback loop, making the
two mechanisms fundamentally incompatible. I know the slot created by
pgrepack doesn't enable failover option but we have not even added any
guards or thought about db-specific snapbuilds with other parts of the
system that rely on cluster-wide running_xact records, so there could
be more problems which we don't see with the current set of tests.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 15:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-06 08:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-08 12:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-08 13:58 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-08 23:22 ` Re: Adding REPACK [concurrently] Masahiko Sawada <sawada.mshk@gmail.com>
2026-05-10 11:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-08 13:58 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-May-08, Amit Kapila wrote:
> On Wed, May 6, 2026 at 1:55 PM Antonin Houska <ah@cybertec.at> wrote:
> > One idea occurred to me yet, effectively it's just a cleanup. Part of it was
> > already proposed [1].
Hmm, I think this cleanup makes sense. If I apply the test patches
(0001 and 0002 here), they fail almost immediately; but after applying
0003 all is again well. I think these tests are a good thing to have in
the tree, even if we end up reverting db-specific snapshots later.
After some back and forth, I modified the tests slightly so that
the search PG_TEST_EXTRA for the string "stress_concurrently=N". The N
can be changed so that the tests run for longer; if not given, it's
taken as 1, and the tests run for around 6 seconds (so N=10 means runs
for a minute). I think this is a convenient gadget for other tests of
this kind on CONCURRENTLY commands, such as the one proposed for CIC
elsewhere.
As written, these tests would run nowhere until we add that string in
some buildfarm animal. I debated with myself whether to assume N=1 when
the string is not given. I still think that's a good idea but perhaps
we should have something to prevent it from running by default when
under Valgrind or other slow things like that. In normal conditions,
the total runtime is not affected when they are run with N=1 as part of
the whole test suite.
> Some issues/inefficiencies regarding this fix and base code related to
> db-specific snapshots built during decoding: [...]
Thanks for spending time reviewing this code. I think none of these
problems are fundamental in nature, but they are obviously worth
addressing for 19. If we hit some roadblock, we can still revert only
db-specific snapshots.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Puedes vivir sólo una vez, pero si lo haces bien, una vez es suficiente"
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 15:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-06 08:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-08 12:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-08 13:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-05-08 23:22 ` Masahiko Sawada <sawada.mshk@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Masahiko Sawada @ 2026-05-08 23:22 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Fri, May 8, 2026 at 6:58 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-May-08, Amit Kapila wrote:
>
> > On Wed, May 6, 2026 at 1:55 PM Antonin Houska <ah@cybertec.at> wrote:
>
> > Some issues/inefficiencies regarding this fix and base code related to
> > db-specific snapshots built during decoding: [...]
>
> Thanks for spending time reviewing this code. I think none of these
> problems are fundamental in nature, but they are obviously worth
> addressing for 19. If we hit some roadblock, we can still revert only
> db-specific snapshots.
>
I'm still studying REPACK (CONCURRENTLY), and I have a question about
db-specific decoder; as far as I read the codes, any decoding plugin
can use db-specific decoder by setting need_shared_catalog=false but
should we prevent such slots from being created on standbys? I'm a bit
concerned that plugin developers inadvertently set
need_shared_catalog=false in the startup callback and users would
create such slots on standbys. For instance, if we create a logical
slot with pgrepack plugin on the standby, we would get an assertion
failure as the db-specific decoder tries to call LogStandbySnapshot()
via SnapBuildProcessRunningXacts(). Even if we add
!RecoveryInProgress() check there, the db-specific decoder on standbys
requires for the primary server to run a REPACK (CONCURRENTLY).
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 15:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-06 08:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-08 12:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-08 13:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-05-10 11:24 ` Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Amit Kapila @ 2026-05-10 11:24 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Fri, May 8, 2026 at 7:28 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-May-08, Amit Kapila wrote:
>
> > Some issues/inefficiencies regarding this fix and base code related to
> > db-specific snapshots built during decoding: [...]
>
> Thanks for spending time reviewing this code. I think none of these
> problems are fundamental in nature, but they are obviously worth
> addressing for 19. If we hit some roadblock, we can still revert only
> db-specific snapshots.
>
IIUC, the emails by Andres [1][2] on db-specific snapshots sound like
concerns which are fundamental in nature. Apart from that as well, I
think the first point mentioned in my email [3] should be at least
addressed as that causes additional WAL even after reaching
consistent_state for each runing_xact record for a db-specific
decoder.
[1] - https://www.postgresql.org/message-id/cdgw4sbbfcgk6du3iv54r2dgiy4tfywoklbotlmj4irxavdcr3%40glxfw5jj2...
[2] - https://www.postgresql.org/message-id/pveffyxhnuurhb44uzqlwo3rkyzorkfh2rot7uwzlf2axhfvbp%407nrs2omys...
[3] - https://www.postgresql.org/message-id/CAA4eK1LygCDP3FiFzXY9iVNFcHxhf7TT_DFf7tryTu2oipmfpA%40mail.gma...
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-10 11:31 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-10 11:31 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, May 5, 2026 at 6:17 PM Antonin Houska <ah@cybertec.at> wrote:
>
> Antonin Houska <ah@cybertec.at> wrote:
>
> I think the problem is that with database-specific snapshot,
> SnapBuildProcessRunningXacts() returns early, w/o adjusting builder->xmin
>
> /*
> * Database specific transaction info may exist to reach CONSISTENT state
> * faster, however the code below makes no use of it. Moreover, such
> * record might cause problems because the following normal (cluster-wide)
> * record can have lower value of oldestRunningXid. In that case, let's
> * wait with the cleanup for the next regular cluster-wide record.
> */
> if (OidIsValid(running->dbid))
> return;
>
> and thus some transactions whose XID is below running->oldestRunningXid may
> continue to be incorrectly considered running.
>
> I originally thought that this should not happen because such transactions
> will be added to the builder's array of committed transactions by
> SnapBuildCommitTxn() anyway. However, I failed to notice that COMMIT record of
> a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> follow the xl_running_xacts record in WAL. In other words, even if
> xl_running_xacts is created before a COMMIT record of the contained
> transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> might not take place.
>
BTW, is it possible to write a test by using injection_points or via
manual steps (by using debugger, etc) so that we can more clearly
understand this problem and proposed fix?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-11 15:17 ` Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 02:25 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-05-11 15:17 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Tue, May 5, 2026 at 6:17 PM Antonin Houska <ah@cybertec.at> wrote:
> >
> > Antonin Houska <ah@cybertec.at> wrote:
> >
> > I think the problem is that with database-specific snapshot,
> > SnapBuildProcessRunningXacts() returns early, w/o adjusting builder->xmin
> >
> > /*
> > * Database specific transaction info may exist to reach CONSISTENT state
> > * faster, however the code below makes no use of it. Moreover, such
> > * record might cause problems because the following normal (cluster-wide)
> > * record can have lower value of oldestRunningXid. In that case, let's
> > * wait with the cleanup for the next regular cluster-wide record.
> > */
> > if (OidIsValid(running->dbid))
> > return;
> >
> > and thus some transactions whose XID is below running->oldestRunningXid may
> > continue to be incorrectly considered running.
> >
> > I originally thought that this should not happen because such transactions
> > will be added to the builder's array of committed transactions by
> > SnapBuildCommitTxn() anyway. However, I failed to notice that COMMIT record of
> > a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> > follow the xl_running_xacts record in WAL. In other words, even if
> > xl_running_xacts is created before a COMMIT record of the contained
> > transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> > might not take place.
> >
>
> BTW, is it possible to write a test by using injection_points or via
> manual steps (by using debugger, etc) so that we can more clearly
> understand this problem and proposed fix?
So far I could observe the situation in WAL, but have no idea how it can
happen. For example, transaction 49242 gets committed here
rmgr: Transaction len (rec/tot): 46/ 46, tx: 49242, lsn: 0/18BC28C8, prev
0/18BC2890, desc: COMMIT 2026-05-11 16:38:16.603265 CEST
and then it appears in the 'xids' list of RUNNING_XACTS:
rmgr: Standby len (rec/tot): 106/ 106, tx: 0, lsn:
0/18BC3140, prev 0/18BC3100, desc: RUNNING_XACTS nextXid 49255
latestCompletedXid 49241 oldestRunningXid 49242; 13 xacts: 49248 49249 49246
49243 49252 49251 49244 49245 49242 49250 49253 49254 49247; dbid:5
I thought the situation is quite common (and therefore nothing of
SnapBuildProcessRunningXacts() should be skipped), but when trying to
reproduce the problem, I noticed that LogStandbySnapshot() shouldn't allow
that ordering issue when logical decoding is enabled:
/*
* GetRunningTransactionData() acquired ProcArrayLock, we must release it.
* For Hot Standby this can be done before inserting the WAL record
* because ProcArrayApplyRecoveryInfo() rechecks the commit status using
* the clog. For logical decoding, though, the lock can't be released
* early because the clog might be "in the future" from the POV of the
* historic snapshot. This would allow for situations where we're waiting
* for the end of a transaction listed in the xl_running_xacts record
* which, according to the WAL, has committed before the xl_running_xacts
* record. Fortunately this routine isn't executed frequently, and it's
* only a shared lock.
*/
if (!logical_decoding_enabled)
LWLockRelease(ProcArrayLock);
So I don't have the answer right now.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-11 19:30 ` Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-05-11 19:30 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > On Tue, May 5, 2026 at 6:17 PM Antonin Houska <ah@cybertec.at> wrote:
> > >
> > > Antonin Houska <ah@cybertec.at> wrote:
> > >
> > > I think the problem is that with database-specific snapshot,
> > > SnapBuildProcessRunningXacts() returns early, w/o adjusting builder->xmin
> > >
> > > /*
> > > * Database specific transaction info may exist to reach CONSISTENT state
> > > * faster, however the code below makes no use of it. Moreover, such
> > > * record might cause problems because the following normal (cluster-wide)
> > > * record can have lower value of oldestRunningXid. In that case, let's
> > > * wait with the cleanup for the next regular cluster-wide record.
> > > */
> > > if (OidIsValid(running->dbid))
> > > return;
> > >
> > > and thus some transactions whose XID is below running->oldestRunningXid may
> > > continue to be incorrectly considered running.
> > >
> > > I originally thought that this should not happen because such transactions
> > > will be added to the builder's array of committed transactions by
> > > SnapBuildCommitTxn() anyway. However, I failed to notice that COMMIT record of
> > > a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> > > follow the xl_running_xacts record in WAL. In other words, even if
> > > xl_running_xacts is created before a COMMIT record of the contained
> > > transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> > > might not take place.
> > >
> >
> > BTW, is it possible to write a test by using injection_points or via
> > manual steps (by using debugger, etc) so that we can more clearly
> > understand this problem and proposed fix?
>
> So far I could observe the situation in WAL, but have no idea how it can
> happen. For example, transaction 49242 gets committed here
>
> rmgr: Transaction len (rec/tot): 46/ 46, tx: 49242, lsn: 0/18BC28C8, prev
> 0/18BC2890, desc: COMMIT 2026-05-11 16:38:16.603265 CEST
>
> and then it appears in the 'xids' list of RUNNING_XACTS:
>
> rmgr: Standby len (rec/tot): 106/ 106, tx: 0, lsn:
> 0/18BC3140, prev 0/18BC3100, desc: RUNNING_XACTS nextXid 49255
> latestCompletedXid 49241 oldestRunningXid 49242; 13 xacts: 49248 49249 49246
> 49243 49252 49251 49244 49245 49242 49250 49253 49254 49247; dbid:5
>
>
> I thought the situation is quite common (and therefore nothing of
> SnapBuildProcessRunningXacts() should be skipped), but when trying to
> reproduce the problem, I noticed that LogStandbySnapshot() shouldn't allow
> that ordering issue when logical decoding is enabled:
>
> /*
> * GetRunningTransactionData() acquired ProcArrayLock, we must release it.
> * For Hot Standby this can be done before inserting the WAL record
> * because ProcArrayApplyRecoveryInfo() rechecks the commit status using
> * the clog. For logical decoding, though, the lock can't be released
> * early because the clog might be "in the future" from the POV of the
> * historic snapshot. This would allow for situations where we're waiting
> * for the end of a transaction listed in the xl_running_xacts record
> * which, according to the WAL, has committed before the xl_running_xacts
> * record. Fortunately this routine isn't executed frequently, and it's
> * only a shared lock.
> */
> if (!logical_decoding_enabled)
> LWLockRelease(ProcArrayLock);
>
> So I don't have the answer right now.
I think now that "waiting for the end of a transaction listed in the
xl_running_xacts record" in the comment is about transaction removal from
procarray. However, the COMMIT record can still be ahead of xl_running_xacts
because RecordTransactionCommit() is called before
ProcArrayEndTransaction(). I'll think again if the whole problem can be
reproduced with injection points.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-12 07:57 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-12 07:57 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, May 12, 2026 at 1:00 AM Antonin Houska <ah@cybertec.at> wrote:
>
> Antonin Houska <ah@cybertec.at> wrote:
>
> > Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > > On Tue, May 5, 2026 at 6:17 PM Antonin Houska <ah@cybertec.at> wrote:
> > > >
> > > > Antonin Houska <ah@cybertec.at> wrote:
> > > >
> > > > I think the problem is that with database-specific snapshot,
> > > > SnapBuildProcessRunningXacts() returns early, w/o adjusting builder->xmin
> > > >
> > > > /*
> > > > * Database specific transaction info may exist to reach CONSISTENT state
> > > > * faster, however the code below makes no use of it. Moreover, such
> > > > * record might cause problems because the following normal (cluster-wide)
> > > > * record can have lower value of oldestRunningXid. In that case, let's
> > > > * wait with the cleanup for the next regular cluster-wide record.
> > > > */
> > > > if (OidIsValid(running->dbid))
> > > > return;
> > > >
> > > > and thus some transactions whose XID is below running->oldestRunningXid may
> > > > continue to be incorrectly considered running.
> > > >
> > > > I originally thought that this should not happen because such transactions
> > > > will be added to the builder's array of committed transactions by
> > > > SnapBuildCommitTxn() anyway. However, I failed to notice that COMMIT record of
> > > > a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> > > > follow the xl_running_xacts record in WAL. In other words, even if
> > > > xl_running_xacts is created before a COMMIT record of the contained
> > > > transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> > > > might not take place.
> > > >
> > >
> > > BTW, is it possible to write a test by using injection_points or via
> > > manual steps (by using debugger, etc) so that we can more clearly
> > > understand this problem and proposed fix?
> >
> > So far I could observe the situation in WAL, but have no idea how it can
> > happen. For example, transaction 49242 gets committed here
> >
> > rmgr: Transaction len (rec/tot): 46/ 46, tx: 49242, lsn: 0/18BC28C8, prev
> > 0/18BC2890, desc: COMMIT 2026-05-11 16:38:16.603265 CEST
> >
> > and then it appears in the 'xids' list of RUNNING_XACTS:
> >
> > rmgr: Standby len (rec/tot): 106/ 106, tx: 0, lsn:
> > 0/18BC3140, prev 0/18BC3100, desc: RUNNING_XACTS nextXid 49255
> > latestCompletedXid 49241 oldestRunningXid 49242; 13 xacts: 49248 49249 49246
> > 49243 49252 49251 49244 49245 49242 49250 49253 49254 49247; dbid:5
> >
> >
> > I thought the situation is quite common (and therefore nothing of
> > SnapBuildProcessRunningXacts() should be skipped), but when trying to
> > reproduce the problem, I noticed that LogStandbySnapshot() shouldn't allow
> > that ordering issue when logical decoding is enabled:
> >
> > /*
> > * GetRunningTransactionData() acquired ProcArrayLock, we must release it.
> > * For Hot Standby this can be done before inserting the WAL record
> > * because ProcArrayApplyRecoveryInfo() rechecks the commit status using
> > * the clog. For logical decoding, though, the lock can't be released
> > * early because the clog might be "in the future" from the POV of the
> > * historic snapshot. This would allow for situations where we're waiting
> > * for the end of a transaction listed in the xl_running_xacts record
> > * which, according to the WAL, has committed before the xl_running_xacts
> > * record. Fortunately this routine isn't executed frequently, and it's
> > * only a shared lock.
> > */
> > if (!logical_decoding_enabled)
> > LWLockRelease(ProcArrayLock);
> >
> > So I don't have the answer right now.
>
> I think now that "waiting for the end of a transaction listed in the
> xl_running_xacts record" in the comment is about transaction removal from
> procarray. However, the COMMIT record can still be ahead of xl_running_xacts
> because RecordTransactionCommit() is called before
> ProcArrayEndTransaction().
>
I see your point. Due to this, once the xmin regresses based on
cluster-wide running_xact, some transaction could appear to be running
when it should have appeared as committed. However, still it is not
clear how it could lead to one update in the transaction as
successfully decoded and another one to be skipped. One theory could
be that before the second update, somehow invalidation happens and
when decoding tries to reload the catalog to decode second update, the
relation is not visible because xmin has regressed and the update is
somehow skipped. I can't see how it can happen in code but something
like that is happening. Assuming, the problematic case is something
like what I described, even than the fix of skipping cluster-wide
running xacts and instead LOG db-specific running_xacts to help
updating builder's xmin sounds inelegant and probably inefficient. For
example, I think such a dependency means we can never enable
db-specific snapshots on standby.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-12 11:08 ` Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-05-12 11:08 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Tue, May 12, 2026 at 1:00 AM Antonin Houska <ah@cybertec.at> wrote:
> >
> > Antonin Houska <ah@cybertec.at> wrote:
> >
> > > Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > > On Tue, May 5, 2026 at 6:17 PM Antonin Houska <ah@cybertec.at> wrote:
> > > > >
> > > > > Antonin Houska <ah@cybertec.at> wrote:
> > > > >
> > > > > I think the problem is that with database-specific snapshot,
> > > > > SnapBuildProcessRunningXacts() returns early, w/o adjusting builder->xmin
> > > > >
> > > > > /*
> > > > > * Database specific transaction info may exist to reach CONSISTENT state
> > > > > * faster, however the code below makes no use of it. Moreover, such
> > > > > * record might cause problems because the following normal (cluster-wide)
> > > > > * record can have lower value of oldestRunningXid. In that case, let's
> > > > > * wait with the cleanup for the next regular cluster-wide record.
> > > > > */
> > > > > if (OidIsValid(running->dbid))
> > > > > return;
> > > > >
> > > > > and thus some transactions whose XID is below running->oldestRunningXid may
> > > > > continue to be incorrectly considered running.
> > > > >
> > > > > I originally thought that this should not happen because such transactions
> > > > > will be added to the builder's array of committed transactions by
> > > > > SnapBuildCommitTxn() anyway. However, I failed to notice that COMMIT record of
> > > > > a transaction listed in the xl_running_xacts WAL record is not guaranteed to
> > > > > follow the xl_running_xacts record in WAL. In other words, even if
> > > > > xl_running_xacts is created before a COMMIT record of the contained
> > > > > transaction, it may end up at higher LSN in WAL. So the cleanup I relied on
> > > > > might not take place.
> > > > >
> > > >
> > > > BTW, is it possible to write a test by using injection_points or via
> > > > manual steps (by using debugger, etc) so that we can more clearly
> > > > understand this problem and proposed fix?
> > >
> > > So far I could observe the situation in WAL, but have no idea how it can
> > > happen. For example, transaction 49242 gets committed here
> > >
> > > rmgr: Transaction len (rec/tot): 46/ 46, tx: 49242, lsn: 0/18BC28C8, prev
> > > 0/18BC2890, desc: COMMIT 2026-05-11 16:38:16.603265 CEST
> > >
> > > and then it appears in the 'xids' list of RUNNING_XACTS:
> > >
> > > rmgr: Standby len (rec/tot): 106/ 106, tx: 0, lsn:
> > > 0/18BC3140, prev 0/18BC3100, desc: RUNNING_XACTS nextXid 49255
> > > latestCompletedXid 49241 oldestRunningXid 49242; 13 xacts: 49248 49249 49246
> > > 49243 49252 49251 49244 49245 49242 49250 49253 49254 49247; dbid:5
> > >
> > >
> > > I thought the situation is quite common (and therefore nothing of
> > > SnapBuildProcessRunningXacts() should be skipped), but when trying to
> > > reproduce the problem, I noticed that LogStandbySnapshot() shouldn't allow
> > > that ordering issue when logical decoding is enabled:
> > >
> > > /*
> > > * GetRunningTransactionData() acquired ProcArrayLock, we must release it.
> > > * For Hot Standby this can be done before inserting the WAL record
> > > * because ProcArrayApplyRecoveryInfo() rechecks the commit status using
> > > * the clog. For logical decoding, though, the lock can't be released
> > > * early because the clog might be "in the future" from the POV of the
> > > * historic snapshot. This would allow for situations where we're waiting
> > > * for the end of a transaction listed in the xl_running_xacts record
> > > * which, according to the WAL, has committed before the xl_running_xacts
> > > * record. Fortunately this routine isn't executed frequently, and it's
> > > * only a shared lock.
> > > */
> > > if (!logical_decoding_enabled)
> > > LWLockRelease(ProcArrayLock);
> > >
> > > So I don't have the answer right now.
> >
> > I think now that "waiting for the end of a transaction listed in the
> > xl_running_xacts record" in the comment is about transaction removal from
> > procarray. However, the COMMIT record can still be ahead of xl_running_xacts
> > because RecordTransactionCommit() is called before
> > ProcArrayEndTransaction().
> >
>
> I see your point. Due to this, once the xmin regresses based on
> cluster-wide running_xact, some transaction could appear to be running
> when it should have appeared as committed.
The problem is that xmin does not advance when it should. Attached is a test
that reproduces the problem (it includes [1], to handle injection points in
background worker), I hope the comments in the specification file are helpful.
It's actually not exactly the problem reported in the stress test, but IMO the
core issue is the same: effects of some transactions are lost. In the stress
test, tuple deletion was lost, so the error was "could not create unique
index". Here I only demonstrate lost INSERT.
> Assuming, the problematic case is something
> like what I described, even than the fix of skipping cluster-wide
> running xacts and instead LOG db-specific running_xacts to help
> updating builder's xmin sounds inelegant and probably inefficient. For
> example, I think such a dependency means we can never enable
> db-specific snapshots on standby.
ok
[1] https://www.postgresql.org/message-id/4703.1774250534%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-13 12:34 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-13 12:34 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, May 12, 2026 at 4:38 PM Antonin Houska <ah@cybertec.at> wrote:
>
> Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> >
> > I see your point. Due to this, once the xmin regresses based on
> > cluster-wide running_xact, some transaction could appear to be running
> > when it should have appeared as committed.
>
> The problem is that xmin does not advance when it should. Attached is a test
> that reproduces the problem (it includes [1], to handle injection points in
> background worker), I hope the comments in the specification file are helpful.
>
Yes, the comments were helpful. IIUC, the test skipped insert into
repack_test because the transaction doing that insert happened before
the snapbuild reached FULL_SNAPSHOT/CONSISTENT state, so its commit is
not decoded. Then we also didn't update builder->xmin after reaching
CONSISTENT state in the last running_xact record for MyDatabase. So,
the insert is neither covered in initial copy, nor decoded, so after
repack (concurrently) is finished the table is empty.
I think the patch proposed will fix this specific issue but apart from
other points raised for this patch few more are: (a) Post CONSISTENT
state, for cleanup of db_specific snapshots, we need to separately
again LOG db-specific running xacts whenever we encounter another
running_xacts, (b) Other point is that because
repack-concurrently always use full-snapshot, the serialization of
snapshot is useless because it can't be used by others.
> It's actually not exactly the problem reported in the stress test, but IMO the
> core issue is the same: effects of some transactions are lost. In the stress
> test, tuple deletion was lost, so the error was "could not create unique
> index". Here I only demonstrate lost INSERT.
>
> > Assuming, the problematic case is something
> > like what I described, even than the fix of skipping cluster-wide
> > running xacts and instead LOG db-specific running_xacts to help
> > updating builder's xmin sounds inelegant and probably inefficient. For
> > example, I think such a dependency means we can never enable
> > db-specific snapshots on standby.
>
> ok
>
So now the question is where do we go from here. I am not confident
that the current code to achieve db-specific snapshots in logical
decoding is the best possible solution both because of the drawbacks
(like we won't be able to enable this on standby) and inefficiencies
pointed out by me in this and previous emails in this work.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-13 16:58 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-13 16:58 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello Amit,
On 2026-May-13, Amit Kapila wrote:
> So now the question is where do we go from here. I am not confident
> that the current code to achieve db-specific snapshots in logical
> decoding is the best possible solution both because of the drawbacks
> (like we won't be able to enable this on standby) and inefficiencies
> pointed out by me in this and previous emails in this work.
This is a fair question. I don't think we have time to go much further
on this aspect before beta 1, so we either accept this patch, fix the
inefficiencies you pointed out and keep db-specific snapshots, or we
revert db-specific snapshots and go back to the standard snapshot-taking
technique for REPACK in 19 and see what we can improve for 20.
Now, the worst consequence of reverting db-specific snapshots is that
you will only be able to run REPACK in a single database at a time
(because any subsequent REPACK will have to wait until the first one
finishes before being able to get its snapshot). In most normal cases
this is probably not a big deal. But if you have a multitenant system,
and you want your users to be able to run REPACK on their tables, you
may be a bit screwed. So I hesitate to just go and revert it without
offering those people any alternative.
(It's also possible that being unable to run more than one REPACK at a
time is not so big a deal. After all, it's supposed to be an infrequent
operation. And users probably don't or shouldn't have multi-terabyte
tables in multitenant databases anyway.)
I'm not sure I understand the point of the standby. I mean, you can't
run REPACK on the standby anyway, so I don't see this as a very
problematic restriction. Do you have other reasons for wanting a
db-specific snapshot in a standby?
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-05-14 07:02 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-14 07:02 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, May 13, 2026 at 10:28 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Hello Amit,
>
> On 2026-May-13, Amit Kapila wrote:
>
> > So now the question is where do we go from here. I am not confident
> > that the current code to achieve db-specific snapshots in logical
> > decoding is the best possible solution both because of the drawbacks
> > (like we won't be able to enable this on standby) and inefficiencies
> > pointed out by me in this and previous emails in this work.
>
> This is a fair question. I don't think we have time to go much further
> on this aspect before beta 1, so we either accept this patch, fix the
> inefficiencies you pointed out and keep db-specific snapshots,
>
I don't think it would be easy to address these inefficiencies before
beta 1. The root of those inefficiencies is that the patch reuses the
cluster-wide running_xact WAL infrastructure to log db-specific
running transactions, and then tries to feed that into the existing
snapbuild machinery to reach a consistent state.
As another example of this mismatch that occurred to me today: in
SnapBuildCommitTxn, we are tracking the committed_xip array for all
cluster-wide XIDs, even when using a db-specific snapshot. A
db-specific snapshot shouldn't need to care about XIDs from other
databases. We only try to take care of it in one part of the system
where process running_xacts record. I admit that I don't know at this
stage what exactly we should do about it but all such things deserve a
discussion and careful thought.
The broader issue is that the entire logical decoding mechanism is
designed to process cluster-wide transactions. This patch tries to
bypass that foundational assumption, but only during the initial
snapshot construction while processing running_xacts record.
To be clear, I am not against the idea of db-specific snapshots to
enable concurrent repacks. My concern is simply the time required to
get the architecture right. In its current state, we need more time to
carefully consider how this db-specific concept interacts with the
rest of the logical decoding machinery, which is built for
cluster-wide records.
> or we
> revert db-specific snapshots and go back to the standard snapshot-taking
> technique for REPACK in 19 and see what we can improve for 20.
>
> Now, the worst consequence of reverting db-specific snapshots is that
> you will only be able to run REPACK in a single database at a time
> (because any subsequent REPACK will have to wait until the first one
> finishes before being able to get its snapshot). In most normal cases
> this is probably not a big deal. But if you have a multitenant system,
> and you want your users to be able to run REPACK on their tables, you
> may be a bit screwed. So I hesitate to just go and revert it without
> offering those people any alternative.
>
I understand your point but I feel we can extend the current feature
in future versions to address such cases (allow REPACK CONCURRENTLY on
tables in multiple-databases simultaneously). For now, they may need
to rely on REPACK without CONCURRENTLY option, if they want to use it
for multiple databases simultaneously.
> (It's also possible that being unable to run more than one REPACK at a
> time is not so big a deal. After all, it's supposed to be an infrequent
> operation. And users probably don't or shouldn't have multi-terabyte
> tables in multitenant databases anyway.)
>
> I'm not sure I understand the point of the standby. I mean, you can't
> run REPACK on the standby anyway, so I don't see this as a very
> problematic restriction. Do you have other reasons for wanting a
> db-specific snapshot in a standby?
>
We are exposing need_shared_catalogs as a generic plugin option,
defined as: 'it can be set to false if one is certain the plugin
functions do not access shared system catalogs.' This implies it can
be used for purposes other than REPACK.
For example, one can imagine a single-database audit plugin that only
cares about data modifications within a specific database. By setting
need_shared_catalogs = false on a standby, it could reach a CONSISTENT
state much faster, perfectly serving its needs.
While such a plugin might not exist right now, my broader point is
this: when we expose a generic facility, it can and will be used in
ways beyond our initial core use cases. We should try to ensure the
design doesn't permanently preclude such extensions. With the current
design choice, we are painting ourselves into a corner where this
feature cannot easily be extended to standbys even in the future.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-19 18:52 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-23 15:29 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-19 18:52 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-May-14, Amit Kapila wrote:
> The broader issue is that the entire logical decoding mechanism is
> designed to process cluster-wide transactions. This patch tries to
> bypass that foundational assumption, but only during the initial
> snapshot construction while processing running_xacts record.
>
> To be clear, I am not against the idea of db-specific snapshots to
> enable concurrent repacks. My concern is simply the time required to
> get the architecture right. In its current state, we need more time to
> carefully consider how this db-specific concept interacts with the
> rest of the logical decoding machinery, which is built for
> cluster-wide records.
Hmm. So at this point I have to admit that the time I'll have before
beta 1 is going to be very scarce. You're probably right that it's
better to revert db-specific snapshots in pg19, and try again for 20.
The revert should be a simple patch.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-05-23 15:29 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-24 11:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-05-23 15:29 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, May 19, 2026 at 11:52 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-May-14, Amit Kapila wrote:
>
> > The broader issue is that the entire logical decoding mechanism is
> > designed to process cluster-wide transactions. This patch tries to
> > bypass that foundational assumption, but only during the initial
> > snapshot construction while processing running_xacts record.
> >
> > To be clear, I am not against the idea of db-specific snapshots to
> > enable concurrent repacks. My concern is simply the time required to
> > get the architecture right. In its current state, we need more time to
> > carefully consider how this db-specific concept interacts with the
> > rest of the logical decoding machinery, which is built for
> > cluster-wide records.
>
> Hmm. So at this point I have to admit that the time I'll have before
> beta 1 is going to be very scarce. You're probably right that it's
> better to revert db-specific snapshots in pg19, and try again for 20.
>
Sounds reasonable.
> The revert should be a simple patch.
>
I also think so.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-23 15:29 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-24 11:29 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-03 17:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-24 11:29 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-May-23, Amit Kapila wrote:
> > The revert should be a simple patch.
>
> I also think so.
Okay, pushed the revert after seeing it pass CI:
https://cirrus-ci.com/build/5520722497372160
Thanks!
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-23 15:29 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-24 11:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-06-03 17:27 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-06-17 18:19 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-06-03 17:27 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Antonin Houska <ah@cybertec.at>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, everyone!
I think it is a good time to remind we still have a potential deadlock
issue leading to loosing vast amount of REPACK work.
Best soltuion I was able to craft so far is [1].
Best regards,
Mikhail.
P.S.
I'll continue working on the repack stress test suite in the near future.
[1]: https://www.postgresql.org/message-id/flat/CADzfLwXbUWuS6H4uJEFVL1jS1kzsVnuJ%2BzX1%2BtAEhQxBnEiGKw%4...
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-23 15:29 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-24 11:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-03 17:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-06-17 18:19 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-17 20:10 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-07-01 14:43 ` Re: Adding REPACK [concurrently] Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-06-17 18:19 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Antonin Houska <ah@cybertec.at>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello Mihail,
On 2026-Jun-03, Mihail Nikalayeu wrote:
> Hello, everyone!
>
> I think it is a good time to remind we still have a potential deadlock
> issue leading to loosing vast amount of REPACK work.
>
> Best soltuion I was able to craft so far is [1].
> [1]: https://www.postgresql.org/message-id/flat/CADzfLwXbUWuS6H4uJEFVL1jS1kzsVnuJ%2BzX1%2BtAEhQxBnEiGKw%4...
Right. I spent some time looking at this patch just before pgconf.dev,
and I have to admit I was a bit scared -- that code is really
non-obvious, and breakage here could mean big trouble in case something
goes wrong in weird situations.
So I think we should let this slip for pg19, and get this patch reviewed
and pushed early in pg20 in the coming weeks, so that we have more of a
chance to discover breakage in the time until pg20 release next year.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Por suerte hoy explotó el califont porque si no me habría muerto
de aburrido" (Papelucho)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-23 15:29 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-24 11:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-03 17:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-06-17 18:19 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-06-17 20:10 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-06-17 20:10 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Antonin Houska <ah@cybertec.at>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Álvaro!
> Right. I spent some time looking at this patch just before pgconf.dev
Thanks!
> I have to admit I was a bit scared -- that code is really
> non-obvious, and breakage here could mean big trouble in case something
> goes wrong in weird situations.
Another option we may consider for pg19 - is just a simple
local-memory flag: "If I detect a deadlock and I am REPACK - cancel
another, not me".
It doesn't handle all tricky cases, but it deals with the most common
ones and is much easier and less invasive to implement.
It reuses mechanics that were already present (though unused, they
were documented as "for future" in comments).
POC of that approach is here - [0].
Best regards,
Mikhail.
[0]: https://www.postgresql.org/message-id/flat/CADzfLwURKVNQ++Dpi7bjoGfj-8pchDQEVex3eWBx0NCYn6TbDQ@mail....
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-23 15:29 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-24 11:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-03 17:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-06-17 18:19 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-07-01 14:43 ` Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 0 replies; 416+ messages in thread
From: Heikki Linnakangas @ 2026-07-01 14:43 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Antonin Houska <ah@cybertec.at>; Andres Freund <andres@anarazel.de>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 17/06/2026 21:19, Alvaro Herrera wrote:
> Hello Mihail,
>
> On 2026-Jun-03, Mihail Nikalayeu wrote:
>
>> Hello, everyone!
>>
>> I think it is a good time to remind we still have a potential deadlock
>> issue leading to loosing vast amount of REPACK work.
>>
>> Best soltuion I was able to craft so far is [1].
>
>> [1]: https://www.postgresql.org/message-id/flat/CADzfLwXbUWuS6H4uJEFVL1jS1kzsVnuJ%2BzX1%2BtAEhQxBnEiGKw%4...
>
> Right. I spent some time looking at this patch just before pgconf.dev,
> and I have to admit I was a bit scared -- that code is really
> non-obvious, and breakage here could mean big trouble in case something
> goes wrong in weird situations.
>
> So I think we should let this slip for pg19, and get this patch reviewed
> and pushed early in pg20 in the coming weeks, so that we have more of a
> chance to discover breakage in the time until pg20 release next year.
There's still an open item for this on the wiki [1]. Is the decision to
let this slip for pg19? If so, please update the open items list. Thanks!
[1] https://wiki.postgresql.org/wiki/PostgreSQL_19_Open_Items
- Heikki
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-05-12 02:25 ` Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
1 sibling, 0 replies; 416+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-05-12 02:25 UTC (permalink / raw)
To: 'Antonin Houska' <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Dear Antonin,
FYI, I have also been spending time to reproduce the failure but I have not done yet.
> So far I could observe the situation in WAL, but have no idea how it can
> happen.
Not sure it matches with your situation, but I could reproduce the situation by
using gdb.
0. initialized an instance with wal_level=logical and defined a table.
1. established a connection
2. attached the backend via gdb
3. added a breakpoint in ProcArrayEndTransaction
4. committed a transaction, and it would stop at the breakpoint
5. established another connection
6. ran REPACK CONCURRENTLY
7. detached from the first backend.
8. all commands would finish.
This could allow that COMMIT record exists ahead the RUNNING_XACTS record.
When the backend reaches CommitTransaction()->ProcArrayEndTransaction(), the commit
record has already been serialized, but the transaction is still marked as active
on the proc array. Above workload allowed that repack worker could check in-between.
[1]:
```
rmgr: Transaction len (rec/tot): 46/ 46, tx: 695, lsn: 0/018B89C0, prev 0/018B8980, desc: COMMIT 2026-05-12 11:11:31.588061 JST
rmgr: Standby len (rec/tot): 58/ 58, tx: 0, lsn: 0/018B89F0, prev 0/018B89C0, desc: RUNNING_XACTS nextXid 696 latestCompletedXid 694 oldestRunningXid 695; 1 xacts: 695; dbid: 0
rmgr: Standby len (rec/tot): 58/ 58, tx: 0, lsn: 0/018B8A30, prev 0/018B89F0, desc: RUNNING_XACTS nextXid 696 latestCompletedXid 694 oldestRunningXid 695; 1 xacts: 695; dbid: 5
```
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-05-05 15:04 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-05-05 15:04 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-30, Mihail Nikalayeu wrote:
> P.S.
> I think it is good idea to add these stress tests to the source tree,
> perhaps with some kind PG_TEST_EXTRA=stress (as done in [1]).
Yeah, agreed. I'll see to that shortly.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"I am amazed at [the pgsql-sql] mailing list for the wonderful support, and
lack of hesitasion in answering a lost soul's question, I just wished the rest
of the mailing list could be like this." (Fotis)
https://postgr.es/m/200606261359.k5QDxE2p004593@auth-smtp.hol.gr
^ permalink raw reply [nested|flat] 416+ messages in thread
* RE: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 12:32 ` Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-07 14:19 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-01 07:30 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
4 siblings, 2 replies; 416+ messages in thread
From: Hayato Kuroda (Fujitsu) @ 2026-04-07 12:32 UTC (permalink / raw)
To: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Dear Álvaro,
Thanks for updating the patch. I have two comments/questions.
01.
```
--- a/src/backend/access/index/genam.c
+++ b/src/backend/access/index/genam.c
@@ -394,6 +394,14 @@ systable_beginscan(Relation heapRelation,
SysScanDesc sysscan;
Relation irel;
+ /*
+ * If this backend promised that it won't access shared catalogs during
+ * logical decoding, this it the right place to verify.
+ */
+ Assert(!HistoricSnapshotActive() ||
+ accessSharedCatalogsInDecoding ||
+ !heapRelation->rd_rel->relisshared);
```
Not sure it's OK to use Assert(). elog(ERROR) might be better if we want to really
avoid the case.
02. SnapBuildProcessRunningXacts
Per my understanding, the db_specic snapshot can be also serialized. Is it
possibility tha normal logical decoding system restores the snapshot and obtain
the wrong result?
Best regards,
Hayato Kuroda
FUJITSU LIMITED
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:32 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
@ 2026-04-07 14:19 ` Antonin Houska <ah@cybertec.at>
2026-04-08 03:49 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-07 14:19 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; +Cc: 'Alvaro Herrera' <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> wrote:
> 02. SnapBuildProcessRunningXacts
>
> Per my understanding, the db_specic snapshot can be also serialized. Is it
> possibility tha normal logical decoding system restores the snapshot and obtain
> the wrong result?
I don't think that the database-specific xl_running_xacts WAL record affects
what SnapBuildSerialize() writes to disk: the contents of builder->committed,
etc. is updated by decoding COMMIT and ABORT records.
On the other hand, with that kind of record, there's probably no reason to
call SnapBuildSerialize(). Good catch, thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:32 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-07 14:19 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-08 03:49 ` Amit Kapila <amit.kapila16@gmail.com>
2026-05-08 23:22 ` Re: Adding REPACK [concurrently] Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-04-08 03:49 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, Apr 7, 2026 at 7:49 PM Antonin Houska <ah@cybertec.at> wrote:
>
> Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> wrote:
>
> > 02. SnapBuildProcessRunningXacts
> >
> > Per my understanding, the db_specic snapshot can be also serialized. Is it
> > possibility tha normal logical decoding system restores the snapshot and obtain
> > the wrong result?
>
> I don't think that the database-specific xl_running_xacts WAL record affects
> what SnapBuildSerialize() writes to disk: the contents of builder->committed,
> etc. is updated by decoding COMMIT and ABORT records.
>
I think the point is that the other process say a walsender could
restore such a snapshot making it take the wrong decision.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:32 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-07 14:19 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-08 03:49 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-05-08 23:22 ` Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Masahiko Sawada @ 2026-05-08 23:22 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, Apr 7, 2026 at 8:49 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Apr 7, 2026 at 7:49 PM Antonin Houska <ah@cybertec.at> wrote:
> >
> > Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> wrote:
> >
> > > 02. SnapBuildProcessRunningXacts
> > >
> > > Per my understanding, the db_specic snapshot can be also serialized. Is it
> > > possibility tha normal logical decoding system restores the snapshot and obtain
> > > the wrong result?
> >
> > I don't think that the database-specific xl_running_xacts WAL record affects
> > what SnapBuildSerialize() writes to disk: the contents of builder->committed,
> > etc. is updated by decoding COMMIT and ABORT records.
> >
>
> I think the point is that the other process say a walsender could
> restore such a snapshot making it take the wrong decision.
>
Right. I think it affects even concurrent REPACK (CONCURRENTLY)
running on other databases. They could end up restoring the snapshot
serialized by another REPACK command running on another database and
becoming the consistent state without waiting for transactions
concurrently running on the same database, which is clearly wrong.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:32 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
@ 2026-05-01 07:30 ` 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: 'Alvaro Herrera' @ 2026-05-01 07:30 UTC (permalink / raw)
To: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
On 2026-Apr-07, Hayato Kuroda (Fujitsu) wrote:
> 01.
> ```
> --- a/src/backend/access/index/genam.c
> +++ b/src/backend/access/index/genam.c
> @@ -394,6 +394,14 @@ systable_beginscan(Relation heapRelation,
> SysScanDesc sysscan;
> Relation irel;
>
> + /*
> + * If this backend promised that it won't access shared catalogs during
> + * logical decoding, this it the right place to verify.
> + */
> + Assert(!HistoricSnapshotActive() ||
> + accessSharedCatalogsInDecoding ||
> + !heapRelation->rd_rel->relisshared);
> ```
>
> Not sure it's OK to use Assert(). elog(ERROR) might be better if we want to really
> avoid the case.
How about the attached?
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 19:43 ` Robert Treat <rob@xzilla.net>
2026-04-08 07:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
4 siblings, 1 reply; 416+ messages in thread
From: Robert Treat @ 2026-04-07 19:43 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 6, 2026 at 6:22 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-06, Mihail Nikalayeu wrote:
<snip>
>
> Anyway, here's the three missing parts. I have not yet edited the
> deadlock-checker one to protect autovacuum from processing tables under
> repack.
>
I have this lingering bit of paranoia that users could end up in a
situation with a large / long running repack that goes past failsafe
age which prevents the simpler fix of failsafe autovacuum from
running. While the repack finishing would resolve this issue, we can't
know ahead of time that the repack would finish in time, and
statistically speaking, failsafe autovacuum should generally run much
quicker than any repack could. I'm not sure if that means we should
let failsafe vacuum cancel repacks (that seems a bit extreme), but
maybe we want to help $operator to think about this decision, except
if we don't allow autovacuum to wait and we don't allow it to respawn,
I wonder if the end user will ever realize they are in this position.
Granted, there doesn't seem like a clean fix for this...
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 19:43 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2026-04-08 07:31 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-08 07:31 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Robert Treat <rob@xzilla.net> wrote:
> On Mon, Apr 6, 2026 at 6:22 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > On 2026-Apr-06, Mihail Nikalayeu wrote:
> <snip>
> >
> > Anyway, here's the three missing parts. I have not yet edited the
> > deadlock-checker one to protect autovacuum from processing tables under
> > repack.
> >
>
> I have this lingering bit of paranoia that users could end up in a
> situation with a large / long running repack that goes past failsafe
> age which prevents the simpler fix of failsafe autovacuum from
> running. While the repack finishing would resolve this issue, we can't
> know ahead of time that the repack would finish in time, and
> statistically speaking, failsafe autovacuum should generally run much
> quicker than any repack could. I'm not sure if that means we should
> let failsafe vacuum cancel repacks (that seems a bit extreme), but
> maybe we want to help $operator to think about this decision, except
> if we don't allow autovacuum to wait and we don't allow it to respawn,
> I wonder if the end user will ever realize they are in this position.
> Granted, there doesn't seem like a clean fix for this...
If REPACK is not going to finish in time, I think it makes little difference
whether VACUUM is allowed to wait or not: even if it waits, it will start just
too late. One reason to avoid waiting might be to allow autovacuum to work on
other tables in between.
I agree that the DBA should have some guidance to asses whether REPACK or
(failsafe) VACUUM is the appropriate action. While failsafe VACUUM is clearly
a means to avoid XID wraparound, I tend to consider REPACK primarily a command
to remove table bloat. Or is there a situation where REPACK is better even to
avoid the wraparound?
Technically, the deadlock can be avoided by not running DDLs on the table
while REPACK is running. I'm just thinking if, by mentioning this in the
REPACK documentation, we'd admit that the REPACK (CONCURRENTLY) feature is
actually incomplete. On the other hand, if we don't mention the risk of
deadlock, it's a similar situation to not mentioning it for commands like
ALTER TABLE: if ALTER TABLE performs table rewrite, deadlock can also result
in a significant amount of wasted resources. (Of course, it's not the same if
the purpose of REPACK is considered substitute for failsafe VACUUM, but I'm
not sure about that.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-07 20:24 ` Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
4 siblings, 1 reply; 416+ messages in thread
From: Andres Freund @ 2026-04-07 20:24 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-07 00:22:32 +0200, Alvaro Herrera wrote:
> From 4303eea0a72408183f9f5afcf8d2801df20f8ffe Mon Sep 17 00:00:00 2001
> From: Antonin Houska <ah@cybertec.at>
> Date: Wed, 1 Apr 2026 17:35:47 +0200
> Subject: [PATCH v56 3/3] Error out any process that would block at REPACK
>
> Any process waiting on REPACK to release its lock would actually cause
> it to deadlock when it tries to upgrade its lock to AEL, losing all work
> done to that point. We avoid this by teaching the deadlock detector to
> raise an error when this condition is detected.
I'm rather doubtful that that is ok. Won't this make it basically unsafe to
use repack in way way too many situations? If sessions start erroring out,
rather than wait, because they briefly lock a relation or such it'll imo make
this rather unsafe in production.
ISTM that the proper fix here might be to allow repack to do a lock upgrade
that jumps to the front of the lock's wait queue. The lock upgrade is the
reason for the deadlock, right? And the reason it will often cause a deadlock
is that the repack will be at the tail, rather than the front of the wait
queue. So let's fix that.
Jumping the queue won't make the lock upgrade immediately, of course, but I
think it would fix the issue of the repack, rather than some other process,
getting cancelled?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-08 08:35 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-04-08 08:35 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, Apr 8, 2026 at 1:54 AM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2026-04-07 00:22:32 +0200, Alvaro Herrera wrote:
> > From 4303eea0a72408183f9f5afcf8d2801df20f8ffe Mon Sep 17 00:00:00 2001
> > From: Antonin Houska <ah@cybertec.at>
> > Date: Wed, 1 Apr 2026 17:35:47 +0200
> > Subject: [PATCH v56 3/3] Error out any process that would block at REPACK
> >
> > Any process waiting on REPACK to release its lock would actually cause
> > it to deadlock when it tries to upgrade its lock to AEL, losing all work
> > done to that point. We avoid this by teaching the deadlock detector to
> > raise an error when this condition is detected.
>
> I'm rather doubtful that that is ok.
>
Another possible idea is that after copying table_data to the new
table, we mark the old table as in_use_by_repack and release the
ShareUpdateExclusiveLock on the old table. Then the function
CheckTableNotInUse() should be updated to give an ERROR if the table
is marked as in_use_by_repack. Now, acquiring AEL by repack
(concurrently) should be safe because all concurrent DDLs should be
errored out due to flag in_use_by_repack. Can this address the problem
we are worried about the lock upgrade?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-08 17:22 ` Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Andres Freund @ 2026-04-08 17:22 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-08 14:05:49 +0530, Amit Kapila wrote:
> On Wed, Apr 8, 2026 at 1:54 AM Andres Freund <andres@anarazel.de> wrote:
> > On 2026-04-07 00:22:32 +0200, Alvaro Herrera wrote:
> > > From 4303eea0a72408183f9f5afcf8d2801df20f8ffe Mon Sep 17 00:00:00 2001
> > > From: Antonin Houska <ah@cybertec.at>
> > > Date: Wed, 1 Apr 2026 17:35:47 +0200
> > > Subject: [PATCH v56 3/3] Error out any process that would block at REPACK
> > >
> > > Any process waiting on REPACK to release its lock would actually cause
> > > it to deadlock when it tries to upgrade its lock to AEL, losing all work
> > > done to that point. We avoid this by teaching the deadlock detector to
> > > raise an error when this condition is detected.
> >
> > I'm rather doubtful that that is ok.
> >
>
> Another possible idea is that after copying table_data to the new
> table, we mark the old table as in_use_by_repack and release the
> ShareUpdateExclusiveLock on the old table. Then the function
> CheckTableNotInUse() should be updated to give an ERROR if the table
> is marked as in_use_by_repack. Now, acquiring AEL by repack
> (concurrently) should be safe because all concurrent DDLs should be
> errored out due to flag in_use_by_repack. Can this address the problem
> we are worried about the lock upgrade?
I don't think this is a viable path. You need to prevent any further lock
acquisitions on the relation to be able to swap it, not just conflicting DDL.
And you need to wait for all pre-existing locks to have been released. That
doesn't really get easier by what you propose.
I don't think CheckTableNotInUse() would work anyway - don't we already hold
locks by the point we call it? And even if that were not the case, there are
several paths to locking relations that don't ever go anywhere near
CheckTableNotInUse().
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-08 23:36 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 14:13 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-08 23:36 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Andres!
On Wed, Apr 8, 2026 at 7:22 PM Andres Freund <andres@anarazel.de> wrote:
> I don't think this is a viable path. You need to prevent any further lock
> acquisitions on the relation to be able to swap it, not just conflicting DDL.
AFAIU, Amit's main idea is that we currently upgrade the lock instead
of **releasing and re-acquiring** it because we fear DDL between those
actions.
DML actions are ok for us; REPACK will wait for them while getting
AEL. Later changes will be applied by REPACK backend while holding
AEL.
One more thing we may prevent from sneaking into that hole is a
VACUUM. It will not break anything, but will be huge waste of time and
resources.
> And you need to wait for all pre-existing locks to have been released. That
> doesn't really get easier by what you propose.
Do you mean locks from other sessions accessing the table? Is it done
automatically while waiting for AEL?
> I don't think CheckTableNotInUse() would work anyway - don't we already hold
> locks by the point we call it?
Yes, the DDL session already holds locks by the time
CheckTableNotInUse is called - but is that really the problem? They
will be released on error.
> And even if that were not the case, there are
> several paths to locking relations that don't ever go anywhere near
> CheckTableNotInUse().
But those aren't DDL, so they shouldn't be the problem (CREATE TRIGGER
might be - it seems to ignore CheckTableNotInUse, but perhaps it's
fine).
So, in my undeerstanding Amit's idea has two parts: "set flag and
release/re-acquire" + "use CheckTableNotInUse (or some place like
that) to check the flag and fail for DDL commands."
Reading your arguments again, they all seem valid under the assumption
that we still hold SUEL while requesting AEL.
I suspect you may have just skimmed past the 'release/re-acquire' part
- but more probably I missed something.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-09 04:54 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-04-09 04:54 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Thu, Apr 9, 2026 at 5:08 AM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
>
> On Wed, Apr 8, 2026 at 7:22 PM Andres Freund <andres@anarazel.de> wrote:
> > I don't think this is a viable path. You need to prevent any further lock
> > acquisitions on the relation to be able to swap it, not just conflicting DDL.
>
> AFAIU, Amit's main idea is that we currently upgrade the lock instead
> of **releasing and re-acquiring** it because we fear DDL between those
> actions.
> DML actions are ok for us; REPACK will wait for them while getting
> AEL. Later changes will be applied by REPACK backend while holding
> AEL.
>
Yes, this is exactly what I had in mind.
> One more thing we may prevent from sneaking into that hole is a
> VACUUM. It will not break anything, but will be huge waste of time and
> resources.
>
We can prevent other commands (if required) by checking the
rel_in_use_by_repack flag but I thought for the initial version it is
better to do what is minimally required.
> > And you need to wait for all pre-existing locks to have been released. That
> > doesn't really get easier by what you propose.
>
> Do you mean locks from other sessions accessing the table? Is it done
> automatically while waiting for AEL?
>
Right and this is already the case with the code where locks
not-conflicting with ShareUpdateExclusiveLock could be present before
we try to upgrade the lock. The point was we will not let DDL execute
on the table after the new flag (rel_in_use_by_repack) is set and we
released the ShareUpdateExclusiveLock.
> > I don't think CheckTableNotInUse() would work anyway - don't we already hold
> > locks by the point we call it?
>
> Yes, the DDL session already holds locks by the time
> CheckTableNotInUse is called - but is that really the problem? They
> will be released on error.
>
Yes, that is the key to solve this problem. Let me take an example to
explain this a bit more. Right now, the problem can happen in the
following kind of sequence.
Session-1:
REPACK (CONCURRENTLY) foo;
-- now say after the above command has acquired
ShareUpdateExclusiveLock and is doing the work of copying the table,
Session-2 did following actions.
Session-2:
Select * from foo; -- this is allowed
ALTER TABLE foo ADD COLUMN c2; -- this will blocked as Session-1
already has acquired ShareUpdateExclusiveLock
Session-1:
-- continues and tries to upgrade the lock to AEL. This leads to
deadlock ERROR in the current session doing REPACK (CONCURRENTLY).
Now, with the solution I proposed, because we will release
ShareUpdateExclusiveLock after setting a flag like
rel_in_use_by_repack, the ALTER TABLE in session-2 will succeed but
will error_out by CheckTableNotInUse(or a similar function that checks
rel_in_use_by_repack). Both with and without this solution, acquiring
AEL by REPACK (CONCURRENTLY) needs to wait concurrent locks like the
one for SELECT in above example that could have been acquired during
the time REPACKing had ShareUpdateExclusiveLock.
> > And even if that were not the case, there are
> > several paths to locking relations that don't ever go anywhere near
> > CheckTableNotInUse().
>
> But those aren't DDL, so they shouldn't be the problem (CREATE TRIGGER
> might be - it seems to ignore CheckTableNotInUse, but perhaps it's
> fine).
>
> So, in my undeerstanding Amit's idea has two parts: "set flag and
> release/re-acquire" + "use CheckTableNotInUse (or some place like
> that) to check the flag and fail for DDL commands."
>
True, this is the core of the idea.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-09 08:43 ` Antonin Houska <ah@cybertec.at>
2026-04-09 09:11 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-09 08:43 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Thu, Apr 9, 2026 at 5:08 AM Mihail Nikalayeu
> <mihailnikalayeu@gmail.com> wrote:
> >
> > On Wed, Apr 8, 2026 at 7:22 PM Andres Freund <andres@anarazel.de> wrote:
> > > I don't think this is a viable path. You need to prevent any further lock
> > > acquisitions on the relation to be able to swap it, not just conflicting DDL.
> >
> > AFAIU, Amit's main idea is that we currently upgrade the lock instead
> > of **releasing and re-acquiring** it because we fear DDL between those
> > actions.
> > DML actions are ok for us; REPACK will wait for them while getting
> > AEL. Later changes will be applied by REPACK backend while holding
> > AEL.
> >
>
> Yes, this is exactly what I had in mind.
>
> > One more thing we may prevent from sneaking into that hole is a
> > VACUUM. It will not break anything, but will be huge waste of time and
> > resources.
> >
>
> We can prevent other commands (if required) by checking the
> rel_in_use_by_repack flag but I thought for the initial version it is
> better to do what is minimally required.
>
> > > And you need to wait for all pre-existing locks to have been released. That
> > > doesn't really get easier by what you propose.
> >
> > Do you mean locks from other sessions accessing the table? Is it done
> > automatically while waiting for AEL?
> >
>
> Right and this is already the case with the code where locks
> not-conflicting with ShareUpdateExclusiveLock could be present before
> we try to upgrade the lock. The point was we will not let DDL execute
> on the table after the new flag (rel_in_use_by_repack) is set and we
> released the ShareUpdateExclusiveLock.
>
> > > I don't think CheckTableNotInUse() would work anyway - don't we already hold
> > > locks by the point we call it?
> >
> > Yes, the DDL session already holds locks by the time
> > CheckTableNotInUse is called - but is that really the problem? They
> > will be released on error.
> >
>
> Yes, that is the key to solve this problem. Let me take an example to
> explain this a bit more. Right now, the problem can happen in the
> following kind of sequence.
>
> Session-1:
> REPACK (CONCURRENTLY) foo;
>
> -- now say after the above command has acquired
> ShareUpdateExclusiveLock and is doing the work of copying the table,
> Session-2 did following actions.
>
> Session-2:
> Select * from foo; -- this is allowed
> ALTER TABLE foo ADD COLUMN c2; -- this will blocked as Session-1
> already has acquired ShareUpdateExclusiveLock
>
> Session-1:
> -- continues and tries to upgrade the lock to AEL. This leads to
> deadlock ERROR in the current session doing REPACK (CONCURRENTLY).
>
> Now, with the solution I proposed, because we will release
> ShareUpdateExclusiveLock after setting a flag like
> rel_in_use_by_repack, the ALTER TABLE in session-2 will succeed but
> will error_out by CheckTableNotInUse(or a similar function that checks
> rel_in_use_by_repack). Both with and without this solution, acquiring
> AEL by REPACK (CONCURRENTLY) needs to wait concurrent locks like the
> one for SELECT in above example that could have been acquired during
> the time REPACKing had ShareUpdateExclusiveLock.
>
> > > And even if that were not the case, there are
> > > several paths to locking relations that don't ever go anywhere near
> > > CheckTableNotInUse().
> >
> > But those aren't DDL, so they shouldn't be the problem (CREATE TRIGGER
> > might be - it seems to ignore CheckTableNotInUse, but perhaps it's
> > fine).
> >
> > So, in my undeerstanding Amit's idea has two parts: "set flag and
> > release/re-acquire" + "use CheckTableNotInUse (or some place like
> > that) to check the flag and fail for DDL commands."
> >
>
> True, this is the core of the idea.
This approach LGTM when it comes to concurrent DDLs. However, consider REPACK
holding ShareUpdateExclusiveLock (SUEL) and VACUUM (w/o VACOPT_SKIP_LOCKED)
waiting for the same lock. Once REPACK releases its SUEL, VACUUM gets it and
processes the table, then REPACK finally gets AccessExclusiveLock (AEL) and
finishes too. Nothing went wrong from the data consistency POV, however the
VACUUM proably wasted a lot of resources, because REPACK does "more than
VACUUM". Furthermore, while REPACK was waiting for the AEL (and the VACUUM was
running), a *lot of* DMLs could have been executed on the table, and REPACK
will have to replay those while holding AEL. The point is that REPACK should
hold AEL for as short time as possible.
What Andres proposed (AFAIU) should help to avoid this problem because
REPACK's request for AEL would get in front of the VACUUM's request for SUEL
in the queue. Of course, VACUUM would eventually run too, but - if REPACK
succeeded - it'd be much cheaper because it would (supposedly) find very
little work to do.
Anti-wraparound (failsafe) VACUUM is a bit different case [1] (i.e. it should
possibly have higher priority than REPACK), but I think this prioritization
should be implemented in other way than just letting it get in the way of
REPACK (at the time REPACK is nearly finished).
[1] https://www.postgresql.org/message-id/CABV9wwMrrP4S54jPGn5D-a8AbJm%2Be5%2BWORs6ykUBgXdc-%2B%2BNtQ%40...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-09 09:11 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 09:26 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-09 09:11 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi!
On Thu, Apr 9, 2026 at 10:43 AM Antonin Houska <ah@cybertec.at> wrote:
> This approach LGTM when it comes to concurrent DDLs. However, consider REPACK
> holding ShareUpdateExclusiveLock (SUEL) and VACUUM (w/o VACOPT_SKIP_LOCKED)
> waiting for the same lock. Once REPACK releases its SUEL, VACUUM gets it and
> processes the table, then REPACK finally gets AccessExclusiveLock (AEL) and
> finishes too.
> One more thing we may prevent from sneaking into that hole is a
> VACUUM. It will not break anything, but will be huge waste of time and
> resources.
I thought about that too, I think we may just add some kind of
CheckTableNotInUse in VACUUM after getting the SUEL.
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 09:11 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-09 09:26 ` Antonin Houska <ah@cybertec.at>
2026-04-09 14:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-09 09:26 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> On Thu, Apr 9, 2026 at 10:43 AM Antonin Houska <ah@cybertec.at> wrote:
> > This approach LGTM when it comes to concurrent DDLs. However, consider REPACK
> > holding ShareUpdateExclusiveLock (SUEL) and VACUUM (w/o VACOPT_SKIP_LOCKED)
> > waiting for the same lock. Once REPACK releases its SUEL, VACUUM gets it and
> > processes the table, then REPACK finally gets AccessExclusiveLock (AEL) and
> > finishes too.
>
> > One more thing we may prevent from sneaking into that hole is a
> > VACUUM. It will not break anything, but will be huge waste of time and
> > resources.
>
>
> I thought about that too, I think we may just add some kind of
> CheckTableNotInUse in VACUUM after getting the SUEL.
Sure, it's possible, but IMO the principal question is whether REPACK should
let VACUUM and DDLs error out, or just let them wait.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 09:11 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 09:26 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-09 14:06 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 14:26 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-09 14:06 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Thu, Apr 9, 2026 at 11:26 AM Antonin Houska <ah@cybertec.at> wrote:
> Sure, it's possible, but IMO the principal question is whether REPACK should
> let VACUUM and DDLs error out, or just let them wait.
One more idea: instead of ERROR in CheckTableNotInUse in case of
in_repack - just release the lock and retry a little later (by that
time, the repack operation will likely have acquired the AEL).
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 09:11 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 09:26 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-09 14:26 ` Andres Freund <andres@anarazel.de>
2026-04-09 14:34 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 416+ messages in thread
From: Andres Freund @ 2026-04-09 14:26 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-09 16:06:17 +0200, Mihail Nikalayeu wrote:
> On Thu, Apr 9, 2026 at 11:26 AM Antonin Houska <ah@cybertec.at> wrote:
> > Sure, it's possible, but IMO the principal question is whether REPACK should
> > let VACUUM and DDLs error out, or just let them wait.
>
> One more idea: instead of ERROR in CheckTableNotInUse in case of
> in_repack - just release the lock and retry a little later (by that
> time, the repack operation will likely have acquired the AEL).
I continue to think this approach has no chance of working. Even if it
theoretically could, there are lots of paths to locking relations that do not
go through CheckTableNotInUse(), and we're not going to just route them all
through CheckTableNotInUse().
What CheckTableNotInUse() is for is to prevent DDL from changing the structure
of the table when it is still being referred to. You can't just call
CheckTableNotInUse() from a LOCK TABLE - it would trigger wrong errors *ALL
THE TIME* because a LOCK TABLE does not need to error out just because there's
also a cursor on the table.
And it'd trigger lots of bogus errors. See the first example in
https://postgr.es/m/fpr4nsmyy3mpfrm2mijspr44dgol2cjeke5tyznb4btsznxsgx%40iifdbfe2wl63
S2 would get the lock and then error out due to the proposed check. Even
though there's no need for it.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 09:11 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 09:26 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 14:26 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-09 14:34 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-09 14:34 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-09 10:26:22 -0400, Andres Freund wrote:
> On 2026-04-09 16:06:17 +0200, Mihail Nikalayeu wrote:
> > On Thu, Apr 9, 2026 at 11:26 AM Antonin Houska <ah@cybertec.at> wrote:
> > > Sure, it's possible, but IMO the principal question is whether REPACK should
> > > let VACUUM and DDLs error out, or just let them wait.
> >
> > One more idea: instead of ERROR in CheckTableNotInUse in case of
> > in_repack - just release the lock and retry a little later (by that
> > time, the repack operation will likely have acquired the AEL).
>
> I continue to think this approach has no chance of working. Even if it
> theoretically could, there are lots of paths to locking relations that do not
> go through CheckTableNotInUse(), and we're not going to just route them all
> through CheckTableNotInUse().
>
> What CheckTableNotInUse() is for is to prevent DDL from changing the structure
> of the table when it is still being referred to. You can't just call
> CheckTableNotInUse() from a LOCK TABLE - it would trigger wrong errors *ALL
> THE TIME* because a LOCK TABLE does not need to error out just because there's
> also a cursor on the table.
>
>
> And it'd trigger lots of bogus errors. See the first example in
> https://postgr.es/m/fpr4nsmyy3mpfrm2mijspr44dgol2cjeke5tyznb4btsznxsgx%40iifdbfe2wl63
>
> S2 would get the lock and then error out due to the proposed check. Even
> though there's no need for it.
Before you protest that you could just let the non-DDL operation have its
lock, sure, but that's an utterly terrible idea, because it will lead to more
and more work accumulating that then has to happen when the AEL is actually
held.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-09 14:20 ` Andres Freund <andres@anarazel.de>
2026-04-10 16:14 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-09 14:20 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-09 10:43:14 +0200, Antonin Houska wrote:
> What Andres proposed (AFAIU) should help to avoid this problem because
> REPACK's request for AEL would get in front of the VACUUM's request for SUEL
> in the queue.
Note that that already happens today.
This works today (without the error triggering patch):
S1: REPACK starts
S2: LOCK TABLE / VACUUM / ... starts waiting
S1: REPACK tries to get AEL
S1: REPACK's lock requests get reordered in the wait queue to be before S2 and
just gets the lock
S1: REPACK finishes
S2: lock acquisition completes.
That's because we do already have this "jumping the wait queue" logic, which I
had forgotten about.
What does *not* work is this:
S1: REPACK starts
S2: BEGIN; SELECT 1 FROM table LIMIT 1;
S2: LOCK TABLE / VACUUM / ... starts waiting
S1: REPACK tries to get AEL
S1: lock is not granted, can't be reordered to be before S2, because S2 holds
conflicting lock, deadlock detector triggers
S2: lock acquisition completes
But with my proposal to properly teach the deadlock detector about assuming
there's a wait edge for the eventual lock upgrade by S1, the first example
would still work, because the lock upgrade would not be considered a hard
cycle, and the second example would have S2 error out.
> Anti-wraparound (failsafe) VACUUM is a bit different case [1] (i.e. it should
> possibly have higher priority than REPACK), but I think this prioritization
> should be implemented in other way than just letting it get in the way of
> REPACK (at the time REPACK is nearly finished).
Yea, it makes no sense to interrupt the long running repack, given that the
new relation will have much less stuff for vacuum to do.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-10 16:14 ` Robert Treat <rob@xzilla.net>
2026-04-10 18:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-12 14:02 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
1 sibling, 2 replies; 416+ messages in thread
From: Robert Treat @ 2026-04-10 16:14 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
On Thu, Apr 9, 2026 at 10:20 AM Andres Freund <andres@anarazel.de> wrote:
> On 2026-04-09 10:43:14 +0200, Antonin Houska wrote:
> > What Andres proposed (AFAIU) should help to avoid this problem because
> > REPACK's request for AEL would get in front of the VACUUM's request for SUEL
> > in the queue.
>
> Note that that already happens today.
>
> This works today (without the error triggering patch):
>
> S1: REPACK starts
> S2: LOCK TABLE / VACUUM / ... starts waiting
> S1: REPACK tries to get AEL
> S1: REPACK's lock requests get reordered in the wait queue to be before S2 and
> just gets the lock
> S1: REPACK finishes
> S2: lock acquisition completes.
>
> That's because we do already have this "jumping the wait queue" logic, which I
> had forgotten about.
>
You know, I was wondering how this wasn't already a problem for
pg_repack/pg_squeeze, and I guess this explains it :-P
>
> What does *not* work is this:
>
> S1: REPACK starts
> S2: BEGIN; SELECT 1 FROM table LIMIT 1;
> S2: LOCK TABLE / VACUUM / ... starts waiting
> S1: REPACK tries to get AEL
> S1: lock is not granted, can't be reordered to be before S2, because S2 holds
> conflicting lock, deadlock detector triggers
> S2: lock acquisition completes
>
> But with my proposal to properly teach the deadlock detector about assuming
> there's a wait edge for the eventual lock upgrade by S1, the first example
> would still work, because the lock upgrade would not be considered a hard
> cycle, and the second example would have S2 error out.
>
In the above S2 will error out if you try to run a VACUUM, but the
point still stands that calling an explicit LOCK or similar could lead
to this issue. In the current repack world, we document the need for
lock escalation at the end of the repacking and caution that doing
things like DDL or explicit LOCKing could cause trouble, so don't do
that. What you're proposing above would be an improvement though,
IMHO.
>
> > Anti-wraparound (failsafe) VACUUM is a bit different case [1] (i.e. it should
> > possibly have higher priority than REPACK), but I think this prioritization
> > should be implemented in other way than just letting it get in the way of
> > REPACK (at the time REPACK is nearly finished).
>
> Yea, it makes no sense to interrupt the long running repack, given that the
> new relation will have much less stuff for vacuum to do.
>
We might be talking about 2 different scenarios. In the case where we
are at the point of lock escalation, you would probably want the
repack to get priority over a waiting vacuum, even a failsafe vacuum.
But outside of that scenario, we can't know that the repack is the
better option (and statistically it probably isn't) since a repack
that is actively copying rows might still need to rebuild some large
number of indexes (or just some really expensive index) which could
take significantly longer than a failsafe vacuum would need to ensure
wraparound avoidance. I don't think we'd go as far as saying the
failsafe vacuum should cancel the repack, but I think ideally we'd
like it to not be canceled either, since that would increase
likelihood for dba/monitoring to pick up on the situation, and in the
case that REPACK fails for some reason, the failsafe vacuum could
immediately start working without having to go through any additional
hoops.
Robert Treat
https://xzilla.net
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-10 16:14 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2026-04-10 18:56 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-10 18:56 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Robert Treat <rob@xzilla.net> wrote:
> On Thu, Apr 9, 2026 at 10:20 AM Andres Freund <andres@anarazel.de> wrote:
> > On 2026-04-09 10:43:14 +0200, Antonin Houska wrote:
> > > Anti-wraparound (failsafe) VACUUM is a bit different case [1] (i.e. it should
> > > possibly have higher priority than REPACK), but I think this prioritization
> > > should be implemented in other way than just letting it get in the way of
> > > REPACK (at the time REPACK is nearly finished).
> >
> > Yea, it makes no sense to interrupt the long running repack, given that the
> > new relation will have much less stuff for vacuum to do.
> >
>
> We might be talking about 2 different scenarios. In the case where we
> are at the point of lock escalation, you would probably want the
> repack to get priority over a waiting vacuum, even a failsafe vacuum.
> But outside of that scenario, we can't know that the repack is the
> better option (and statistically it probably isn't) since a repack
> that is actively copying rows might still need to rebuild some large
> number of indexes (or just some really expensive index) which could
> take significantly longer than a failsafe vacuum would need to ensure
> wraparound avoidance. I don't think we'd go as far as saying the
> failsafe vacuum should cancel the repack, but I think ideally we'd
> like it to not be canceled either, since that would increase
> likelihood for dba/monitoring to pick up on the situation, and in the
> case that REPACK fails for some reason, the failsafe vacuum could
> immediately start working without having to go through any additional
> hoops.
What about just teaching the anti-wraparound VACUUM to print out a WARNING if
it could not lock the table in "reasonable" time? The DBA would then have to
consider if the blocking command should be cancelled, whether it's REPACK or
something else.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-10 16:14 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
@ 2026-04-12 14:02 ` Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-12 14:02 UTC (permalink / raw)
To: Robert Treat <rob@xzilla.net>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-04-10 12:14:59 -0400, Robert Treat wrote:
> On Thu, Apr 9, 2026 at 10:20 AM Andres Freund <andres@anarazel.de> wrote:
> > > Anti-wraparound (failsafe) VACUUM is a bit different case [1] (i.e. it should
> > > possibly have higher priority than REPACK), but I think this prioritization
> > > should be implemented in other way than just letting it get in the way of
> > > REPACK (at the time REPACK is nearly finished).
> >
> > Yea, it makes no sense to interrupt the long running repack, given that the
> > new relation will have much less stuff for vacuum to do.
> >
>
> We might be talking about 2 different scenarios. In the case where we
> are at the point of lock escalation, you would probably want the
> repack to get priority over a waiting vacuum, even a failsafe vacuum.
> But outside of that scenario, we can't know that the repack is the
> better option (and statistically it probably isn't) since a repack
> that is actively copying rows might still need to rebuild some large
> number of indexes (or just some really expensive index) which could
> take significantly longer than a failsafe vacuum would need to ensure
> wraparound avoidance.
I don't really understand why you consider REPACK CONCURRENTLY to be so
different from other existing long-running operations?
> I don't think we'd go as far as saying the failsafe vacuum should cancel the
> repack, but I think ideally we'd like it to not be canceled either
I don't think what I suggested could lead to (auto-)vacuum getting cancelled?
Antonin's earlier patch could, but as long as you only do cancellations if
there's an actual deadlock cycle, VACUUM should not get cancelled, since it
won't already hold a lower level lock?
> , since that would increase likelihood for dba/monitoring to pick up on the
> situation, and in the case that REPACK fails for some reason, the failsafe
> vacuum could immediately start working without having to go through any
> additional hoops.
Not sure I buy this (leaving aside the premise doesn't hold, I think). If you
have monitoring for either long running tasks or tables not getting
autovacuumed, it'd pick up on that regardless of whether autovacuum is getting
cancelled or not. What realistic monitoring would pick up on autovacuum just
being stuck waiting for a lock for long, but would not pick up on repack
running forever or age(datfrozenxid) getting a bit long in the tooth?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-12 13:31 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-12 13:31 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Thu, Apr 9, 2026 at 4:20 PM Andres Freund <andres@anarazel.de> wrote:
> But with my proposal to properly teach the deadlock detector about
assuming
> there's a wait edge for the eventual lock upgrade by S1, the first example
> would still work, because the lock upgrade would not be considered a hard
> cycle, and the second example would have S2 error out.
Attached patch contains some (maybe naive) POC for similar approach.
It adds a 'deadlock_protected' flag, which changes how the deadlock
detector cancels backends.
Instead of cancelling the backend entered the deadlock detector - it
cancel some another (nearest hard edge) until it is possible to get the
lock (either by
reordering or directly).
Also, I added test cases with the scenarios you mentioned into the repack
spec - to ensure repack is still working while other backend are cancelled.
> > Anti-wraparound (failsafe) VACUUM is a bit different case [1] (i.e. it
should
> > possibly have higher priority than REPACK), but I think this
prioritization
> > should be implemented in other way than just letting it get in the way
of
> > REPACK (at the time REPACK is nearly finished).
>
> Yea, it makes no sense to interrupt the long running repack, given that
the
> new relation will have much less stuff for vacuum to do.
For now I decided to keep anti-wraparound unaffected because the
feature is may be used not only for repack but also for other potential
scenarios.
But yes, maybe it's worth canceling antiwraparound in the repack case.
Also, checking proc flags to detect antuwraparound requires
LWLockConditionalAcquire(ProcArrayLock) - something I don't like in
deadlock detector.
Best regards,
Mikhail.
Attachments:
[application/x-patch] nocfbot-v1-0001-Prefer-canceling-blockers-for-deadlock-protected-.patch (25.7K, ../../CADzfLwURKVNQ++Dpi7bjoGfj-8pchDQEVex3eWBx0NCYn6TbDQ@mail.gmail.com/3-nocfbot-v1-0001-Prefer-canceling-blockers-for-deadlock-protected-.patch)
download | inline diff:
From 72414a340b39bc814c4a77b5a6250bae4f85e079 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 12 Apr 2026 14:24:02 +0200
Subject: [PATCH v1] Prefer canceling blockers for deadlock-protected repack
waits
Teach the lock manager to treat selected waits as deadlock-protected, so
REPACK CONCURRENTLY can preserve the work done before waiting for the
final AccessExclusiveLock set.
When deadlock_protected is set and a hard deadlock is found, cancel a
blocking lock wait and let the protected backend continue waiting
instead of making it the deadlock victim. Keep anti-wraparound
autovacuum workers ineligible as cancellation targets, using a
conditional ProcArrayLock check for the wraparound flag so deadlock
detection does not block on LWLocks.
Set this protection around repack's final lock acquisition and clear it
through the normal wait/transaction cleanup paths.
---
src/backend/commands/repack.c | 8 ++
src/backend/storage/ipc/procarray.c | 3 +
src/backend/storage/lmgr/deadlock.c | 103 +++++++++++++-
src/backend/storage/lmgr/lock.c | 18 ++-
src/backend/storage/lmgr/proc.c | 63 +++++++--
src/include/storage/lock.h | 6 +-
src/include/storage/proc.h | 9 +-
.../injection_points/expected/repack.out | 130 +++++++++++++++++-
.../injection_points/specs/repack.spec | 80 +++++++++++
9 files changed, 399 insertions(+), 21 deletions(-)
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 58e3867246f..cee2aa49c06 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -3055,7 +3055,15 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
/*
* Acquire AccessExclusiveLock on the table, its TOAST relation (if there
* is one), all its indexes, so that we can swap the files.
+ *
+ * Set deadlock_protected so that if a deadlock occurs while we are
+ * waiting for these locks, the deadlock detector cancels the blocking
+ * proc's lock wait instead of ours. At this point we have done the
+ * expensive data copy and index rebuild - aborting now would waste all
+ * that work.
*/
+ deadlock_protected = true;
+
LockRelationOid(old_table_oid, AccessExclusiveLock);
/*
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 9299bcebbda..1dbbae54017 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -714,6 +714,9 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
LWLockRelease(ProcArrayLock);
}
}
+
+ /* Clear backend-local deadlock protection flag at end of transaction. */
+ deadlock_protected = false;
}
/*
diff --git a/src/backend/storage/lmgr/deadlock.c b/src/backend/storage/lmgr/deadlock.c
index b8962d875b6..7b53e2fb19a 100644
--- a/src/backend/storage/lmgr/deadlock.c
+++ b/src/backend/storage/lmgr/deadlock.c
@@ -125,6 +125,15 @@ static int maxPossibleConstraints;
static DEADLOCK_INFO *deadlockDetails;
static int nDeadlockDetails;
+/*
+ * Per-edge parallel array to deadlockDetails[]: the waiting proc whose wait
+ * forms the edge at the corresponding index. Unlike deadlockDetails[], these
+ * pointers reference live PGPROCs and are only valid while all partition
+ * locks are held (i.e., during DeadLockCheck). Used by deadlock_protected
+ * resolution in DeadLockCheck to pick a cancellation victim.
+ */
+static PGPROC **deadlockProcs;
+
/* PGPROC pointer of any blocking autovacuum worker found */
static PGPROC *blocking_autovacuum_proc = NULL;
@@ -148,11 +157,12 @@ InitDeadLockChecking(void)
oldcxt = MemoryContextSwitchTo(TopMemoryContext);
/*
- * FindLockCycle needs at most MaxBackends entries in visitedProcs[] and
- * deadlockDetails[].
+ * FindLockCycle needs at most MaxBackends entries in visitedProcs[],
+ * deadlockDetails[], and deadlockProcs[].
*/
visitedProcs = (PGPROC **) palloc(MaxBackends * sizeof(PGPROC *));
deadlockDetails = (DEADLOCK_INFO *) palloc(MaxBackends * sizeof(DEADLOCK_INFO));
+ deadlockProcs = (PGPROC **) palloc(MaxBackends * sizeof(PGPROC *));
/*
* TopoSort needs to consider at most MaxBackends wait-queue entries, and
@@ -219,6 +229,9 @@ InitDeadLockChecking(void)
DeadLockState
DeadLockCheck(PGPROC *proc)
{
+ bool victims_canceled = false;
+
+retry:
/* Initialize to "no constraints" */
nCurConstraints = 0;
nPossibleConstraints = 0;
@@ -242,6 +255,76 @@ DeadLockCheck(PGPROC *proc)
if (!FindLockCycle(proc, possibleConstraints, &nSoftEdges))
elog(FATAL, "deadlock seems to have disappeared");
+ /*
+ * If deadlock_protected is set, try to resolve the deadlock by
+ * canceling a blocking proc's lock wait, then restarting the whole
+ * deadlock check. We re-run DeadLockCheckRecurse (not just
+ * FindLockCycle) after each cancellation so that any remaining cycle
+ * also gets a chance to be resolved by the ordinary soft-deadlock
+ * queue-reordering path.
+ *
+ * The victim is chosen by walking deadlockProcs[] which FindLockCycle
+ * just populated with one live PGPROC pointer per edge in the cycle.
+ * Index 0 is always MyProc (we started the search from it), so we
+ * start at index 1, which is the direct blocker of MyProc.
+ *
+ * We skip any proc running an anti-wraparound autovacuum: canceling that
+ * would risk XID wraparound, which is strictly worse than losing the
+ * work the deadlock_protected caller is trying to preserve. XXX: is it true?
+ *
+ * If no acceptable victim exists, we fall through to DS_HARD_DEADLOCK
+ * and the protected proc cancels itself as usual.
+ */
+ if (deadlock_protected)
+ {
+ PGPROC *victim = NULL;
+
+ for (int i = 1; i < nDeadlockDetails; i++)
+ {
+ PGPROC *w = deadlockProcs[i];
+
+ if (w == MyProc)
+ continue;
+
+ /*
+ * PROC_IS_AUTOVACUUM is safe to check locklessly because it is
+ * set at process start and never reset. For autovac workers,
+ * avoid waiting for ProcArrayLock here; if we can acquire it
+ * conditionally, check the mirrored wraparound flag to avoid
+ * canceling an emergency vacuum. If not, skip this candidate
+ * conservatively and keep looking for another victim.
+ */
+ if (w->statusFlags & PROC_IS_AUTOVACUUM)
+ {
+ uint8 statusFlags;
+
+ if (!LWLockConditionalAcquire(ProcArrayLock, LW_SHARED))
+ continue;
+
+ statusFlags = ProcGlobal->statusFlags[w->pgxactoff];
+ LWLockRelease(ProcArrayLock);
+
+ if (statusFlags & PROC_VACUUM_FOR_WRAPAROUND)
+ continue;
+ }
+
+ Assert(w->waitLock != NULL);
+ Assert(!dlist_node_is_detached(&w->waitLink));
+ victim = w;
+ break;
+ }
+
+ if (victim != NULL)
+ {
+ RemoveFromWaitQueue(victim,
+ LockTagHashCode(&(victim->waitLock->tag)),
+ PROC_WAIT_STATUS_DEADLOCK_CANCEL);
+ SetLatch(&victim->procLatch);
+ victims_canceled = true;
+ goto retry;
+ }
+ }
+
return DS_HARD_DEADLOCK; /* cannot find a non-deadlocked state */
}
@@ -272,8 +355,16 @@ DeadLockCheck(PGPROC *proc)
ProcLockWakeup(GetLocksMethodTable(lock), lock);
}
- /* Return code tells caller if we had to escape a deadlock or not */
- if (nWaitOrders > 0)
+ /*
+ * Return code tells caller if we had to escape a deadlock or not.
+ *
+ * If we canceled any waiter because deadlock_protected was set, report
+ * DS_HARD_DEADLOCK_CANCELED regardless of whether the remainder was
+ * resolved by queue rearrangement or needed nothing further.
+ */
+ if (victims_canceled)
+ return DS_HARD_DEADLOCK_CANCELED;
+ else if (nWaitOrders > 0)
return DS_SOFT_DEADLOCK;
else if (blocking_autovacuum_proc != NULL)
return DS_BLOCKED_BY_AUTOVACUUM;
@@ -590,6 +681,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ deadlockProcs[depth] = checkProc;
return true;
}
@@ -679,6 +771,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ deadlockProcs[depth] = checkProc;
/*
* Add this edge to the list of soft edges in the cycle
@@ -753,6 +846,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ deadlockProcs[depth] = checkProc;
/*
* Add this edge to the list of soft edges in the cycle
@@ -1159,4 +1253,5 @@ RememberSimpleDeadLock(PGPROC *proc1,
info->lockmode = proc2->waitLockMode;
info->pid = proc2->pid;
nDeadlockDetails = 2;
+ deadlockProcs[0] = deadlockProcs[1] = NULL;
}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c221fe96889..1a268f05965 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -1245,6 +1245,14 @@ LockAcquireExtended(const LOCKTAG *locktag,
DeadLockReport();
/* DeadLockReport() will not return */
}
+ else if (waitResult == PROC_WAIT_STATUS_DEADLOCK_CANCEL)
+ {
+ Assert(!dontWait);
+ ereport(ERROR,
+ (errcode(ERRCODE_T_R_DEADLOCK_DETECTED),
+ errmsg("deadlock detected"),
+ errdetail("This backend's lock wait was canceled to resolve a deadlock with a protected process.")));
+ }
}
else
LWLockRelease(partitionLock);
@@ -2043,8 +2051,8 @@ waitonlock_error_callback(void *arg)
/*
* Remove a proc from the wait-queue it is on (caller must know it is on one).
- * This is only used when the proc has failed to get the lock, so we set its
- * waitStatus to PROC_WAIT_STATUS_ERROR.
+ * This is only used when the proc has failed to get the lock, so caller must
+ * specify the failure waitStatus to store.
*
* Appropriate partition lock must be held by caller. Also, caller is
* responsible for signaling the proc if needed.
@@ -2052,7 +2060,7 @@ waitonlock_error_callback(void *arg)
* NB: this does not clean up any locallock object that may exist for the lock.
*/
void
-RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode)
+RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode, ProcWaitStatus waitStatus)
{
LOCK *waitLock = proc->waitLock;
PROCLOCK *proclock = proc->waitProcLock;
@@ -2065,6 +2073,8 @@ RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode)
Assert(waitLock);
Assert(!dclist_is_empty(&waitLock->waitProcs));
Assert(0 < lockmethodid && lockmethodid < lengthof(LockMethods));
+ Assert(waitStatus == PROC_WAIT_STATUS_ERROR ||
+ waitStatus == PROC_WAIT_STATUS_DEADLOCK_CANCEL);
/* Remove proc from lock's wait queue */
dclist_delete_from_thoroughly(&waitLock->waitProcs, &proc->waitLink);
@@ -2082,7 +2092,7 @@ RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode)
/* Clean up the proc's own state, and pass it the ok/fail signal */
proc->waitLock = NULL;
proc->waitProcLock = NULL;
- proc->waitStatus = PROC_WAIT_STATUS_ERROR;
+ proc->waitStatus = waitStatus;
/*
* Delete the proclock immediately if it represents no already-held locks.
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 1ac25068d62..8afc247bebe 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -92,6 +92,9 @@ static size_t FastPathLockArrayShmemSize;
/* Is a deadlock check pending? */
static volatile sig_atomic_t got_deadlock_timeout;
+/* See proc.h */
+bool deadlock_protected = false;
+
static void RemoveProcFromArray(int code, Datum arg);
static void ProcKill(int code, Datum arg);
static void AuxiliaryProcKill(int code, Datum arg);
@@ -825,6 +828,8 @@ LockErrorCleanup(void)
AbortStrongLockAcquire();
+ deadlock_protected = false;
+
/* Nothing to do if we weren't waiting for a lock */
lockAwaited = GetAwaitedLock();
if (lockAwaited == NULL)
@@ -854,7 +859,7 @@ LockErrorCleanup(void)
if (!dlist_node_is_detached(&MyProc->waitLink))
{
/* We could not have been granted the lock yet */
- RemoveFromWaitQueue(MyProc, lockAwaited->hashcode);
+ RemoveFromWaitQueue(MyProc, lockAwaited->hashcode, PROC_WAIT_STATUS_ERROR);
}
else
{
@@ -1266,8 +1271,16 @@ JoinWaitQueue(LOCALLOCK *locallock, LockMethod lockMethodTable, bool dontWait)
/*
* If we detected deadlock, give up without waiting. This must agree with
* CheckDeadLock's recovery code.
+ *
+ * However, if deadlock_protected is set, skip the early exit and proceed
+ * to ProcSleep so the full deadlock detector can handle it (by making a
+ * blocking proc the victim instead of us).
+ *
+ * XXX: when we bypass the early-exit here, we lose the ability to react to
+ * the deadlock immediately: ProcSleep will only run the full detector
+ * after deadlock_timeout elapses. Should we fix that?
*/
- if (early_deadlock)
+ if (early_deadlock && !deadlock_protected)
return PROC_WAIT_STATUS_ERROR;
/*
@@ -1308,8 +1321,10 @@ JoinWaitQueue(LOCALLOCK *locallock, LockMethod lockMethodTable, bool dontWait)
*
* Result is one of the following:
*
- * PROC_WAIT_STATUS_OK - lock was granted
- * PROC_WAIT_STATUS_ERROR - a deadlock was detected
+ * PROC_WAIT_STATUS_OK - lock was granted
+ * PROC_WAIT_STATUS_ERROR - a deadlock was detected
+ * PROC_WAIT_STATUS_DEADLOCK_CANCEL - this backend was chosen as the
+ * victim of a protected deadlock
*/
ProcWaitStatus
ProcSleep(LOCALLOCK *locallock)
@@ -1345,8 +1360,8 @@ ProcSleep(LOCALLOCK *locallock)
/*
* Set timer so we can wake up after awhile and check for a deadlock. If a
* deadlock is detected, the handler sets MyProc->waitStatus =
- * PROC_WAIT_STATUS_ERROR, allowing us to know that we must report failure
- * rather than success.
+ * one of the failure wait statuses, allowing us to know that we must
+ * report failure rather than success.
*
* By delaying the check until we've waited for a bit, we can avoid
* running the rather expensive deadlock-check code in most cases.
@@ -1560,6 +1575,17 @@ ProcSleep(LOCALLOCK *locallock)
allow_autovacuum_cancel = false;
}
+ /*
+ * If the deadlock detector resolved a hard deadlock by canceling a
+ * blocking proc's lock wait (because we have deadlock_protected set),
+ * restart the deadlock timeout and continue waiting for the lock.
+ */
+ if (deadlock_state == DS_HARD_DEADLOCK_CANCELED)
+ {
+ got_deadlock_timeout = false;
+ enable_timeout_after(DEADLOCK_TIMEOUT, DeadlockTimeout);
+ }
+
/*
* If awoken after the deadlock check interrupt has run, increment the
* lock statistics counters and if log_lock_waits is on, then report
@@ -1627,6 +1653,13 @@ ProcSleep(LOCALLOCK *locallock)
"Processes holding the lock: %s. Wait queue: %s.",
lockHoldersNum, lock_holders_sbuf.data, lock_waiters_sbuf.data))));
}
+ else if (deadlock_state == DS_HARD_DEADLOCK_CANCELED)
+ ereport(LOG,
+ (errmsg("process %d resolved deadlock for %s on %s by canceling a blocking lock wait after %ld.%03d ms",
+ MyProcPid, modename, buf.data, msecs, usecs),
+ (errdetail_log_plural("Process holding the lock: %s. Wait queue: %s.",
+ "Processes holding the lock: %s. Wait queue: %s.",
+ lockHoldersNum, lock_holders_sbuf.data, lock_waiters_sbuf.data))));
if (myWaitStatus == PROC_WAIT_STATUS_WAITING)
{
@@ -1657,13 +1690,21 @@ ProcSleep(LOCALLOCK *locallock)
ereport(LOG,
(errmsg("process %d acquired %s on %s after %ld.%03d ms",
MyProcPid, modename, buf.data, msecs, usecs)));
+ else if (myWaitStatus == PROC_WAIT_STATUS_DEADLOCK_CANCEL)
+ ereport(LOG,
+ (errmsg("process %d was chosen as deadlock victim for %s on %s after %ld.%03d ms",
+ MyProcPid, modename, buf.data, msecs, usecs),
+ (errdetail_log_plural("Process holding the lock: %s. Wait queue: %s.",
+ "Processes holding the lock: %s. Wait queue: %s.",
+ lockHoldersNum, lock_holders_sbuf.data, lock_waiters_sbuf.data))));
else
{
Assert(myWaitStatus == PROC_WAIT_STATUS_ERROR);
/*
- * Currently, the deadlock checker always kicks its own
- * process, which means that we'll only see
+ * A plain PROC_WAIT_STATUS_ERROR means the deadlock
+ * checker kicked our own process, which means that we'll
+ * only see
* PROC_WAIT_STATUS_ERROR when deadlock_state ==
* DS_HARD_DEADLOCK, and there's no need to print
* redundant messages. But for completeness and
@@ -1870,14 +1911,16 @@ CheckDeadLock(void)
* Get this process out of wait state. (Note: we could do this more
* efficiently by relying on lockAwaited, but use this coding to
* preserve the flexibility to kill some other transaction than the
- * one detecting the deadlock.)
+ * one detecting the deadlock. (DS_HARD_DEADLOCK_CANCELED above
+ * already exercises that flexibility: when deadlock_protected is set,
+ * we cancel a *blocking* proc's lock wait instead of our own.)
*
* RemoveFromWaitQueue sets MyProc->waitStatus to
* PROC_WAIT_STATUS_ERROR, so ProcSleep will report an error after we
* return.
*/
Assert(MyProc->waitLock != NULL);
- RemoveFromWaitQueue(MyProc, LockTagHashCode(&(MyProc->waitLock->tag)));
+ RemoveFromWaitQueue(MyProc, LockTagHashCode(&(MyProc->waitLock->tag)), PROC_WAIT_STATUS_ERROR);
/*
* We're done here. Transaction abort caused by the error that
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ee3cb1dc203..5e13de529c5 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -29,6 +29,7 @@
/* struct PGPROC is declared in proc.h, but must forward-reference it */
typedef struct PGPROC PGPROC;
+typedef enum ProcWaitStatus ProcWaitStatus;
/* GUC variables */
extern PGDLLIMPORT int max_locks_per_xact;
@@ -342,6 +343,9 @@ typedef enum
DS_NO_DEADLOCK, /* no deadlock detected */
DS_SOFT_DEADLOCK, /* deadlock avoided by queue rearrangement */
DS_HARD_DEADLOCK, /* deadlock, no way out but ERROR */
+ DS_HARD_DEADLOCK_CANCELED, /* deadlock resolved by canceling a blocking
+ * proc's lock wait (when
+ * deadlock_protected is set) */
DS_BLOCKED_BY_AUTOVACUUM, /* no deadlock; queue blocked by autovacuum
* worker */
} DeadLockState;
@@ -418,7 +422,7 @@ extern void GrantAwaitedLock(void);
extern LOCALLOCK *GetAwaitedLock(void);
extern void ResetAwaitedLock(void);
-extern void RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode);
+extern void RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode, ProcWaitStatus waitStatus);
extern LockData *GetLockStatusData(void);
extern BlockedProcsData *GetBlockerStatusData(int blocked_pid);
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 3e1d1fad5f9..ef02639834e 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -146,11 +146,12 @@ extern PGDLLIMPORT int FastPathLockGroupsPerBackend;
#define DELAY_CHKPT_COMPLETE (1<<1)
#define DELAY_CHKPT_IN_COMMIT (DELAY_CHKPT_START | 1<<2)
-typedef enum
+typedef enum ProcWaitStatus
{
PROC_WAIT_STATUS_OK,
PROC_WAIT_STATUS_WAITING,
PROC_WAIT_STATUS_ERROR,
+ PROC_WAIT_STATUS_DEADLOCK_CANCEL,
} ProcWaitStatus;
/*
@@ -543,6 +544,12 @@ extern PGDLLIMPORT int TransactionTimeout;
extern PGDLLIMPORT int IdleSessionTimeout;
extern PGDLLIMPORT bool log_lock_waits;
+/*
+ * Backend-local flag: when true, the deadlock detector will make a blocking
+ * proc the victim instead of this backend.
+ */
+extern PGDLLIMPORT bool deadlock_protected;
+
#ifdef EXEC_BACKEND
extern PGDLLIMPORT PGPROC *AuxiliaryProcs;
#endif
diff --git a/src/test/modules/injection_points/expected/repack.out b/src/test/modules/injection_points/expected/repack.out
index b575e9052ee..ae706125360 100644
--- a/src/test/modules/injection_points/expected/repack.out
+++ b/src/test/modules/injection_points/expected/repack.out
@@ -1,4 +1,4 @@
-Parsed test spec with 2 sessions
+Parsed test spec with 3 sessions
starting permutation: wait_before_lock change_existing change_new change_subxact1 change_subxact2 check2 wakeup_before_lock check1
injection_points_attach
@@ -111,3 +111,131 @@ injection_points_detach
(1 row)
+
+starting permutation: wait_before_lock begin_txn lock_table s3_wakeup end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_txn:
+ BEGIN;
+
+step lock_table:
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+ <waiting ...>
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step lock_table: <... completed>
+step end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s1_timeout_fast s2_timeout_slow wait_before_lock begin_and_read lock_table s3_wakeup end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s1_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step s2_timeout_slow:
+ SET deadlock_timeout = '10s';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step lock_table:
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+ <waiting ...>
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step lock_table: <... completed>
+ERROR: deadlock detected
+step end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s1_timeout_slow s2_timeout_fast wait_before_lock begin_and_read lock_table s3_wakeup end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s1_timeout_slow:
+ SET deadlock_timeout = '10s';
+
+step s2_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step lock_table:
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+ <waiting ...>
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step lock_table: <... completed>
+ERROR: deadlock detected
+step wait_before_lock: <... completed>
+step end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index d727a9b056b..f95f6782355 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -28,6 +28,14 @@ setup
SELECT injection_points_set_local();
SELECT injection_points_attach('repack-concurrently-before-lock', 'wait');
}
+step s1_timeout_fast
+{
+ SET deadlock_timeout = '10ms';
+}
+step s1_timeout_slow
+{
+ SET deadlock_timeout = '10s';
+}
# Perform the initial load and wait for s2 to do some data changes.
step wait_before_lock
{
@@ -61,6 +69,14 @@ teardown
}
session s2
+step s2_timeout_fast
+{
+ SET deadlock_timeout = '10ms';
+}
+step s2_timeout_slow
+{
+ SET deadlock_timeout = '10s';
+}
# Change the existing data. UPDATE changes both key and non-key columns. Also
# update one row twice to test whether tuple version generated by this session
# can be found.
@@ -128,6 +144,30 @@ step wakeup_before_lock
{
SELECT injection_points_wakeup('repack-concurrently-before-lock');
}
+# Steps used in lock contention tests.
+step begin_txn
+{
+ BEGIN;
+}
+step begin_and_read
+{
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+}
+step lock_table
+{
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+}
+step end_txn
+{
+ COMMIT;
+}
+
+session s3
+step s3_wakeup
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
# Test if data changes introduced while one session is performing REPACK
# CONCURRENTLY find their way into the table.
@@ -140,3 +180,43 @@ permutation
check2
wakeup_before_lock
check1
+
+# Test the soft-deadlock path of REPACK CONCURRENTLY's deadlock_protected
+# behavior. s2 is queued ahead of s1 (REPACK) waiting for AccessExclusiveLock
+# but holds no lock on the table itself, so the cycle has only soft edges and
+# the deadlock detector resolves it by reordering the wait queue. Both
+# sessions complete successfully.
+permutation
+ wait_before_lock
+ begin_txn
+ lock_table
+ s3_wakeup
+ end_txn
+
+# Test the hard-deadlock path of REPACK CONCURRENTLY's deadlock_protected
+# behavior when REPACK's deadlock detector runs first. s2 holds
+# AccessShareLock (from the SELECT) and then requests AccessExclusiveLock,
+# creating a hard cycle with s1 (REPACK) that cannot be resolved by queue
+# reordering. Because s1 has a much shorter deadlock_timeout and sets
+# deadlock_protected before acquiring AccessExclusiveLock, its deadlock check
+# runs first, cancels s2's lock wait, and lets REPACK complete.
+permutation
+ s1_timeout_fast
+ s2_timeout_slow
+ wait_before_lock
+ begin_and_read
+ lock_table(wait_before_lock)
+ s3_wakeup
+ end_txn
+
+# Same hard-deadlock setup, but with s2's deadlock detector forced to run
+# first. In this case s2 follows the ordinary hard-deadlock path and cancels
+# itself before REPACK's protected deadlock detector has a chance to act.
+permutation
+ s1_timeout_slow
+ s2_timeout_fast
+ wait_before_lock(lock_table)
+ begin_and_read
+ lock_table
+ s3_wakeup
+ end_txn
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-12 14:05 ` Andres Freund <andres@anarazel.de>
2026-04-12 14:58 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-12 14:05 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-12 15:31:20 +0200, Mihail Nikalayeu wrote:
> On Thu, Apr 9, 2026 at 4:20 PM Andres Freund <andres@anarazel.de> wrote:
> > But with my proposal to properly teach the deadlock detector about
> assuming
> > there's a wait edge for the eventual lock upgrade by S1, the first example
> > would still work, because the lock upgrade would not be considered a hard
> > cycle, and the second example would have S2 error out.
>
> Attached patch contains some (maybe naive) POC for similar approach.
> It adds a 'deadlock_protected' flag, which changes how the deadlock
> detector cancels backends.
>
> Instead of cancelling the backend entered the deadlock detector - it
> cancel some another (nearest hard edge) until it is possible to get the
> lock (either by
> reordering or directly).
I don't think that's as good. The problem is that that way you're only
detecting the deadlocks once they have materialized (i.e. once repack actually
does the lock upgrade), rather than cancelling when we know that the problem
starts. Having sessions pointlessly blocked for many hours is bad.
> Also, I added test cases with the scenarios you mentioned into the repack
> spec - to ensure repack is still working while other backend are cancelled.
I think we should perhaps commit spec tests for these (I've not yet reviewed
them, but in principle), even before we fix the problem. It's good to document
the current behavior and have a comment that the wrongly cancelled case should
not trigger an error.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-12 14:58 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-12 14:58 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi!
On Sun, Apr 12, 2026 at 4:05 PM Andres Freund <andres@anarazel.de> wrote:
> I don't think that's as good. The problem is that that way you're only
> detecting the deadlocks once they have materialized (i.e. once repack actually
> does the lock upgrade), rather than cancelling when we know that the problem
> starts. Having sessions pointlessly blocked for many hours is bad.
O, I think I understand you now.
You propose to somehow mark SUEL as "going to become AEL, so, the
deadlock detector should treat it as such".
In that case LOCK TABLE should get some kind of "future deadlock detected".
Yep, that feels much better. I'll try to check that approach tomorrow.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-14 13:37 ` Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-14 13:37 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Andres Freund <andres@anarazel.de> wrote:
> On 2026-04-12 15:31:20 +0200, Mihail Nikalayeu wrote:
> > Instead of cancelling the backend entered the deadlock detector - it
> > cancel some another (nearest hard edge) until it is possible to get the
> > lock (either by
> > reordering or directly).
>
> I don't think that's as good. The problem is that that way you're only
> detecting the deadlocks once they have materialized (i.e. once repack actually
> does the lock upgrade), rather than cancelling when we know that the problem
> starts.
This is my hack that tries to do that.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-14 13:58 ` Andres Freund <andres@anarazel.de>
2026-04-14 16:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-14 13:58 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-14 15:37:56 +0200, Antonin Houska wrote:
> Andres Freund <andres@anarazel.de> wrote:
>
> > On 2026-04-12 15:31:20 +0200, Mihail Nikalayeu wrote:
> > > Instead of cancelling the backend entered the deadlock detector - it
> > > cancel some another (nearest hard edge) until it is possible to get the
> > > lock (either by
> > > reordering or directly).
> >
> > I don't think that's as good. The problem is that that way you're only
> > detecting the deadlocks once they have materialized (i.e. once repack actually
> > does the lock upgrade), rather than cancelling when we know that the problem
> > starts.
>
> This is my hack that tries to do that.
I still think this needs to be in the deadlock detector. The lock cycle just
needs to be a bit more complicated for a hack in JoinWaitQueue not to work.
There's no guarantee that the wait that triggers the deadlock is actually on
the relation being repacked.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-14 16:55 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-14 16:55 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Antonin Houska <ah@cybertec.at>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Tue, Apr 14, 2026 at 3:58 PM Andres Freund <andres@anarazel.de> wrote:
> I still think this needs to be in the deadlock detector. The lock cycle just
> needs to be a bit more complicated for a hack in JoinWaitQueue not to work.
> There's no guarantee that the wait that triggers the deadlock is actually on
> the relation being repacked.
I have started prototyping a way to declare a "future" lock which the
deadlock detector treats as a hard edge.
But I currently stuck on issues related to the fact that SUE doesn't
force weak locks (fast-path) to go through
FastPathTransferRelationLocks, so the deadlock detector can't handle
the case when another backend tries to execute LOCK TABLE repack_test
IN SHARE UPDATE EXCLUSIVE MODE;
Also, VACUUM takes the same lock.
I'm not sure how to deal with this in a non-hacky way. One option is
to force SUE to transfer locks if relation it is trying to lock
relation which marked with "future lock". But I am not sure it is good
enough or covers all tricky cases (multiple backends in the loop).
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-15 14:50 ` Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 15:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-15 14:50 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Andres Freund <andres@anarazel.de> wrote:
> On 2026-04-14 15:37:56 +0200, Antonin Houska wrote:
> > Andres Freund <andres@anarazel.de> wrote:
> >
> > > On 2026-04-12 15:31:20 +0200, Mihail Nikalayeu wrote:
> > > > Instead of cancelling the backend entered the deadlock detector - it
> > > > cancel some another (nearest hard edge) until it is possible to get the
> > > > lock (either by
> > > > reordering or directly).
> > >
> > > I don't think that's as good. The problem is that that way you're only
> > > detecting the deadlocks once they have materialized (i.e. once repack actually
> > > does the lock upgrade), rather than cancelling when we know that the problem
> > > starts.
> >
> > This is my hack that tries to do that.
>
> I still think this needs to be in the deadlock detector. The lock cycle just
> needs to be a bit more complicated for a hack in JoinWaitQueue not to work.
> There's no guarantee that the wait that triggers the deadlock is actually on
> the relation being repacked.
ok, I see.
I thought of a "hypothetical graph", which would include the to-be-granted
lock, but the major issue is that it will not work correctly without the
locking the LMGR's LW locks we do in CheckDeadLock():
for (i = 0; i < NUM_LOCK_PARTITIONS; i++)
LWLockAcquire(LockHashPartitionLockByIndex(i), LW_EXCLUSIVE);
And obviously, doing this each time we want to insert a lock into the queue
would be bad for performance. It's even mentioned in the storage/lmgr/README
that the current approach is optimistic, so I think that major rework would be
needed if we wanted to entirely avoid waiting that leads to deadlock.
The approach proposed by Mihail [1] seems the least problematic to me, and
something like that occurred to me when I thought about the problem the first
time. However, when we wake up the other processes in order to run the
deadlock detection, they should do that immediately. I've got no good idea
about implementation at the moment, since latch can be set for unrelated
reasons. (Besides that, I have some more questions about this patch, which I
can post separately.)
[1] https://www.postgresql.org/message-id/CADzfLwURKVNQ%2B%2BDpi7bjoGfj-8pchDQEVex3eWBx0NCYn6TbDQ%40mail...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-15 14:59 ` Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Andres Freund @ 2026-04-15 14:59 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-15 16:50:11 +0200, Antonin Houska wrote:
> I thought of a "hypothetical graph", which would include the to-be-granted
> lock, but the major issue is that it will not work correctly without the
> locking the LMGR's LW locks we do in CheckDeadLock():
>
> for (i = 0; i < NUM_LOCK_PARTITIONS; i++)
> LWLockAcquire(LockHashPartitionLockByIndex(i), LW_EXCLUSIVE);
>
> And obviously, doing this each time we want to insert a lock into the queue
> would be bad for performance.
Hence my suggestion to do this as part of the deadlock check. Then we don't do
this unnecessary work outside of the case where we actually need it.
That does need to deal with the case of the deadlock check running first in
the backend doing repack, but that's not that hard - I think it'd be good
enough to set its deadlock timeout temporarily to a higher value. The backend
*should* still run the deadlock detector, because it could probably still get
into a deadlock (e.g. due to a pg_class access or something).
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
@ 2026-04-15 18:28 ` Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-15 18:28 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Andres Freund <andres@anarazel.de> wrote:
> On 2026-04-15 16:50:11 +0200, Antonin Houska wrote:
> > I thought of a "hypothetical graph", which would include the to-be-granted
> > lock, but the major issue is that it will not work correctly without the
> > locking the LMGR's LW locks we do in CheckDeadLock():
> >
> > for (i = 0; i < NUM_LOCK_PARTITIONS; i++)
> > LWLockAcquire(LockHashPartitionLockByIndex(i), LW_EXCLUSIVE);
> >
> > And obviously, doing this each time we want to insert a lock into the queue
> > would be bad for performance.
>
> Hence my suggestion to do this as part of the deadlock check. Then we don't do
> this unnecessary work outside of the case where we actually need it.
> That does need to deal with the case of the deadlock check running first in
> the backend doing repack, but that's not that hard - I think it'd be good
> enough to set its deadlock timeout temporarily to a higher value. The backend
> *should* still run the deadlock detector, because it could probably still get
> into a deadlock (e.g. due to a pg_class access or something).
Yes, the question is when we should run that check. I thought that it should
happen during each lock acquisition, and that made me worried about
performance. AFAIU you suggest REPACK to do something like:
1. Acquire ShareUpdateExclusiveLock
2. Perform the "enhanced check" to see if a future request for
AccessExclusiveLock can trigger a deadlock.
3. Do major part of the work (copy the table, build indexes, ...)
4. Request AccessExclusive lock.
5. Finish the work (process the remaining concurrent changes and swap the
table files).
This makes me concerned that if another session does
BEGIN;
TABLE t;
between steps 2 and 4, and something like
-- Get AccessExclusiveLock
ALTER TABLE t ADD COLUMN j int;
just after step 4, the two session will probably up in a deadlock anyway. In
other words, even if REPACK does the check early, it does not prevent other
sessions from getting in the way.
Maybe I'm still missing something.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-16 11:18 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-16 11:18 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Wed, Apr 15, 2026 at 8:28 PM Antonin Houska <ah@cybertec.at> wrote:
> just after step 4, the two session will probably up in a deadlock anyway. In
> other words, even if REPACK does the check early, it does not prevent other
> sessions from getting in the way.
>
> Maybe I'm still missing something.
I was trying to solve that by using another approach: the ability to
define a "future lock" [0]. It is declared even before taking the
actual lock, so, no race is possible.
That approach works correctly except one case -
ShareUpdateExclusiveLock from another backend, I described it here [1]
a little bit.
For now, I don't know how to solve it without a performance downgrade.
[0]: https://github.com/michail-nikolaev/postgres/commit/ba0f4247dad3d96b8282cd18056b7776cd69317c
[1]: https://www.postgresql.org/message-id/flat/CADzfLwU8Qw6LXFHO7Tbjc-O7o%2BtM26jdnOJBWqYLu61rf7bO%2Bg%4...
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-17 02:01 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-18 19:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-17 02:01 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
I think I got working POC for deadlock-detector enhancements for
REPACK (and potentially other).
Each PGPROC gets one FutureWaitLock slot: a (locktag, mode) pair,
which is empty when locktag_lockmethodid == 0.
A backend calls LockDeclareFutureWait() to publish its intent. Only
REPACK (CONCURRENTLY) uses it today, declaring its future
AccessExclusiveLock.
Only the owning backend writes its slot; it does so under the
partition lock covering the declared tag. Remote readers (the deadlock
detector) must hold that partition lock to observe it.
The detector holds all partition locks during its graph walk, so it
can read any slot safely. The slot clears in LockAcquireExtended() at
the exact point the backend attaches its waitLink for the same tag and
mode.
It is also cleared on abort, backend exit, and end of transaction.
FindLockCycleRecurseFuture() treats the declared future lock as a hard
edge to (a) current holders whose mode conflicts and (b)
already-queued waiters that JoinWaitQueue() would place ahead of the
future request.
Weak relation locks are kept in per-backend fast-path arrays and
remain invisible to the detector's graph walk.
Normally LockAcquireExtended() migrates them to the main lock table
when a conflicting strong lock is requested.
This action is gated by FastPathStrongRelationLocks->count to prevent
new fast-path lock processing.
A future-wait declaration does not trigger that migration due to
performance considerations.
Instead CheckDeadLock() now snapshots currently declared
fast-path-incompatible future waits and increments
FastPathStrongRelationLocks->count for those tags (blocking new
fast-path acquisitions).
It then calls FastPathTransferRelationLocks() for each such tag to
move existing holders into the main table, making everything visible
to the deadlock walker.
Once the deadlock checker finishes, FastPathStrongRelationLocks->count
is decremented enabling fast-path processing again. Performance
degradation occurs only during deadlock processing and only if REPACK
is active.
If a deadlock contains a "future" edge it is reported as "future
deadlock detected".
Repack first declares future AE and only then proceeds to actually get SUE.
Added test cases for different scenarios, including SUE from another
backend and a 3-backend deadlock.
Attachments:
[application/octet-stream] nocfbot-v2-0001-Detect-deadlocks-involving-declared-future-lock-r.patch (46.1K, ../../CADzfLwUSnGnkfLwCWHQ=VVuAY1YTo+0Lr7pb+OPWUZbcYKSRUw@mail.gmail.com/2-nocfbot-v2-0001-Detect-deadlocks-involving-declared-future-lock-r.patch)
download | inline diff:
From 7e052a5f3aafa3a9ca1d7fc866c597001251bcc2 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Fri, 17 Apr 2026 02:09:40 +0200
Subject: [PATCH v2] Detect deadlocks involving declared future lock requests
Introduce a mechanism for a backend to declare its intent to acquire a
lock at a future point, before actually calling LockAcquire(). The
deadlock detector treats each declaration as a hard waits-for edge from
the declarer to current holders whose mode conflicts, and to already-
queued waiters whose requests would be ordered before the future
request. This makes it possible to detect cycles that the existing
detector cannot see, because the declarer has not yet queued for its
stronger lock.
The motivating caller is REPACK (CONCURRENTLY), which holds a
ShareUpdateExclusiveLock throughout and later needs AccessExclusiveLock
to swap relfilenodes. A concurrent backend that takes an intermediate
lock and waits for REPACK to finish can form a cycle that today is
invisible until REPACK finally requests its strong lock. Declaring the
future AccessExclusiveLock from RangeVarCallbackForRepack surfaces the
edge immediately after the initial lookup.
Implementation notes:
* PGPROC gains a single FutureWaitLock slot (locktag + mode). Only the
owning proc writes it; remote readers must hold the partition lock
covering the tag. The invariant "slot set implies not yet really
waiting for that (tag, mode)" is maintained by clearing the slot
inside LockAcquireExtended() at the point the proc attaches its
waitLink for that same request. InitProcess, ProcKill,
LockErrorCleanup, and LockReleaseAll also clear the slot.
* Fast-path holders of relation locks are invisible to the deadlock
detector, so CheckDeadLock() now snapshots the currently declared
fast-path-relevant future waits, bumps FastPathStrongRelationLocks
counters for those tags, and transfers existing fast-path holders
into the main lock table before walking the waits-for graph. A
shared atomic count lets the common case (no active declarations)
skip this work entirely. If a new fast-path-relevant declaration
appears between the snapshot and the partition-locked walk, the
check restarts with a fresh snapshot.
* FindLockCycleRecurseFuture() follows the declared edge to current
holders and to earlier queued waiters that JoinWaitQueue() would
place ahead of the future request. These edges are not recorded as
soft edges: the declarer is not in the wait queue, so reordering
cannot break such a cycle.
* DeadLockReport() emits "future deadlock detected" and a
"will request ... (declared future intent)" detail line when the
cycle involves a declared edge. SQLSTATE remains
ERRCODE_T_R_DEADLOCK_DETECTED.
New isolation permutations under src/test/modules/injection_points
cover the no-deadlock case (waiter with no conflicting held lock),
two two-backend future-deadlock cycles (held AccessShare + future
AccessExclusive / ShareUpdateExclusive), and a three-backend cycle
through an unrelated table.
---
src/backend/commands/repack.c | 4 +-
src/backend/commands/tablecmds.c | 33 ++
src/backend/storage/lmgr/deadlock.c | 195 ++++++++-
src/backend/storage/lmgr/lock.c | 410 ++++++++++++++++++
src/backend/storage/lmgr/proc.c | 36 ++
src/include/commands/tablecmds.h | 3 +
src/include/storage/lmgr.h | 2 +
src/include/storage/lock.h | 33 ++
src/include/storage/proc.h | 9 +
.../injection_points/expected/repack.out | 181 +++++++-
.../injection_points/specs/repack.spec | 118 +++++
11 files changed, 1018 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 58e3867246f..8261c58daca 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -2326,8 +2326,8 @@ process_single_relation(RepackStmt *stmt, LOCKMODE lockmode, bool isTopLevel,
tableOid = RangeVarGetRelidExtended(stmt->relation->relation,
lockmode,
0,
- RangeVarCallbackMaintainsTable,
- NULL);
+ RangeVarCallbackForRepack,
+ params);
rel = table_open(tableOid, NoLock);
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index eec09ba1ded..b08f57017e3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -19761,6 +19761,39 @@ RangeVarCallbackMaintainsTable(const RangeVar *relation,
relation->relname);
}
+/*
+ * Callback to RangeVarGetRelidExtended() for REPACK. For concurrent repack,
+ * declare the future AccessExclusiveLock before the caller locks the table
+ * with ShareUpdateExclusiveLock, so deadlock checks can see the pending
+ * request while the weaker lock is held or awaited.
+ */
+void
+RangeVarCallbackForRepack(const RangeVar *relation,
+ Oid relId, Oid oldRelId, void *arg)
+{
+ ClusterParams *params = arg;
+ LOCKTAG locktag;
+ Oid dbid;
+
+ RangeVarCallbackMaintainsTable(relation, relId, oldRelId, NULL);
+
+ if ((params->options & CLUOPT_CONCURRENT) == 0)
+ return;
+
+ if (relId == oldRelId)
+ return;
+
+ if (OidIsValid(oldRelId))
+ LockClearFutureWaitSlot(false);
+
+ if (!OidIsValid(relId))
+ return;
+
+ dbid = IsSharedRelation(relId) ? InvalidOid : MyDatabaseId;
+ SET_LOCKTAG_RELATION(locktag, dbid, relId);
+ LockDeclareFutureWait(&locktag, AccessExclusiveLock);
+}
+
/*
* Callback to RangeVarGetRelidExtended() for TRUNCATE processing.
*/
diff --git a/src/backend/storage/lmgr/deadlock.c b/src/backend/storage/lmgr/deadlock.c
index b8962d875b6..a8ea9337e8d 100644
--- a/src/backend/storage/lmgr/deadlock.c
+++ b/src/backend/storage/lmgr/deadlock.c
@@ -74,6 +74,7 @@ typedef struct
LOCKTAG locktag; /* ID of awaited lock object */
LOCKMODE lockmode; /* type of lock we're waiting for */
int pid; /* PID of blocked backend */
+ bool is_future; /* was this a declared future-wait edge? */
} DEADLOCK_INFO;
@@ -86,6 +87,11 @@ static bool FindLockCycleRecurse(PGPROC *checkProc, int depth,
static bool FindLockCycleRecurseMember(PGPROC *checkProc,
PGPROC *checkProcLeader,
int depth, EDGE *softEdges, int *nSoftEdges);
+static bool FindLockCycleRecurseWait(PGPROC *checkProc,
+ PGPROC *checkProcLeader,
+ int depth, EDGE *softEdges, int *nSoftEdges);
+static bool FindLockCycleRecurseFuture(PGPROC *checkProc,
+ int depth, EDGE *softEdges, int *nSoftEdges);
static bool ExpandConstraints(EDGE *constraints, int nConstraints);
static bool TopoSort(LOCK *lock, EDGE *constraints, int nConstraints,
PGPROC **ordering);
@@ -504,7 +510,8 @@ FindLockCycleRecurse(PGPROC *checkProc,
* If the process is waiting, there is an outgoing waits-for edge to each
* process that blocks it.
*/
- if (!dlist_node_is_detached(&checkProc->waitLink) &&
+ if ((!dlist_node_is_detached(&checkProc->waitLink) ||
+ FutureWaitLockIsSet(&checkProc->futureWaitLock)) &&
FindLockCycleRecurseMember(checkProc, checkProc, depth, softEdges,
nSoftEdges))
return true;
@@ -538,6 +545,34 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
int depth,
EDGE *softEdges, /* output argument */
int *nSoftEdges) /* output argument */
+{
+ /* Follow outgoing edges from a real lock wait, if any. */
+ if (!dlist_node_is_detached(&checkProc->waitLink) &&
+ FindLockCycleRecurseWait(checkProc, checkProcLeader, depth,
+ softEdges, nSoftEdges))
+ return true;
+
+ /*
+ * A future-wait slot is only useful when another backend is already
+ * waiting and reaches this proc through the waits-for graph. Do not start
+ * from our own future slot: until we actually request that lock, there is
+ * no current wait to break.
+ */
+ if (checkProc != MyProc &&
+ checkProc == checkProcLeader &&
+ FutureWaitLockIsSet(&checkProc->futureWaitLock) &&
+ FindLockCycleRecurseFuture(checkProc, depth, softEdges, nSoftEdges))
+ return true;
+
+ return false;
+}
+
+static bool
+FindLockCycleRecurseWait(PGPROC *checkProc,
+ PGPROC *checkProcLeader,
+ int depth,
+ EDGE *softEdges, /* output argument */
+ int *nSoftEdges) /* output argument */
{
PGPROC *proc;
LOCK *lock = checkProc->waitLock;
@@ -590,6 +625,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ info->is_future = false;
return true;
}
@@ -679,6 +715,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ info->is_future = false;
/*
* Add this edge to the list of soft edges in the cycle
@@ -753,6 +790,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ info->is_future = false;
/*
* Add this edge to the list of soft edges in the cycle
@@ -774,6 +812,149 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
return false;
}
+/*
+ * Follow a declared future-wait edge as a hard edge.
+ */
+static bool
+FindLockCycleRecurseFuture(PGPROC *checkProc,
+ int depth,
+ EDGE *softEdges, /* output argument */
+ int *nSoftEdges) /* output argument */
+{
+ FutureWaitLock *futureWaitLock = &checkProc->futureWaitLock;
+ LOCK *lock;
+ LockMethod lockMethodTable;
+ uint32 hashcode;
+ int conflictMask;
+ LOCKMASK myHeldLocks = 0;
+ dlist_iter proclock_iter;
+ dlist_iter proc_iter;
+
+ Assert(FutureWaitLockIsSet(futureWaitLock));
+ Assert(checkProc != MyProc);
+
+ /*
+ * Invariant maintained by LockAcquireExtended(): if a proc is really
+ * waiting on a lock, its future-wait slot (if any) must not describe the
+ * same (locktag, lockmode). The slot is cleared at the point we attach
+ * our waitLink for that exact lock and mode, so the wait walker's
+ * procLocks scan already subsumes anything this walker would find.
+ */
+ Assert(dlist_node_is_detached(&checkProc->waitLink) ||
+ checkProc->waitLockMode != futureWaitLock->mode ||
+ memcmp(&checkProc->waitLock->tag, &futureWaitLock->locktag,
+ sizeof(LOCKTAG)) != 0);
+
+ /*
+ * Look up the shared LOCK object for the declared tag. It is legal for
+ * the entry not to exist: the hash table only contains LOCK objects that
+ * currently have at least one holder or requester, and a future-wait
+ * declaration can be made before any proclock for that tag is created
+ * (e.g. RangeVarCallbackForRepack runs inside RangeVarGetRelidExtended
+ * *before* the caller has actually taken the initial SUE lock on the
+ * relation, and no other backend may have any lock on it either). With
+ * no LOCK object there are no current holders, hence no outgoing edges
+ * from this future-wait slot, so simply return.
+ */
+ hashcode = LockTagHashCode(&futureWaitLock->locktag);
+ lock = LockHashLookup(&futureWaitLock->locktag, hashcode);
+ if (lock == NULL)
+ return false;
+
+ lockMethodTable = GetLocksMethodTable(lock);
+ Assert(futureWaitLock->mode > 0 &&
+ futureWaitLock->mode <= lockMethodTable->numLockModes);
+ conflictMask = lockMethodTable->conflictTab[futureWaitLock->mode];
+
+ /*
+ * First, follow edges to current holders of conflicting modes. These
+ * edges are the future-wait equivalent of the normal hard edges from a
+ * real waiter to current holders.
+ */
+ dlist_foreach(proclock_iter, &lock->procLocks)
+ {
+ PROCLOCK *proclock = dlist_container(PROCLOCK, lockLink, proclock_iter.cur);
+ PGPROC *proc = proclock->tag.myProc;
+ PGPROC *leader;
+
+ leader = proc->lockGroupLeader == NULL ? proc : proc->lockGroupLeader;
+
+ /* A proc never blocks itself or any other lock group member. */
+ if (leader == checkProc)
+ {
+ myHeldLocks |= proclock->holdMask;
+ continue;
+ }
+
+ if ((proclock->holdMask & conflictMask) != 0)
+ {
+ if (FindLockCycleRecurse(proc, depth + 1,
+ softEdges, nSoftEdges))
+ {
+ DEADLOCK_INFO *info = &deadlockDetails[depth];
+
+ info->locktag = futureWaitLock->locktag;
+ info->lockmode = futureWaitLock->mode;
+ info->pid = checkProc->pid;
+ info->is_future = true;
+
+ return true;
+ }
+ }
+ }
+
+ /*
+ * Existing waiters can also become blockers for the future request. A
+ * real LockAcquireExtended() first checks lock->waitMask, and if there
+ * are conflicting waiters it calls JoinWaitQueue(). Model the queue
+ * position JoinWaitQueue() would choose today, and follow only queued
+ * requests that would be ahead of that position.
+ *
+ * Do not record these as soft edges. The future requester is not present
+ * in lock->waitProcs, so the deadlock detector cannot repair such an edge
+ * by reordering the current wait queue.
+ */
+ dclist_foreach(proc_iter, &lock->waitProcs)
+ {
+ PGPROC *proc = dlist_container(PGPROC, waitLink, proc_iter.cur);
+ PGPROC *leader;
+ LOCKMODE waitLockMode = proc->waitLockMode;
+
+ leader = proc->lockGroupLeader == NULL ? proc : proc->lockGroupLeader;
+
+ if (leader == checkProc)
+ continue;
+
+ /*
+ * If checkProc already holds locks that conflict with this waiter's
+ * request, JoinWaitQueue() would insert the future request before
+ * this waiter. We are done scanning the queue after considering
+ * earlier waiters.
+ */
+ if (myHeldLocks != 0 &&
+ (lockMethodTable->conflictTab[waitLockMode] & myHeldLocks) != 0)
+ break;
+
+ if ((LOCKBIT_ON(waitLockMode) & conflictMask) != 0)
+ {
+ if (FindLockCycleRecurse(proc, depth + 1,
+ softEdges, nSoftEdges))
+ {
+ DEADLOCK_INFO *info = &deadlockDetails[depth];
+
+ info->locktag = futureWaitLock->locktag;
+ info->lockmode = futureWaitLock->mode;
+ info->pid = checkProc->pid;
+ info->is_future = true;
+
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* ExpandConstraints -- expand a list of constraints into a set of
@@ -1078,6 +1259,7 @@ DeadLockReport(void)
StringInfoData logbuf; /* errdetail for server log */
StringInfoData locktagbuf;
int i;
+ bool any_future = false;
initStringInfo(&clientbuf);
initStringInfo(&logbuf);
@@ -1099,12 +1281,15 @@ DeadLockReport(void)
resetStringInfo(&locktagbuf);
DescribeLockTag(&locktagbuf, &info->locktag);
+ any_future |= info->is_future;
if (i > 0)
appendStringInfoChar(&clientbuf, '\n');
appendStringInfo(&clientbuf,
- _("Process %d waits for %s on %s; blocked by process %d."),
+ info->is_future
+ ? _("Process %d will request %s on %s (declared future intent); blocked by process %d.")
+ : _("Process %d waits for %s on %s; blocked by process %d."),
info->pid,
GetLockmodeName(info->locktag.locktag_lockmethodid,
info->lockmode),
@@ -1132,7 +1317,9 @@ DeadLockReport(void)
ereport(ERROR,
(errcode(ERRCODE_T_R_DEADLOCK_DETECTED),
- errmsg("deadlock detected"),
+ errmsg(any_future
+ ? "future deadlock detected"
+ : "deadlock detected"),
errdetail_internal("%s", clientbuf.data),
errdetail_log("%s", logbuf.data),
errhint("See server log for query details.")));
@@ -1154,9 +1341,11 @@ RememberSimpleDeadLock(PGPROC *proc1,
info->locktag = lock->tag;
info->lockmode = lockmode;
info->pid = proc1->pid;
+ info->is_future = false;
info++;
info->locktag = proc2->waitLock->tag;
info->lockmode = proc2->waitLockMode;
info->pid = proc2->pid;
+ info->is_future = false;
nDeadlockDetails = 2;
}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c221fe96889..12107e07bec 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -314,6 +314,19 @@ typedef struct
static volatile FastPathStrongRelationLockData *FastPathStrongRelationLocks;
+/*
+ * Count of active future-wait declarations that could conflict with relation
+ * fast-path locks. This is only a skip hint for CheckDeadLock(); the actual
+ * future-wait locktags are read from PGPROC.futureWaitLock while all lock
+ * partitions are held.
+ */
+typedef struct FutureWaitFastPathCtlData
+{
+ pg_atomic_uint32 count;
+} FutureWaitFastPathCtlData;
+
+static FutureWaitFastPathCtlData *FutureWaitFastPathCtl = NULL;
+
static void LockManagerShmemRequest(void *arg);
static void LockManagerShmemInit(void *arg);
@@ -488,12 +501,18 @@ LockManagerShmemRequest(void *arg)
.size = sizeof(FastPathStrongRelationLockData),
.ptr = (void **) (void *) &FastPathStrongRelationLocks,
);
+
+ ShmemRequestStruct(.name = "Future Wait Fast Path Control",
+ .size = sizeof(FutureWaitFastPathCtlData),
+ .ptr = (void **) &FutureWaitFastPathCtl,
+ );
}
static void
LockManagerShmemInit(void *arg)
{
SpinLockInit(&FastPathStrongRelationLocks->mutex);
+ pg_atomic_init_u32(&FutureWaitFastPathCtl->count, 0);
}
/*
@@ -628,6 +647,374 @@ DoLockModesConflict(LOCKMODE mode1, LOCKMODE mode2)
return false;
}
+/*
+ * LockHashLookup -- look up a LOCK by locktag and precomputed hashcode.
+ *
+ * Returns NULL if no LOCK exists for this tag.
+ *
+ * Callers MUST hold the partition lock covering hashcode in LW_EXCLUSIVE
+ * mode, and MUST treat the returned pointer as valid only while that
+ * partition lock remains held: the LOCK entry can be removed from the hash
+ * table by LockRelease() under the same partition lock, so dropping the
+ * lock invalidates the pointer.
+ *
+ * Today the only caller is the deadlock detector, which satisfies both
+ * requirements by acquiring every lock-manager partition lock exclusively
+ * before walking the wait-for graph, and uses the result to resolve a
+ * declared future-wait edge's locktag into a live LOCK *. New callers
+ * from outside that context should be reviewed carefully.
+ */
+LOCK *
+LockHashLookup(const LOCKTAG *locktag, uint32 hashcode)
+{
+ Assert(LWLockHeldByMeInMode(LockHashPartitionLock(hashcode),
+ LW_EXCLUSIVE));
+
+ return (LOCK *) hash_search_with_hash_value(LockMethodLockHash,
+ locktag,
+ hashcode,
+ HASH_FIND,
+ NULL);
+}
+
+static inline bool
+FutureWaitLockMatches(const FutureWaitLock *futureWaitLock,
+ const LOCKTAG *locktag, LOCKMODE lockmode)
+{
+ return FutureWaitLockIsSet(futureWaitLock) &&
+ futureWaitLock->mode == lockmode &&
+ memcmp(&futureWaitLock->locktag, locktag, sizeof(LOCKTAG)) == 0;
+}
+
+static inline bool
+FutureWaitNeedsFastPathMigration(const FutureWaitLock *futureWaitLock)
+{
+ return FutureWaitLockIsSet(futureWaitLock) &&
+ ConflictsWithRelationFastPath(&futureWaitLock->locktag,
+ futureWaitLock->mode);
+}
+
+/*
+ * LockClearFutureWaitSlotIfMatch -- clear our future-wait slot if it matches
+ * a lock that we have now acquired or started waiting for.
+ */
+static void
+LockClearFutureWaitSlotIfMatch(const LOCKTAG *locktag, LOCKMODE lockmode,
+ bool partitionLockHeld)
+{
+ if (!FutureWaitLockMatches(&MyProc->futureWaitLock, locktag, lockmode))
+ return;
+
+ LockClearFutureWaitSlot(partitionLockHeld);
+}
+
+/*
+ * LockDeclareFutureWait -- publish a hard future waits-for edge.
+ *
+ * The caller declares that this backend intends to request lockmode on
+ * locktag later. The declaration becomes visible to deadlock detection
+ * before the real lock acquisition happens.
+ *
+ * Only one future-wait slot is supported per backend, and parallel workers
+ * (non-leader lock-group members) cannot declare a future wait. REPACK
+ * CONCURRENTLY is the only current caller and satisfies both constraints.
+ */
+void
+LockDeclareFutureWait(const LOCKTAG *locktag, LOCKMODE lockmode)
+{
+ uint32 hashcode;
+ LWLock *partitionLock;
+
+ Assert(MyProc != NULL);
+ Assert(MyProc->lockGroupLeader == NULL ||
+ MyProc->lockGroupLeader == MyProc);
+ Assert(!LockHeldByMe(locktag, lockmode, true));
+ Assert(FutureWaitLockIsEmpty(&MyProc->futureWaitLock));
+
+ hashcode = LockTagHashCode(locktag);
+ partitionLock = LockHashPartitionLock(hashcode);
+
+ /*
+ * Bump the fast-path migration hint before publishing the slot. A
+ * concurrent CheckDeadLock() may see the hint before the slot is visible,
+ * which is harmless; the reverse ordering would allow it to observe a
+ * future edge without migrating fast-path holders for that relation.
+ */
+ if (ConflictsWithRelationFastPath(locktag, lockmode))
+ pg_atomic_fetch_add_u32(&FutureWaitFastPathCtl->count, 1);
+
+ LWLockAcquire(partitionLock, LW_EXCLUSIVE);
+
+ MyProc->futureWaitLock.locktag = *locktag;
+ MyProc->futureWaitLock.mode = lockmode;
+
+ LWLockRelease(partitionLock);
+}
+
+/*
+ * LockClearFutureWaitSlot -- clear this backend's declared future wait, if any.
+ */
+void
+LockClearFutureWaitSlot(bool partitionLockHeld)
+{
+ LWLock *partitionLock;
+ LOCKTAG clearedTag;
+ LOCKMODE clearedMode;
+ bool cleared = false;
+
+ if (MyProc == NULL || FutureWaitLockIsEmpty(&MyProc->futureWaitLock))
+ return;
+
+ partitionLock =
+ LockHashPartitionLock(LockTagHashCode(&MyProc->futureWaitLock.locktag));
+ if (!partitionLockHeld)
+ LWLockAcquire(partitionLock, LW_EXCLUSIVE);
+ else
+ Assert(LWLockHeldByMeInMode(partitionLock, LW_EXCLUSIVE));
+
+ if (FutureWaitLockIsSet(&MyProc->futureWaitLock))
+ {
+ clearedTag = MyProc->futureWaitLock.locktag;
+ clearedMode = MyProc->futureWaitLock.mode;
+ MemSet(&MyProc->futureWaitLock, 0, sizeof(FutureWaitLock));
+ cleared = true;
+ }
+
+ if (!partitionLockHeld)
+ LWLockRelease(partitionLock);
+
+ if (cleared && ConflictsWithRelationFastPath(&clearedTag, clearedMode))
+ {
+ Assert(pg_atomic_read_u32(&FutureWaitFastPathCtl->count) > 0);
+ pg_atomic_fetch_sub_u32(&FutureWaitFastPathCtl->count, 1);
+ }
+}
+
+/*
+ * FutureWaitLocktagSnapshotContains -- linear membership test over a
+ * locktag snapshot.
+ *
+ * n is bounded by the number of declared future waits (effectively the
+ * number of concurrent REPACK CONCURRENTLY commands), which in practice
+ * is 0 or 1, so a linear scan is cheaper than building a hash or keeping
+ * the array sorted.
+ */
+static bool
+FutureWaitLocktagSnapshotContains(LOCKTAG *locktags, int n,
+ const LOCKTAG *locktag)
+{
+ for (int i = 0; i < n; i++)
+ {
+ if (memcmp(&locktags[i], locktag, sizeof(LOCKTAG)) == 0)
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * SnapshotFutureWaitFastPathLocks -- copy the current fast-path-relevant
+ * future-wait tags while all lock partitions are held.
+ *
+ * The caller holds every lock-manager partition lock exclusively, so this
+ * function must not take any LWLock. palloc is safe here because the
+ * memory context machinery does not take LWLocks; a palloc failure would
+ * ereport out of the deadlock check but not deadlock against itself.
+ */
+static LOCKTAG *
+SnapshotFutureWaitFastPathLocks(int *out_count)
+{
+ LOCKTAG *locktags;
+ int n = 0;
+
+ *out_count = 0;
+
+ locktags = palloc_array(LOCKTAG, ProcGlobal->allProcCount);
+
+ for (int i = 0; i < ProcGlobal->allProcCount; i++)
+ {
+ PGPROC *proc = GetPGProcByNumber(i);
+ FutureWaitLock *futureWaitLock = &proc->futureWaitLock;
+
+ if (!FutureWaitNeedsFastPathMigration(futureWaitLock))
+ continue;
+
+ Assert(LWLockHeldByMeInMode(LockHashPartitionLock(
+ LockTagHashCode(&futureWaitLock->locktag)),
+ LW_EXCLUSIVE));
+
+ if (!FutureWaitLocktagSnapshotContains(locktags, n,
+ &futureWaitLock->locktag))
+ {
+ Assert(n < ProcGlobal->allProcCount);
+ locktags[n++] = futureWaitLock->locktag;
+ }
+ }
+
+ if (n == 0)
+ {
+ pfree(locktags);
+ return NULL;
+ }
+
+ *out_count = n;
+ return locktags;
+}
+
+/*
+ * MigrateFutureWaitFastPathLocks -- make fast-path holders visible to the
+ * deadlock detector for currently declared future waits.
+ *
+ * Future-wait declarations do not take a real strong lock yet, so they do not
+ * trigger the normal fast-path transfer performed by LockAcquireExtended().
+ * This helper takes a snapshot of current fast-path-relevant future waits,
+ * installs temporary strong-lock counter bumps for those relation hash
+ * partitions, and transfers existing fast-path holders of those relations
+ * into the main lock table.
+ *
+ * The returned snapshot must remain active until CheckDeadLock() has finished
+ * its partition-locked graph walk. New fast-path holders cannot appear for
+ * migrated tags while the temporary strong-lock counter bumps are active. If
+ * a new future-wait tag appears after this snapshot, CheckDeadLock() detects
+ * that with FutureWaitFastPathSnapshotCoversCurrentLocks() and retries.
+ */
+LOCKTAG *
+MigrateFutureWaitFastPathLocks(int *out_count)
+{
+ LockMethod lockMethodTable = LockMethods[DEFAULT_LOCKMETHOD];
+ LOCKTAG *locktags;
+ int n;
+
+ *out_count = 0;
+
+ /* Cheap path: no active declaration can involve fast-path holders. */
+ if (pg_atomic_read_u32(&FutureWaitFastPathCtl->count) == 0)
+ return NULL;
+
+ for (int i = 0; i < NUM_LOCK_PARTITIONS; i++)
+ LWLockAcquire(LockHashPartitionLockByIndex(i), LW_EXCLUSIVE);
+
+ locktags = SnapshotFutureWaitFastPathLocks(&n);
+
+ for (int i = NUM_LOCK_PARTITIONS; --i >= 0;)
+ LWLockRelease(LockHashPartitionLockByIndex(i));
+
+ if (locktags == NULL)
+ return NULL;
+
+ /*
+ * Block new fast-path acquisitions for this snapshot before transferring
+ * existing holders to the main table.
+ */
+ SpinLockAcquire(&FastPathStrongRelationLocks->mutex);
+ for (int i = 0; i < n; i++)
+ {
+ uint32 hashcode = LockTagHashCode(&locktags[i]);
+ uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode);
+
+ FastPathStrongRelationLocks->count[fasthashcode]++;
+ }
+ SpinLockRelease(&FastPathStrongRelationLocks->mutex);
+
+ for (int i = 0; i < n; i++)
+ {
+ uint32 hashcode = LockTagHashCode(&locktags[i]);
+
+ if (!FastPathTransferRelationLocks(lockMethodTable, &locktags[i],
+ hashcode))
+ {
+ SpinLockAcquire(&FastPathStrongRelationLocks->mutex);
+ for (int j = 0; j < n; j++)
+ {
+ uint32 restorehashcode = LockTagHashCode(&locktags[j]);
+ uint32 restorefasthashcode =
+ FastPathStrongLockHashPartition(restorehashcode);
+
+ Assert(FastPathStrongRelationLocks->count[restorefasthashcode]
+ > 0);
+ FastPathStrongRelationLocks->count[restorefasthashcode]--;
+ }
+ SpinLockRelease(&FastPathStrongRelationLocks->mutex);
+ pfree(locktags);
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of shared memory"),
+ errhint("You might need to increase \"%s\".", "max_locks_per_transaction")));
+ }
+ }
+
+ *out_count = n;
+ return locktags;
+}
+
+/*
+ * FutureWaitFastPathSnapshotCoversCurrentLocks -- does the active snapshot
+ * cover all fast-path-relevant future waits currently visible to the deadlock
+ * detector?
+ *
+ * Caller must hold all lock partitions. If this returns false, a declaration
+ * raced with the previous snapshot and CheckDeadLock() must retry migration
+ * before walking the waits-for graph.
+ */
+bool
+FutureWaitFastPathSnapshotCoversCurrentLocks(LOCKTAG *locktags, int n)
+{
+ /*
+ * With no snapshot and no in-progress fast-path-relevant declaration,
+ * there cannot be anything to validate. If the counter is nonzero, still
+ * scan: a declaration may have incremented the counter before publishing
+ * its slot, causing the snapshot to be empty.
+ */
+ if (locktags == NULL &&
+ pg_atomic_read_u32(&FutureWaitFastPathCtl->count) == 0)
+ return true;
+
+ for (int i = 0; i < ProcGlobal->allProcCount; i++)
+ {
+ PGPROC *proc = GetPGProcByNumber(i);
+ FutureWaitLock *futureWaitLock = &proc->futureWaitLock;
+
+ if (!FutureWaitNeedsFastPathMigration(futureWaitLock))
+ continue;
+
+ Assert(LWLockHeldByMeInMode(LockHashPartitionLock(
+ LockTagHashCode(&futureWaitLock->locktag)),
+ LW_EXCLUSIVE));
+
+ if (!FutureWaitLocktagSnapshotContains(locktags, n,
+ &futureWaitLock->locktag))
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * RestoreFutureWaitFastPathSnapshot -- undo the fast-path strong-lock counter
+ * bumps for a snapshot returned by MigrateFutureWaitFastPathLocks().
+ *
+ * Must be called after CheckDeadLock() has released all partition locks.
+ */
+void
+RestoreFutureWaitFastPathSnapshot(LOCKTAG *locktags, int n)
+{
+ if (locktags == NULL)
+ return;
+
+ SpinLockAcquire(&FastPathStrongRelationLocks->mutex);
+ for (int i = 0; i < n; i++)
+ {
+ uint32 hashcode = LockTagHashCode(&locktags[i]);
+ uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode);
+
+ Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0);
+ FastPathStrongRelationLocks->count[fasthashcode]--;
+ }
+ SpinLockRelease(&FastPathStrongRelationLocks->mutex);
+
+ pfree(locktags);
+}
+
/*
* LockHeldByMe -- test whether lock 'locktag' is held by the current
* transaction
@@ -937,6 +1324,8 @@ LockAcquireExtended(const LOCKTAG *locktag,
*/
if (locallock->nLocks > 0)
{
+ Assert(!FutureWaitLockMatches(&MyProc->futureWaitLock, locktag,
+ lockmode));
GrantLockLocal(locallock, owner);
if (locallock->lockCleared)
return LOCKACQUIRE_ALREADY_CLEAR;
@@ -1012,6 +1401,7 @@ LockAcquireExtended(const LOCKTAG *locktag,
*/
locallock->lock = NULL;
locallock->proclock = NULL;
+ LockClearFutureWaitSlotIfMatch(locktag, lockmode, false);
GrantLockLocal(locallock, owner);
return LOCKACQUIRE_OK;
}
@@ -1225,6 +1615,20 @@ LockAcquireExtended(const LOCKTAG *locktag,
Assert(!dontWait);
PROCLOCK_PRINT("LockAcquire: sleeping on lock", proclock);
LOCK_PRINT("LockAcquire: sleeping on lock", lock, lockmode);
+
+ /*
+ * We have attached waitLink and are about to sleep on exactly the
+ * (locktag, lockmode) we had declared as a future wait. Remote
+ * walkers will now see us as a real waiter on this lock, and
+ * FindLockCycleRecurseWait() will scan lock->procLocks with the same
+ * conflictMask the future walker would have used. Clearing the
+ * future slot now, while we still hold the partition lock, keeps
+ * the invariant "future slot set means not yet really waiting for the
+ * same lock and mode" and removes duplicate work from the deadlock
+ * detector.
+ */
+ LockClearFutureWaitSlotIfMatch(locktag, lockmode, true);
+
LWLockRelease(partitionLock);
waitResult = WaitOnLock(locallock, owner);
@@ -1247,7 +1651,10 @@ LockAcquireExtended(const LOCKTAG *locktag,
}
}
else
+ {
+ LockClearFutureWaitSlotIfMatch(locktag, lockmode, true);
LWLockRelease(partitionLock);
+ }
Assert(waitResult == PROC_WAIT_STATUS_OK);
/* The lock was granted to us. Update the local lock entry accordingly */
@@ -2328,6 +2735,9 @@ LockReleaseAll(LOCKMETHODID lockmethodid, bool allLocks)
elog(ERROR, "unrecognized lock method: %d", lockmethodid);
lockMethodTable = LockMethods[lockmethodid];
+ if (lockmethodid == DEFAULT_LOCKMETHOD)
+ LockClearFutureWaitSlot(false);
+
#ifdef LOCK_DEBUG
if (*(lockMethodTable->trace_flag))
elog(LOG, "LockReleaseAll: lockmethod=%d", lockmethodid);
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 1ac25068d62..b6b9de2d51c 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -494,6 +494,7 @@ InitProcess(void)
MyProc->waitLock = NULL;
dlist_node_init(&MyProc->waitLink);
MyProc->waitProcLock = NULL;
+ MemSet(&MyProc->futureWaitLock, 0, sizeof(FutureWaitLock));
pg_atomic_write_u64(&MyProc->waitStart, 0);
#ifdef USE_ASSERT_CHECKING
{
@@ -691,6 +692,7 @@ InitAuxiliaryProcess(void)
MyProc->waitLock = NULL;
dlist_node_init(&MyProc->waitLink);
MyProc->waitProcLock = NULL;
+ MemSet(&MyProc->futureWaitLock, 0, sizeof(FutureWaitLock));
pg_atomic_write_u64(&MyProc->waitStart, 0);
#ifdef USE_ASSERT_CHECKING
{
@@ -824,6 +826,7 @@ LockErrorCleanup(void)
HOLD_INTERRUPTS();
AbortStrongLockAcquire();
+ LockClearFutureWaitSlot(false);
/* Nothing to do if we weren't waiting for a lock */
lockAwaited = GetAwaitedLock();
@@ -934,6 +937,7 @@ ProcKill(int code, Datum arg)
/* Make sure we're out of the sync rep lists */
SyncRepCleanupAtProcExit();
+ LockClearFutureWaitSlot(false);
#ifdef USE_ASSERT_CHECKING
{
@@ -1824,6 +1828,17 @@ CheckDeadLock(void)
{
int i;
DeadLockState result;
+ LOCKTAG *futureLocks;
+ int futureLockCount = 0;
+
+retry:
+ /*
+ * Migrate fast-path holders for relation locks named by declared
+ * future-wait declarations. The helper takes and releases lock
+ * partitions internally while taking its snapshot, so it must run before
+ * this routine freezes the lock table for the real deadlock check.
+ */
+ futureLocks = MigrateFutureWaitFastPathLocks(&futureLockCount);
/*
* Acquire exclusive lock on the entire shared lock data structures. Must
@@ -1838,6 +1853,25 @@ CheckDeadLock(void)
for (i = 0; i < NUM_LOCK_PARTITIONS; i++)
LWLockAcquire(LockHashPartitionLockByIndex(i), LW_EXCLUSIVE);
+ /*
+ * A future-wait declaration can appear after the migration snapshot but
+ * before this partition-locked graph walk. If so, release the temporary
+ * strong-lock counts and retry, so the newly visible future edge cannot
+ * miss holders that are still in fast-path arrays.
+ *
+ * In theory a stream of concurrent declarations could force repeated
+ * retries, but in practice future-wait declarations are issued only by
+ * REPACK CONCURRENTLY and are rare, so the loop terminates quickly.
+ */
+ if (!FutureWaitFastPathSnapshotCoversCurrentLocks(futureLocks,
+ futureLockCount))
+ {
+ for (i = NUM_LOCK_PARTITIONS; --i >= 0;)
+ LWLockRelease(LockHashPartitionLockByIndex(i));
+ RestoreFutureWaitFastPathSnapshot(futureLocks, futureLockCount);
+ goto retry;
+ }
+
/*
* Check to see if we've been awoken by anyone in the interim.
*
@@ -1902,6 +1936,8 @@ check_done:
for (i = NUM_LOCK_PARTITIONS; --i >= 0;)
LWLockRelease(LockHashPartitionLockByIndex(i));
+ RestoreFutureWaitFastPathSnapshot(futureLocks, futureLockCount);
+
return result;
}
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index c3d8518cb62..648435eb618 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -102,6 +102,9 @@ extern void AtEOSubXact_on_commit_actions(bool isCommit,
extern void RangeVarCallbackMaintainsTable(const RangeVar *relation,
Oid relId, Oid oldRelId,
void *arg);
+extern void RangeVarCallbackForRepack(const RangeVar *relation,
+ Oid relId, Oid oldRelId,
+ void *arg);
extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 2a985ce5e15..b301edafb31 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -51,6 +51,8 @@ extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
extern bool CheckRelationOidLockedByMe(Oid relid, LOCKMODE lockmode,
bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
+extern void LockDeclareFutureWait(const LOCKTAG *locktag, LOCKMODE lockmode);
+extern void LockClearFutureWaitSlot(bool partitionLockHeld);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
extern void UnlockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ee3cb1dc203..dca5a3d33d6 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -275,6 +275,30 @@ typedef struct LOCALLOCK
#define LOCALLOCK_LOCKTAG(llock) ((LockTagType) (llock).tag.lock.locktag_type)
+/*
+ * Declared "future wait" for a lock the backend intends to acquire later.
+ *
+ * A process may publish its intent to acquire a particular lock mode on a
+ * particular locktag before actually calling LockAcquire(). The deadlock
+ * detector treats a populated slot as a hard waits-for edge from the
+ * declarer to every current holder whose mode conflicts with .mode. The
+ * slot is empty when locktag.locktag_lockmethodid == 0.
+ *
+ * Grouped in a struct so a future change can turn the slot into a short
+ * list without touching every reader.
+ */
+typedef struct FutureWaitLock
+{
+ LOCKTAG locktag; /* locktag_lockmethodid == 0 when empty */
+ LOCKMODE mode; /* mode the proc will eventually request */
+} FutureWaitLock;
+
+#define FutureWaitLockIsSet(futureWaitLock) \
+ ((futureWaitLock)->locktag.locktag_lockmethodid != 0)
+#define FutureWaitLockIsEmpty(futureWaitLock) \
+ (!FutureWaitLockIsSet(futureWaitLock))
+
+
/*
* These structures hold information passed from lmgr internals to the lock
* listing user-level functions (in lockfuncs.c).
@@ -419,6 +443,15 @@ extern LOCALLOCK *GetAwaitedLock(void);
extern void ResetAwaitedLock(void);
extern void RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode);
+
+extern LOCK *LockHashLookup(const LOCKTAG *locktag, uint32 hashcode);
+
+/* Future-wait fast-path migration, called by CheckDeadLock(). */
+extern LOCKTAG *MigrateFutureWaitFastPathLocks(int *out_count);
+extern bool FutureWaitFastPathSnapshotCoversCurrentLocks(LOCKTAG *locktags,
+ int n);
+extern void RestoreFutureWaitFastPathSnapshot(LOCKTAG *locktags, int n);
+
extern LockData *GetLockStatusData(void);
extern BlockedProcsData *GetBlockerStatusData(int blocked_pid);
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 3e1d1fad5f9..66bfce6f8b4 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -314,6 +314,15 @@ typedef struct PGPROC
LOCKMASK heldLocks; /* bitmask for lock types already held on this
* lock object by this backend */
+ /*
+ * Declared future-wait slot: a lock this proc intends to acquire later.
+ * Empty when .locktag.locktag_lockmethodid == 0. Publishing, clearing,
+ * and remote reads are protected by the partition lock of .locktag
+ * (deadlock walkers hold all partition locks and can safely inspect the
+ * slot). The owning backend may inspect its own slot locklessly.
+ */
+ FutureWaitLock futureWaitLock;
+
pg_atomic_uint64 waitStart; /* time at which wait for lock acquisition
* started */
diff --git a/src/test/modules/injection_points/expected/repack.out b/src/test/modules/injection_points/expected/repack.out
index b575e9052ee..22c051f702e 100644
--- a/src/test/modules/injection_points/expected/repack.out
+++ b/src/test/modules/injection_points/expected/repack.out
@@ -1,4 +1,4 @@
-Parsed test spec with 2 sessions
+Parsed test spec with 4 sessions
starting permutation: wait_before_lock change_existing change_new change_subxact1 change_subxact2 check2 wakeup_before_lock check1
injection_points_attach
@@ -111,3 +111,182 @@ injection_points_detach
(1 row)
+
+starting permutation: wait_before_lock begin_txn lock_table_ae s3_wakeup end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_txn:
+ BEGIN;
+
+step lock_table_ae:
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+ <waiting ...>
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step lock_table_ae: <... completed>
+step end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s2_timeout_fast wait_before_lock begin_and_read lock_table_ae end_txn s3_wakeup
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s2_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step lock_table_ae:
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+ <waiting ...>
+step lock_table_ae: <... completed>
+ERROR: future deadlock detected
+step end_txn:
+ COMMIT;
+
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s2_timeout_fast wait_before_lock begin_and_read lock_table_sue end_txn s3_wakeup
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s2_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step lock_table_sue:
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+ <waiting ...>
+step lock_table_sue: <... completed>
+ERROR: future deadlock detected
+step end_txn:
+ COMMIT;
+
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s2_timeout_fast wait_before_lock begin_and_read s3_begin_txn s3_lock_table_y_sue s3_lock_table_sue lock_table_y_sue end_txn s4_wakeup s3_end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s2_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step s3_begin_txn:
+ BEGIN;
+
+step s3_lock_table_y_sue:
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+
+step s3_lock_table_sue:
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+ <waiting ...>
+step lock_table_y_sue:
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+ <waiting ...>
+step lock_table_y_sue: <... completed>
+ERROR: future deadlock detected
+step end_txn:
+ COMMIT;
+
+step s4_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step s3_lock_table_sue: <... completed>
+step s3_end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index d727a9b056b..d214821a2da 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -5,6 +5,7 @@ setup
CREATE TABLE repack_test(i int PRIMARY KEY, j int);
INSERT INTO repack_test(i, j) VALUES (1, 1), (2, 2), (3, 3), (4, 4);
+ CREATE TABLE repack_test_y(i int);
CREATE TABLE relfilenodes(node oid);
@@ -15,6 +16,7 @@ setup
teardown
{
DROP TABLE repack_test;
+ DROP TABLE repack_test_y;
DROP EXTENSION injection_points;
DROP TABLE relfilenodes;
@@ -61,6 +63,10 @@ teardown
}
session s2
+step s2_timeout_fast
+{
+ SET deadlock_timeout = '10ms';
+}
# Change the existing data. UPDATE changes both key and non-key columns. Also
# update one row twice to test whether tuple version generated by this session
# can be found.
@@ -128,6 +134,60 @@ step wakeup_before_lock
{
SELECT injection_points_wakeup('repack-concurrently-before-lock');
}
+# Steps used in lock contention tests.
+step begin_txn
+{
+ BEGIN;
+}
+step begin_and_read
+{
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+}
+step lock_table_ae
+{
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+}
+step lock_table_sue
+{
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step lock_table_y_sue
+{
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step end_txn
+{
+ COMMIT;
+}
+
+session s3
+step s3_begin_txn
+{
+ BEGIN;
+}
+step s3_lock_table_y_sue
+{
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step s3_lock_table_sue
+{
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step s3_wakeup
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
+step s3_end_txn
+{
+ COMMIT;
+}
+
+session s4
+step s4_wakeup
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
# Test if data changes introduced while one session is performing REPACK
# CONCURRENTLY find their way into the table.
@@ -140,3 +200,61 @@ permutation
check2
wakeup_before_lock
check1
+
+# A waiter that does not already hold a conflicting lock on the table is not a
+# future deadlock. It waits until REPACK finishes and then acquires its lock.
+permutation
+ wait_before_lock
+ begin_txn
+ lock_table_ae
+ s3_wakeup
+ end_txn
+
+# In the deadlock-expecting permutations below, all deadlock detections must
+# run while s1 is still parked at the injection point, i.e. before s1 wakes
+# up and attempts AccessExclusiveLock. Once s1 attaches waitLink for AEL,
+# it clears its future-wait slot, and any cycle from that point on would be
+# reported as a plain "deadlock detected" rather than "future deadlock
+# detected". To force this ordering, every wakeup step is placed after the
+# COMMIT from the session whose 10 ms deadlock_timeout we rely on: the
+# framework cannot run that COMMIT until the blocked session has unblocked
+# (via its deadlock check), so the wakeup necessarily follows.
+
+# A waiter that already holds AccessShareLock then waits for AccessExclusiveLock
+# behind REPACK's ShareUpdateExclusiveLock.
+permutation
+ s2_timeout_fast
+ wait_before_lock
+ begin_and_read
+ lock_table_ae(*)
+ end_txn
+ s3_wakeup
+
+# Same shape as above, but the waiter requests ShareUpdateExclusiveLock.
+permutation
+ s2_timeout_fast
+ wait_before_lock
+ begin_and_read
+ lock_table_sue(*)
+ end_txn
+ s3_wakeup
+
+# Three-backend future deadlock:
+#
+# - s1 holds ShareUpdateExclusiveLock on repack_test and has declared a future
+# AccessExclusiveLock on it.
+# - s2 holds AccessShareLock on repack_test.
+# - s3 holds ShareUpdateExclusiveLock on repack_test_y, then waits for
+# ShareUpdateExclusiveLock on repack_test behind s1.
+# - s2 waits for ShareUpdateExclusiveLock on repack_test_y behind s3.
+permutation
+ s2_timeout_fast
+ wait_before_lock
+ begin_and_read
+ s3_begin_txn
+ s3_lock_table_y_sue
+ s3_lock_table_sue
+ lock_table_y_sue(*)
+ end_txn
+ s4_wakeup
+ s3_end_txn
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-18 19:23 ` Antonin Houska <ah@cybertec.at>
2026-04-18 22:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-19 11:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-18 19:23 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> I think I got working POC for deadlock-detector enhancements for
> REPACK (and potentially other).
That looks interesting, I'll check it.
I've also thought about the problem quite a bit this week. I tried to add a
pointer to PGPROC that, like ->waitLock, points to the lock being acquired,
but it's initialized before the actual waiting starts. I adjusted the deadlock
detector to use that pointer too, but it did not work. The problem was
probably that the lock wasn't in the queue during the check.
Finally it occurred to me that a new field can be added to the LOCK structure,
indicating that the lock is being upgraded. It enforces some extra deadlock
checks by other processes, so that the upgrading process does not have to care
about deadlock detection at all. More info in the commit message.
It should handle all the cases in your tests, however a new injection point
would be needed. (Not added yet.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-18 19:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-18 22:46 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-20 17:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-18 22:46 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
I've briefly looked at your patch. As I understand it, it cancels the
other process only when REPACK actually tries to upgrade the lock. In
that sense, its approach is close to my variant at [0].
AFAIU, Andres's concern is that the "victim" should be cancelled
sooner, rather than waiting until REPACK actually attempts the
upgrade. I was trying to solve it by
[1].
-------------------------------
There are some comments on [1]:
Some details related to POC from previous letter (once I re-read that
in the morning I relized it is not so easy to understand, sorry).
So, goal of patch to:
* make sure REPACK will survive deadlock which may caused by lock upgrade
* reject some other backend with error early than deadlock really
occur - to avoid pointless waiting for other commands
* implement it at deadlock detector level to correctly handle
different 2+ backend-involved scenarious
To achive it the first idea is to add some kind of "future lock"
(FutureWaitLock). It may declare an intention to acquire a lock of a
certain level as high-priority in the future.
Once deadlock detector starts to walking graph it treat that intention
like an actual "waiting".
That way deadlock detector looks for one more step in the future -
moment of actual acquiring the "future" lock - and if it ends with
cycle - reject waiting backend ("future deadlock detected").
Looks like best place to put that FutureWaitLock is PGPROC itself.
Few moments to consider:
* it is not allowed to declare a future lock if backend already holds
some kind of lock for the same tag (this may cause a race condition).
* so, REPACK first declares future Access Excluvie and only after it
Share Update Exclusive
* declared future lock should be >= SUE
So far everything looks good.
In case of that scenario:
S1: BEGIN; SELECT * FROM t;
S2: REPACK (CONCURRENTLY) t;
S1: LOCK TABLE t in ACCESS EXCLUSIVE MODE <--- "future deadlock detected"
Deadlock detector sees:
S1 ----> S2 (waiting for AE conflicting with SUE)
S2 ----> S1 (future wait for AE conflict with current Access Share)
But there is a tricky case related to SUE:
S1: BEGIN; SELECT * FROM t;
S2: REPACK (CONCURRENTLY) t;
S1: LOCK TABLE t in SHARE UPDATE EXCLUSIVE MODE;
In that case we have almost the same deadlock scenario - but that is
not visible to deadlock detector.
It happens because SUE does not force all locks taken on 't' to be
transfered using FastPathTransferRelationLocks into the main table
(SUE is does not ConflictsWithRelationFastPath).
Because of it S2 -> S1 edge is not visible by deadlock detector
(Access Share is held using fast-path).
To deal with it we may force any relation with FutureWaitLock to
through slow-path locking - but I don't think it is acceptable.
Instead next approach is proposed:
* deadlock detector checks if any "future" locks are present in the
system (counter in shared memory)
* if so - it iterates over all PROCs to collect relations which are
"future locked"
* for each such relations - FastPathTransferRelationLocks called and
slow-path is forced (FastPathStrongRelationLocks->count)
* deadlock detector start looking for cycles
* once ready - FastPathStrongRelationLocks->count is decremented to
allow fast-path
That way performance degradation happens only during deadlock detector
processing and only if some future locks present.
Due tue LW ordering we need to use some tricks to avoid LW-level
deadlocks (using some kind of retry logic, but that is more explained
in the patch).
[0]: https://www.postgresql.org/message-id/flat/CADzfLwURKVNQ%2B%2BDpi7bjoGfj-8pchDQEVex3eWBx0NCYn6TbDQ%4....
[1}] postgresql.org/message-id/flat/CADzfLwU8Qw6LXFHO7Tbjc-O7o%2BtM26jdnOJBWqYLu61rf7bO%2Bg%40mail.gmail.com#1e96f8882363afb2fc53c2f08346f527
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-18 19:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-18 22:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-20 17:44 ` Antonin Houska <ah@cybertec.at>
2026-04-23 11:43 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-20 17:44 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> I've briefly looked at your patch. As I understand it, it cancels the other
> process only when REPACK actually tries to upgrade the lock. ...
>
> AFAIU, Andres's concern is that the "victim" should be cancelled
> sooner, rather than waiting until REPACK actually attempts the
> upgrade.
I thought the point is that the deadlock should be resolved in a controlled
way, i.e. w/o relying on deadlock timeout. Once both processes sleep, the
decision which one should be kicked off is effectively random. In other words,
the actual deadlock IMO starts exactly at the moment both processes end up
sleeping. But I may be wrong.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-18 19:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-18 22:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-20 17:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-23 11:43 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-26 13:34 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-23 11:43 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Mon, Apr 20, 2026 at 7:44 PM Antonin Houska <ah@cybertec.at> wrote:
> When another process tries to get the lock after that, it checks this field,
> and if it's set, it checks for deadlocks even if deadlock timeout hasn't
> expired yet. If the process is already in the lock's queue and sleeping, the
> lock upgrading process wakes it up so it checks the flag immediately.
In any way there is no sense in waking up other backends just to force
them to cancel themselves.
Better to go in the [0] way - and just cancel another backend if we
are repack and found cycle in the deadlock detector.
We currently have all required infrastructure, even as the comment
described the possibility [1]:
> * Get this process out of wait state. (Note: we could do this more
> * efficiently by relying on lockAwaited, but use this coding to
> * preserve the flexibility to kill some other transaction than the
> * one detecting the deadlock.)
At the same time version [2] (with FutureWaitLock, explained in more
detail in [3]) is correct and cancels other backends without pointless
waiting, but it feels too complicated due to the complexity of
supporting the FastLock path.
Also, here's one simple idea inspired by your version.
What about adding a new field "do not try to upgrade that lock" to the
LOCK structure? If some backends try to (it has some lockmode and
tries to upgrade it) and it does not has flag 'deadlock_protected'
from [0] - just fail fast with "future deadlock detected".
That way [0] ensures correctness and "do not try to upgrade that lock"
just optimization to fail quickly for the most common scenarios.
Some 2+ backend loops might still wait a long time to be cancelled
(which is solved by [2], but complicated) - but I think this is a
super rare case we can ignore (because repack is still protected from
deadlock).
Mikhail.
[0]: https://www.postgresql.org/message-id/flat/CADzfLwURKVNQ%2B%2BDpi7bjoGfj-8pchDQEVex3eWBx0NCYn6TbDQ%4....
[1]: https://github.com/postgres/postgres/blob/master/src/backend/storage/lmgr/proc.c#L1870-L1873
[2]: https://www.postgresql.org/message-id/flat/CADzfLwUSnGnkfLwCWHQ%3DVVuAY1YTo%2B0Lr7pb%2BOPWUZbcYKSRUw...
[3]: https://www.postgresql.org/message-id/flat/CADzfLwVf-3mjMwSTOcj9djNzGd-UjBOYbFjxgXRhtKuH_4rajA%40mai...
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-18 19:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-18 22:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-20 17:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-23 11:43 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-26 13:34 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-26 13:34 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
I think I have a good enough approach now (at least balancing
complexity and outcome).
Patch (and commit message) is quite explanatory, but in a few words:
- add 'upgradeIntent' to PROCLOCK (set by REPACK)
- check that in the deadlock detector. If the backend finds the cycle
and is part of it, but because it's upgrading an already announced
lock, it cancels another backend instead of itself.
- use that in the fast path of simple deadlock detection to avoid
pointless waiting (for the easy case involving two backends)
It doesn't cover all scenarios (explained in patch details) but for
majority of realistic scenarios - yes.
It may be extended to cover all of them, but I'm not sure it's worth
the additional complexity.
Best regards,
Mikhail.
Attachments:
[application/octet-stream] nocfbot-v2-0001-Protect-concurrent-repack-lock-upgrades.patch (53.1K, ../../CADzfLwXbUWuS6H4uJEFVL1jS1kzsVnuJ+zX1+tAEhQxBnEiGKw@mail.gmail.com/2-nocfbot-v2-0001-Protect-concurrent-repack-lock-upgrades.patch)
download | inline diff:
From 0717db080b079d0242fc0c3826f1284d793911fd Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 26 Apr 2026 11:50:34 +0200
Subject: [PATCH v2] Protect concurrent repack lock upgrades
REPACK CONCURRENTLY loses significant work when its final ShareUpdateExclusiveLock to AccessExclusiveLock upgrade is canceled by a deadlock. This commit introduces "protected lock upgrades", allowing a backend to atomically record a future upgrade intention (upgradeIntent) on its PROCLOCK via LockAcquireExtended. When a deadlock cycle involves a protected backend's announced upgrade, the detector preempts a blocking waiter rather than the protected upgrader.
Preemption triggers via two paths. JoinWaitQueue fast-fails incoming requests if a holder's upgradeIntent and current modes guarantee a future cycle. Alternatively, if DeadLockCheck evaluates a cycle where the waiter is a protected upgrader, it selects a blocking waiter as the cancellation victim.
---
src/backend/catalog/namespace.c | 32 +++-
src/backend/commands/repack.c | 29 ++-
src/backend/storage/lmgr/README | 26 +++
src/backend/storage/lmgr/deadlock.c | 90 ++++++++-
src/backend/storage/lmgr/lmgr.c | 49 +++--
src/backend/storage/lmgr/lock.c | 154 +++++++++++++--
src/backend/storage/lmgr/proc.c | 134 ++++++++++---
src/include/catalog/namespace.h | 7 +
src/include/storage/lmgr.h | 3 +
src/include/storage/lock.h | 19 +-
src/include/storage/proc.h | 7 -
.../injection_points/expected/repack.out | 181 +++++++++++++++++-
.../injection_points/specs/repack.spec | 108 +++++++++++
13 files changed, 757 insertions(+), 82 deletions(-)
diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index 56b87d878e8..31a958198e0 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -415,8 +415,18 @@ spcache_insert(const char *searchPath, Oid roleid)
}
}
+/* Wrapper preserving the historical signature (no upgrade intent). */
+Oid
+RangeVarGetRelidExtended(const RangeVar *relation, LOCKMODE lockmode,
+ uint32 flags,
+ RangeVarGetRelidCallback callback, void *callback_arg)
+{
+ return RangeVarGetRelidWithUpgradeIntent(relation, lockmode, NoLock,
+ flags, callback, callback_arg);
+}
+
/*
- * RangeVarGetRelidExtended
+ * RangeVarGetRelidWithUpgradeIntent
* Given a RangeVar describing an existing relation,
* select the proper namespace and look up the relation OID.
*
@@ -435,13 +445,17 @@ spcache_insert(const char *searchPath, Oid roleid)
* return value of InvalidOid could either mean the relation is missing or it
* could not be locked.
*
+ * If upgradeMode is not NoLock, an upgrade-intent announcement is installed
+ * atomically with the grant; see LockAcquireExtended.
+ *
* Callback allows caller to check permissions or acquire additional locks
* prior to grabbing the relation lock.
*/
Oid
-RangeVarGetRelidExtended(const RangeVar *relation, LOCKMODE lockmode,
- uint32 flags,
- RangeVarGetRelidCallback callback, void *callback_arg)
+RangeVarGetRelidWithUpgradeIntent(const RangeVar *relation, LOCKMODE lockmode,
+ LOCKMODE upgradeMode, uint32 flags,
+ RangeVarGetRelidCallback callback,
+ void *callback_arg)
{
uint64 inval_count;
Oid relId;
@@ -451,6 +465,9 @@ RangeVarGetRelidExtended(const RangeVar *relation, LOCKMODE lockmode,
/* verify that flags do no conflict */
Assert(!((flags & RVR_NOWAIT) && (flags & RVR_SKIP_LOCKED)));
+ /* Upgrade intent requires the blocking acquisition path. */
+ Assert(upgradeMode == NoLock ||
+ !(flags & (RVR_NOWAIT | RVR_SKIP_LOCKED)));
/*
* We check the catalog name and then ignore it.
@@ -590,7 +607,12 @@ RangeVarGetRelidExtended(const RangeVar *relation, LOCKMODE lockmode,
if (!OidIsValid(relId))
AcceptInvalidationMessages();
else if (!(flags & (RVR_NOWAIT | RVR_SKIP_LOCKED)))
- LockRelationOid(relId, lockmode);
+ {
+ if (upgradeMode != NoLock)
+ LockRelationOidWithUpgradeIntent(relId, lockmode, upgradeMode);
+ else
+ LockRelationOid(relId, lockmode);
+ }
else if (!ConditionalLockRelationOid(relId, lockmode))
{
int elevel = (flags & RVR_SKIP_LOCKED) ? DEBUG1 : ERROR;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index bafdca80810..c63d2107ef2 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -2322,12 +2322,25 @@ process_single_relation(RepackStmt *stmt, LOCKMODE lockmode, bool isTopLevel,
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ANALYZE option must be specified when a column list is provided"));
- /* Find, lock, and check permissions on the table. */
- tableOid = RangeVarGetRelidExtended(stmt->relation->relation,
- lockmode,
- 0,
- RangeVarCallbackMaintainsTable,
- NULL);
+ /*
+ * Find, lock, and check permissions on the table.
+ *
+ * For CONCURRENTLY, announce the future AEL upgrade so a conflicting
+ * holder is preempted instead of stalling us through the copy phase.
+ */
+ if ((params->options & CLUOPT_CONCURRENT) != 0)
+ tableOid = RangeVarGetRelidWithUpgradeIntent(stmt->relation->relation,
+ lockmode,
+ AccessExclusiveLock,
+ 0,
+ RangeVarCallbackMaintainsTable,
+ NULL);
+ else
+ tableOid = RangeVarGetRelidExtended(stmt->relation->relation,
+ lockmode,
+ 0,
+ RangeVarCallbackMaintainsTable,
+ NULL);
rel = table_open(tableOid, NoLock);
/*
@@ -3055,6 +3068,10 @@ rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
/*
* Acquire AccessExclusiveLock on the table, its TOAST relation (if there
* is one), all its indexes, so that we can swap the files.
+ *
+ * The upgrade intent announced at initial SUEL acquisition makes the
+ * detector preempt a blocker on a deadlock here, preserving the work
+ * already done.
*/
LockRelationOid(old_table_oid, AccessExclusiveLock);
diff --git a/src/backend/storage/lmgr/README b/src/backend/storage/lmgr/README
index 45de0fd2bd6..ced7d285279 100644
--- a/src/backend/storage/lmgr/README
+++ b/src/backend/storage/lmgr/README
@@ -586,6 +586,32 @@ The caller can then send a cancellation signal. This implements the
principle that autovacuum has a low locking priority (eg it must not block
DDL on the table).
+Protected Lock Upgrades
+-----------------------
+
+A backend can announce at initial-lock acquisition that it intends to upgrade
+to a stronger mode later (PROCLOCK->upgradeIntent), installed atomically with
+the grant via LockAcquireExtended. REPACK CONCURRENTLY uses this so that a
+deadlock on its final SUEL->AEL swap cancels a blocker rather than throwing
+away the copy/rebuild phase.
+
+When a cycle involving the protected backend is detected, a blocking waiter is
+canceled instead of the protected backend. This can happen either in
+JoinWaitQueue's fast-fail check or in the full DeadLockCheck() during the
+announced upgrade wait. The fast-fail check is only a cheap early test for
+cycles already visible in the main lock table. It does not force transfer of
+unrelated fast-path relation locks, so it may miss cycles whose edges are still
+hidden in backend-local fast-path state. Such cycles become visible later when a
+conflicting strong lock request transfers the relevant fast-path locks.
+The full deadlock detector handles that case.
+
+Other protected upgraders and anti-wraparound autovacuums are skipped as
+victims. If no acceptable victim exists, the protected backend is canceled as
+usual.
+
+The victim observes preemption via PROC_WAIT_STATUS_PREEMPTED on its
+waitStatus, which LockAcquireExtended turns into a deadlock ereport.
+
Group Locking
-------------
diff --git a/src/backend/storage/lmgr/deadlock.c b/src/backend/storage/lmgr/deadlock.c
index b8962d875b6..35143f0ab1e 100644
--- a/src/backend/storage/lmgr/deadlock.c
+++ b/src/backend/storage/lmgr/deadlock.c
@@ -125,6 +125,13 @@ static int maxPossibleConstraints;
static DEADLOCK_INFO *deadlockDetails;
static int nDeadlockDetails;
+/*
+ * Parallel to deadlockDetails[]: live PGPROCs forming each edge of the
+ * detected cycle. Valid only while all partition locks are held. Used by
+ * DeadLockCheck to pick a victim under the protected-upgrade rule.
+ */
+static PGPROC **deadlockProcs;
+
/* PGPROC pointer of any blocking autovacuum worker found */
static PGPROC *blocking_autovacuum_proc = NULL;
@@ -148,11 +155,12 @@ InitDeadLockChecking(void)
oldcxt = MemoryContextSwitchTo(TopMemoryContext);
/*
- * FindLockCycle needs at most MaxBackends entries in visitedProcs[] and
- * deadlockDetails[].
+ * FindLockCycle needs at most MaxBackends entries in visitedProcs[],
+ * deadlockDetails[], and deadlockProcs[].
*/
visitedProcs = (PGPROC **) palloc(MaxBackends * sizeof(PGPROC *));
deadlockDetails = (DEADLOCK_INFO *) palloc(MaxBackends * sizeof(DEADLOCK_INFO));
+ deadlockProcs = (PGPROC **) palloc(MaxBackends * sizeof(PGPROC *));
/*
* TopoSort needs to consider at most MaxBackends wait-queue entries, and
@@ -219,10 +227,14 @@ InitDeadLockChecking(void)
DeadLockState
DeadLockCheck(PGPROC *proc)
{
+ bool victims_canceled = false;
+
+retry:
/* Initialize to "no constraints" */
nCurConstraints = 0;
nPossibleConstraints = 0;
nWaitOrders = 0;
+ nDeadlockDetails = 0;
/* Initialize to not blocked by an autovacuum worker */
blocking_autovacuum_proc = NULL;
@@ -242,6 +254,72 @@ DeadLockCheck(PGPROC *proc)
if (!FindLockCycle(proc, possibleConstraints, &nSoftEdges))
elog(FATAL, "deadlock seems to have disappeared");
+ /*
+ * Protected-upgrade resolution: if our wait is the announced upgrade
+ * for our proclock, cancel a blocker instead of MyProc and re-run the
+ * full check (not just FindLockCycle) so soft cycles still get a
+ * chance at queue rearrangement. Partition locks are held throughout,
+ * so the loop is bounded by MaxBackends.
+ *
+ * deadlockProcs[0] is always MyProc; start at the direct blocker.
+ * Skip other protected upgraders (don't preempt them) and
+ * anti-wraparound autovacuums (canceling them is worse than losing
+ * our work). If no acceptable victim exists, fall through to
+ * DS_HARD_DEADLOCK.
+ */
+ if (MyProc->waitProcLock != NULL &&
+ MyProc->waitProcLock->upgradeIntent == MyProc->waitLockMode)
+ {
+ PGPROC *victim = NULL;
+
+ for (int i = 1; i < nDeadlockDetails; i++)
+ {
+ PGPROC *w = deadlockProcs[i];
+
+ if (w == NULL || w == MyProc)
+ continue;
+
+ /* Don't preempt another protected upgrader. */
+ if (w->waitProcLock != NULL &&
+ w->waitProcLock->upgradeIntent == w->waitLockMode)
+ continue;
+
+ /*
+ * PROC_IS_AUTOVACUUM is set once at startup, safe lockless.
+ * Read PROC_VACUUM_FOR_WRAPAROUND only if we can grab
+ * ProcArrayLock conditionally; otherwise skip this candidate.
+ */
+ if (w->statusFlags & PROC_IS_AUTOVACUUM)
+ {
+ uint8 statusFlags;
+
+ if (!LWLockConditionalAcquire(ProcArrayLock, LW_SHARED))
+ continue;
+
+ statusFlags = ProcGlobal->statusFlags[w->pgxactoff];
+ LWLockRelease(ProcArrayLock);
+
+ if (statusFlags & PROC_VACUUM_FOR_WRAPAROUND)
+ continue;
+ }
+
+ Assert(w->waitLock != NULL);
+ Assert(!dlist_node_is_detached(&w->waitLink));
+ victim = w;
+ break;
+ }
+
+ if (victim != NULL)
+ {
+ RemoveFromWaitQueue(victim,
+ LockTagHashCode(&(victim->waitLock->tag)),
+ PROC_WAIT_STATUS_PREEMPTED);
+ SetLatch(&victim->procLatch);
+ victims_canceled = true;
+ goto retry;
+ }
+ }
+
return DS_HARD_DEADLOCK; /* cannot find a non-deadlocked state */
}
@@ -273,7 +351,9 @@ DeadLockCheck(PGPROC *proc)
}
/* Return code tells caller if we had to escape a deadlock or not */
- if (nWaitOrders > 0)
+ if (victims_canceled)
+ return DS_PREEMPT_DEADLOCK;
+ else if (nWaitOrders > 0)
return DS_SOFT_DEADLOCK;
else if (blocking_autovacuum_proc != NULL)
return DS_BLOCKED_BY_AUTOVACUUM;
@@ -590,6 +670,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ deadlockProcs[depth] = checkProc;
return true;
}
@@ -679,6 +760,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ deadlockProcs[depth] = checkProc;
/*
* Add this edge to the list of soft edges in the cycle
@@ -753,6 +835,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc,
info->locktag = lock->tag;
info->lockmode = checkProc->waitLockMode;
info->pid = checkProc->pid;
+ deadlockProcs[depth] = checkProc;
/*
* Add this edge to the list of soft edges in the cycle
@@ -1159,4 +1242,5 @@ RememberSimpleDeadLock(PGPROC *proc1,
info->lockmode = proc2->waitLockMode;
info->pid = proc2->pid;
nDeadlockDetails = 2;
+ deadlockProcs[0] = deadlockProcs[1] = NULL;
}
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index 2ccf7237fee..10b0e0a5b9a 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -105,6 +105,19 @@ SetLocktagRelationOid(LOCKTAG *tag, Oid relid)
*/
void
LockRelationOid(Oid relid, LOCKMODE lockmode)
+{
+ LockRelationOidWithUpgradeIntent(relid, lockmode, NoLock);
+}
+
+/*
+ * LockRelationOidWithUpgradeIntent
+ *
+ * Lock and atomically announce a future upgrade to `upgradeMode`. See
+ * LockAcquireExtended for semantics.
+ */
+void
+LockRelationOidWithUpgradeIntent(Oid relid, LOCKMODE lockmode,
+ LOCKMODE upgradeMode)
{
LOCKTAG tag;
LOCALLOCK *locallock;
@@ -112,8 +125,8 @@ LockRelationOid(Oid relid, LOCKMODE lockmode)
SetLocktagRelationOid(&tag, relid);
- res = LockAcquireExtended(&tag, lockmode, false, false, true, &locallock,
- false);
+ res = LockAcquireExtended(&tag, lockmode, upgradeMode, false, false, true,
+ &locallock, false);
/*
* Now that we have the lock, check for invalidation messages, so that we
@@ -156,8 +169,8 @@ ConditionalLockRelationOid(Oid relid, LOCKMODE lockmode)
SetLocktagRelationOid(&tag, relid);
- res = LockAcquireExtended(&tag, lockmode, false, true, true, &locallock,
- false);
+ res = LockAcquireExtended(&tag, lockmode, NoLock, false, true, true,
+ &locallock, false);
if (res == LOCKACQUIRE_NOT_AVAIL)
return false;
@@ -190,8 +203,8 @@ LockRelationId(LockRelId *relid, LOCKMODE lockmode)
SET_LOCKTAG_RELATION(tag, relid->dbId, relid->relId);
- res = LockAcquireExtended(&tag, lockmode, false, false, true, &locallock,
- false);
+ res = LockAcquireExtended(&tag, lockmode, NoLock, false, false, true,
+ &locallock, false);
/*
* Now that we have the lock, check for invalidation messages; see notes
@@ -253,8 +266,8 @@ LockRelation(Relation relation, LOCKMODE lockmode)
relation->rd_lockInfo.lockRelId.dbId,
relation->rd_lockInfo.lockRelId.relId);
- res = LockAcquireExtended(&tag, lockmode, false, false, true, &locallock,
- false);
+ res = LockAcquireExtended(&tag, lockmode, NoLock, false, false, true,
+ &locallock, false);
/*
* Now that we have the lock, check for invalidation messages; see notes
@@ -285,8 +298,8 @@ ConditionalLockRelation(Relation relation, LOCKMODE lockmode)
relation->rd_lockInfo.lockRelId.dbId,
relation->rd_lockInfo.lockRelId.relId);
- res = LockAcquireExtended(&tag, lockmode, false, true, true, &locallock,
- false);
+ res = LockAcquireExtended(&tag, lockmode, NoLock, false, true, true,
+ &locallock, false);
if (res == LOCKACQUIRE_NOT_AVAIL)
return false;
@@ -590,8 +603,8 @@ ConditionalLockTuple(Relation relation, const ItemPointerData *tid, LOCKMODE loc
ItemPointerGetBlockNumber(tid),
ItemPointerGetOffsetNumber(tid));
- return (LockAcquireExtended(&tag, lockmode, false, true, true, NULL,
- logLockFailure) != LOCKACQUIRE_NOT_AVAIL);
+ return (LockAcquireExtended(&tag, lockmode, NoLock, false, true, true,
+ NULL, logLockFailure) != LOCKACQUIRE_NOT_AVAIL);
}
/*
@@ -748,8 +761,8 @@ ConditionalXactLockTableWait(TransactionId xid, bool logLockFailure)
SET_LOCKTAG_TRANSACTION(tag, xid);
- if (LockAcquireExtended(&tag, ShareLock, false, true, true, NULL,
- logLockFailure)
+ if (LockAcquireExtended(&tag, ShareLock, NoLock, false, true, true,
+ NULL, logLockFailure)
== LOCKACQUIRE_NOT_AVAIL)
return false;
@@ -1042,8 +1055,8 @@ ConditionalLockDatabaseObject(Oid classid, Oid objid, uint16 objsubid,
objid,
objsubid);
- res = LockAcquireExtended(&tag, lockmode, false, true, true, &locallock,
- false);
+ res = LockAcquireExtended(&tag, lockmode, NoLock, false, true, true,
+ &locallock, false);
if (res == LOCKACQUIRE_NOT_AVAIL)
return false;
@@ -1122,8 +1135,8 @@ ConditionalLockSharedObject(Oid classid, Oid objid, uint16 objsubid,
objid,
objsubid);
- res = LockAcquireExtended(&tag, lockmode, false, true, true, &locallock,
- false);
+ res = LockAcquireExtended(&tag, lockmode, NoLock, false, true, true,
+ &locallock, false);
if (res == LOCKACQUIRE_NOT_AVAIL)
return false;
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 8d246ed5a4e..0e666a025ef 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -59,6 +59,9 @@ bool log_lock_failures = false;
#define NLOCKENTS() \
mul_size(max_locks_per_xact, add_size(MaxBackends, max_prepared_xacts))
+#define LOCKMODE_SELF_CONFLICTS(conflictTab, lockmode) \
+ (((conflictTab)[(lockmode)] & LOCKBIT_ON(lockmode)) != 0)
+
/*
* Data structures defining the semantics of the standard lock methods.
@@ -384,7 +387,7 @@ LOCK_PRINT(const char *where, const LOCK *lock, LOCKMODE type)
elog(LOG,
"%s: lock(%p) id(%u,%u,%u,%u,%u,%u) grantMask(%x) "
"req(%d,%d,%d,%d,%d,%d,%d)=%d "
- "grant(%d,%d,%d,%d,%d,%d,%d)=%d wait(%d) type(%s)",
+ "grant(%d,%d,%d,%d,%d,%d,%d)=%d wait(%d) upgradeIntent(%d) type(%s)",
where, lock,
lock->tag.locktag_field1, lock->tag.locktag_field2,
lock->tag.locktag_field3, lock->tag.locktag_field4,
@@ -397,6 +400,7 @@ LOCK_PRINT(const char *where, const LOCK *lock, LOCKMODE type)
lock->granted[4], lock->granted[5], lock->granted[6],
lock->granted[7], lock->nGranted,
dclist_count(&lock->waitProcs),
+ lock->nUpgradeIntent,
LockMethods[LOCK_LOCKMETHOD(*lock)]->lockModeNames[type]);
}
@@ -406,10 +410,11 @@ PROCLOCK_PRINT(const char *where, const PROCLOCK *proclockP)
{
if (LOCK_DEBUG_ENABLED(&proclockP->tag.myLock->tag))
elog(LOG,
- "%s: proclock(%p) lock(%p) method(%u) proc(%p) hold(%x)",
+ "%s: proclock(%p) lock(%p) method(%u) proc(%p) hold(%x) upgradeIntent(%d)",
where, proclockP, proclockP->tag.myLock,
PROCLOCK_LOCKMETHOD(*(proclockP)),
- proclockP->tag.myProc, (int) proclockP->holdMask);
+ proclockP->tag.myProc, (int) proclockP->holdMask,
+ proclockP->upgradeIntent);
}
#else /* not LOCK_DEBUG */
@@ -427,6 +432,7 @@ static void BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode);
static void FinishStrongLockAcquire(void);
static ProcWaitStatus WaitOnLock(LOCALLOCK *locallock, ResourceOwner owner);
static void waitonlock_error_callback(void *arg);
+pg_noreturn static void ReportPreemptDeadlock(LOCALLOCK *locallock);
static void ReleaseLockIfHeld(LOCALLOCK *locallock, bool sessionLock);
static void LockReassignOwner(LOCALLOCK *locallock, ResourceOwner parent);
static bool UnGrantLock(LOCK *lock, LOCKMODE lockmode,
@@ -808,13 +814,17 @@ LockAcquire(const LOCKTAG *locktag,
bool sessionLock,
bool dontWait)
{
- return LockAcquireExtended(locktag, lockmode, sessionLock, dontWait,
- true, NULL, false);
+ return LockAcquireExtended(locktag, lockmode, NoLock, sessionLock,
+ dontWait, true, NULL, false);
}
/*
* LockAcquireExtended - allows us to specify additional options
*
+ * upgradeIntent is NoLock or a stronger mode the caller will try to acquire
+ * later; when set, it is installed on the PROCLOCK atomically with the grant.
+ * See README "Protected Lock Upgrades".
+ *
* reportMemoryError specifies whether a lock request that fills the lock
* table should generate an ERROR or not. Passing "false" allows the caller
* to attempt to recover from lock-table-full situations, perhaps by forcibly
@@ -832,6 +842,7 @@ LockAcquire(const LOCKTAG *locktag,
LockAcquireResult
LockAcquireExtended(const LOCKTAG *locktag,
LOCKMODE lockmode,
+ LOCKMODE upgradeIntent,
bool sessionLock,
bool dontWait,
bool reportMemoryError,
@@ -851,6 +862,7 @@ LockAcquireExtended(const LOCKTAG *locktag,
bool found_conflict;
ProcWaitStatus waitResult;
bool log_lock = false;
+ bool upgradeIntentSet = false;
if (lockmethodid <= 0 || lockmethodid >= lengthof(LockMethods))
elog(ERROR, "unrecognized lock method: %d", lockmethodid);
@@ -858,6 +870,16 @@ LockAcquireExtended(const LOCKTAG *locktag,
if (lockmode <= 0 || lockmode > lockMethodTable->numLockModes)
elog(ERROR, "unrecognized lock mode: %d", lockmode);
+ /*
+ * Announcement requires a self-conflicting non-fast-path lockmode (the
+ * fast path can't carry one) and a strictly stronger target.
+ */
+ Assert(upgradeIntent == NoLock ||
+ (!EligibleForRelationFastPath(locktag, lockmode) &&
+ LOCKMODE_SELF_CONFLICTS(lockMethodTable->conflictTab, lockmode) &&
+ upgradeIntent > lockmode &&
+ upgradeIntent <= lockMethodTable->numLockModes));
+
if (RecoveryInProgress() && !InRecovery &&
(locktag->locktag_type == LOCKTAG_OBJECT ||
locktag->locktag_type == LOCKTAG_RELATION) &&
@@ -933,9 +955,13 @@ LockAcquireExtended(const LOCKTAG *locktag,
*
* If lockCleared is already set, caller need not worry about absorbing
* sinval messages related to the lock's object.
+ *
+ * Upgrade intent must be installed at the first grant, not on re-entry.
*/
if (locallock->nLocks > 0)
{
+ Assert(upgradeIntent == NoLock);
+
GrantLockLocal(locallock, owner);
if (locallock->lockCleared)
return LOCKACQUIRE_ALREADY_CLEAR;
@@ -1094,6 +1120,19 @@ LockAcquireExtended(const LOCKTAG *locktag,
lock = proclock->tag.myLock;
locallock->lock = lock;
+ if (upgradeIntent != NoLock)
+ {
+ /* It is not allowed to change upgradeIntent mode */
+ Assert(proclock->upgradeIntent == NoLock ||
+ proclock->upgradeIntent == upgradeIntent);
+ upgradeIntentSet = (proclock->upgradeIntent == NoLock);
+ if (upgradeIntentSet)
+ {
+ proclock->upgradeIntent = upgradeIntent;
+ lock->nUpgradeIntent++;
+ }
+ }
+
/*
* If lock requested conflicts with locks requested by waiters, must join
* wait queue. Otherwise, check for conflict with already-held locks.
@@ -1121,18 +1160,26 @@ LockAcquireExtended(const LOCKTAG *locktag,
waitResult = JoinWaitQueue(locallock, lockMethodTable, dontWait);
}
- if (waitResult == PROC_WAIT_STATUS_ERROR)
+ if (waitResult == PROC_WAIT_STATUS_ERROR ||
+ waitResult == PROC_WAIT_STATUS_PREEMPTED)
{
/*
- * We're not getting the lock because a deadlock was detected already
- * while trying to join the wait queue, or because we would have to
- * wait but the caller requested no blocking.
- *
+ * Not getting the lock: deadlock detected while joining the queue,
+ * preempted by another backend's protected upgrade, or dontWait.
* Undo the changes to shared entries before releasing the partition
* lock.
*/
AbortStrongLockAcquire();
+ /* If we just installed upgrade intent but never got the lock, undo it. */
+ if (upgradeIntentSet &&
+ (proclock->holdMask & LOCKBIT_ON(lockmode)) == 0)
+ {
+ proclock->upgradeIntent = NoLock;
+ Assert(lock->nUpgradeIntent > 0);
+ lock->nUpgradeIntent--;
+ }
+
if (proclock->holdMask == 0)
{
uint32 proclock_hashcode;
@@ -1208,6 +1255,8 @@ LockAcquireExtended(const LOCKTAG *locktag,
*locallockp = NULL;
return LOCKACQUIRE_NOT_AVAIL;
}
+ else if (waitResult == PROC_WAIT_STATUS_PREEMPTED)
+ ReportPreemptDeadlock(locallock);
else
{
DeadLockReport();
@@ -1236,14 +1285,15 @@ LockAcquireExtended(const LOCKTAG *locktag,
if (waitResult == PROC_WAIT_STATUS_ERROR)
{
- /*
- * We failed as a result of a deadlock, see CheckDeadLock(). Quit
- * now.
- */
Assert(!dontWait);
DeadLockReport();
/* DeadLockReport() will not return */
}
+ else if (waitResult == PROC_WAIT_STATUS_PREEMPTED)
+ {
+ Assert(!dontWait);
+ ReportPreemptDeadlock(locallock);
+ }
}
else
LWLockRelease(partitionLock);
@@ -1319,6 +1369,7 @@ SetupLockInTable(LockMethod lockMethodTable, PGPROC *proc,
dclist_init(&lock->waitProcs);
lock->nRequested = 0;
lock->nGranted = 0;
+ lock->nUpgradeIntent = 0;
MemSet(lock->requested, 0, sizeof(int) * MAX_LOCKMODES);
MemSet(lock->granted, 0, sizeof(int) * MAX_LOCKMODES);
LOCK_PRINT("LockAcquire: new", lock, lockmode);
@@ -1390,6 +1441,7 @@ SetupLockInTable(LockMethod lockMethodTable, PGPROC *proc,
proc->lockGroupLeader : proc;
proclock->holdMask = 0;
proclock->releaseMask = 0;
+ proclock->upgradeIntent = NoLock;
/* Add proclock to appropriate lists */
dlist_push_tail(&lock->procLocks, &proclock->lockLink);
dlist_push_tail(&proc->myProcLocks[partition], &proclock->procLink);
@@ -1413,9 +1465,13 @@ SetupLockInTable(LockMethod lockMethodTable, PGPROC *proc,
* about user-level coding practices that are in fact safe in context.
* It can be enabled to help find system-level problems.
*
+ * Skip for an announced upgrade: the detector handles those cycles
+ * by canceling a blocker.
+ *
* XXX Doing numeric comparison on the lockmodes is a hack; it'd be
* better to use a table. For now, though, this works.
*/
+ if (proclock->upgradeIntent != lockmode)
{
int i;
@@ -1671,6 +1727,19 @@ GrantLock(LOCK *lock, PROCLOCK *proclock, LOCKMODE lockmode)
if (lock->granted[lockmode] == lock->requested[lockmode])
lock->waitMask &= LOCKBIT_OFF(lockmode);
proclock->holdMask |= LOCKBIT_ON(lockmode);
+
+ /* The announced target was just granted; clear the announcement. */
+ if (proclock->upgradeIntent != NoLock)
+ {
+ Assert(proclock->upgradeIntent >= lockmode);
+ if (proclock->upgradeIntent == lockmode)
+ {
+ proclock->upgradeIntent = NoLock;
+ Assert(lock->nUpgradeIntent > 0);
+ lock->nUpgradeIntent--;
+ }
+ }
+
LOCK_PRINT("GrantLock", lock, lockmode);
Assert((lock->nGranted > 0) && (lock->granted[lockmode] > 0));
Assert(lock->nGranted <= lock->nRequested);
@@ -1727,6 +1796,16 @@ UnGrantLock(LOCK *lock, LOCKMODE lockmode,
* Now fix the per-proclock state.
*/
proclock->holdMask &= LOCKBIT_OFF(lockmode);
+
+ /* Releasing the last held mode also retires any pending announcement. */
+ if (proclock->upgradeIntent != NoLock &&
+ proclock->holdMask == 0)
+ {
+ proclock->upgradeIntent = NoLock;
+ Assert(lock->nUpgradeIntent > 0);
+ lock->nUpgradeIntent--;
+ }
+
PROCLOCK_PRINT("UnGrantLock: updated", proclock);
return wakeupNeeded;
@@ -1883,6 +1962,29 @@ AbortStrongLockAcquire(void)
SpinLockRelease(&FastPathStrongRelationLocks->mutex);
}
+/*
+ * ereport a deadlock for a protected-upgrade preemption. Use log_lock_waits
+ * for the cycle details from the protected backend's side.
+ */
+pg_noreturn static void
+ReportPreemptDeadlock(LOCALLOCK *locallock)
+{
+ StringInfoData buf;
+ const char *modename;
+
+ initStringInfo(&buf);
+ DescribeLockTag(&buf, &locallock->tag.lock);
+ modename = GetLockmodeName(locallock->tag.lock.locktag_lockmethodid,
+ locallock->tag.mode);
+
+ pgstat_report_deadlock();
+ ereport(ERROR,
+ (errcode(ERRCODE_T_R_DEADLOCK_DETECTED),
+ errmsg("deadlock detected"),
+ errdetail("Process %d could not acquire %s on %s because another backend holds a conflicting lock and has announced a future upgrade that would form a deadlock cycle.",
+ MyProcPid, modename, buf.data)));
+}
+
/*
* GrantAwaitedLock -- call GrantLockLocal for the lock we are doing
* WaitOnLock on.
@@ -2042,8 +2144,10 @@ waitonlock_error_callback(void *arg)
/*
* Remove a proc from the wait-queue it is on (caller must know it is on one).
- * This is only used when the proc has failed to get the lock, so we set its
- * waitStatus to PROC_WAIT_STATUS_ERROR.
+ * Used when the proc has failed to get the lock; the caller passes the
+ * waitStatus to publish (PROC_WAIT_STATUS_ERROR for ordinary failure or
+ * PROC_WAIT_STATUS_PREEMPTED when the wait was canceled to break a cycle
+ * with a protected upgrade).
*
* Appropriate partition lock must be held by caller. Also, caller is
* responsible for signaling the proc if needed.
@@ -2051,7 +2155,7 @@ waitonlock_error_callback(void *arg)
* NB: this does not clean up any locallock object that may exist for the lock.
*/
void
-RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode)
+RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode, ProcWaitStatus newStatus)
{
LOCK *waitLock = proc->waitLock;
PROCLOCK *proclock = proc->waitProcLock;
@@ -2060,6 +2164,8 @@ RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode)
/* Make sure proc is waiting */
Assert(proc->waitStatus == PROC_WAIT_STATUS_WAITING);
+ Assert(newStatus == PROC_WAIT_STATUS_ERROR ||
+ newStatus == PROC_WAIT_STATUS_PREEMPTED);
Assert(!dlist_node_is_detached(&proc->waitLink));
Assert(waitLock);
Assert(!dclist_is_empty(&waitLock->waitProcs));
@@ -2078,10 +2184,18 @@ RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode)
if (waitLock->granted[lockmode] == waitLock->requested[lockmode])
waitLock->waitMask &= LOCKBIT_OFF(lockmode);
+ /* Wait canceled => the announcement (if any) no longer applies. */
+ if (proclock->upgradeIntent != NoLock)
+ {
+ proclock->upgradeIntent = NoLock;
+ Assert(waitLock->nUpgradeIntent > 0);
+ waitLock->nUpgradeIntent--;
+ }
+
/* Clean up the proc's own state, and pass it the ok/fail signal */
proc->waitLock = NULL;
proc->waitProcLock = NULL;
- proc->waitStatus = PROC_WAIT_STATUS_ERROR;
+ proc->waitStatus = newStatus;
/*
* Delete the proclock immediately if it represents no already-held locks.
@@ -3697,6 +3811,8 @@ PostPrepare_Locks(FullTransactionId fxid)
Assert(lock->nGranted >= 0);
Assert(lock->nGranted <= lock->nRequested);
Assert((proclock->holdMask & ~lock->grantMask) == 0);
+ /* upgradeIntent is not supported for PREPARE TRANSACTION. */
+ Assert(proclock->upgradeIntent == NoLock);
/* Ignore it if nothing to release (must be a session lock) */
if (proclock->releaseMask == 0)
@@ -4397,6 +4513,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info,
dclist_init(&lock->waitProcs);
lock->nRequested = 0;
lock->nGranted = 0;
+ lock->nUpgradeIntent = 0;
MemSet(lock->requested, 0, sizeof(int) * MAX_LOCKMODES);
MemSet(lock->granted, 0, sizeof(int) * MAX_LOCKMODES);
LOCK_PRINT("lock_twophase_recover: new", lock, lockmode);
@@ -4460,6 +4577,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info,
proclock->groupLeader = proc;
proclock->holdMask = 0;
proclock->releaseMask = 0;
+ proclock->upgradeIntent = NoLock;
/* Add proclock to appropriate lists */
dlist_push_tail(&lock->procLocks, &proclock->lockLink);
dlist_push_tail(&proc->myProcLocks[partition],
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 1ac25068d62..2ce7073d9d2 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -854,7 +854,8 @@ LockErrorCleanup(void)
if (!dlist_node_is_detached(&MyProc->waitLink))
{
/* We could not have been granted the lock yet */
- RemoveFromWaitQueue(MyProc, lockAwaited->hashcode);
+ RemoveFromWaitQueue(MyProc, lockAwaited->hashcode,
+ PROC_WAIT_STATUS_ERROR);
}
else
{
@@ -1135,10 +1136,11 @@ AuxiliaryPidGetProc(int pid)
*
* Result is one of the following:
*
- * PROC_WAIT_STATUS_OK - lock was immediately granted
- * PROC_WAIT_STATUS_WAITING - joined the wait queue; call ProcSleep()
- * PROC_WAIT_STATUS_ERROR - immediate deadlock was detected, or would
- * need to wait and dontWait == true
+ * PROC_WAIT_STATUS_OK - lock immediately granted
+ * PROC_WAIT_STATUS_WAITING - joined the wait queue; call ProcSleep()
+ * PROC_WAIT_STATUS_ERROR - immediate deadlock or dontWait failure
+ * PROC_WAIT_STATUS_PREEMPTED - cycle proven against an announced upgrade;
+ * caller must raise a deadlock error
*
* NOTES: The process queue is now a priority queue for locking.
*/
@@ -1239,9 +1241,15 @@ JoinWaitQueue(LOCALLOCK *locallock, LockMethod lockMethodTable, bool dontWait)
* can't do that until we are *on* the wait queue. So, set
* a flag to check below, and break out of loop. Also,
* record deadlock info for later message.
+ *
+ * If this wait is our announced upgrade, skip the early
+ * exit so the full detector can preempt a blocker.
*/
- RememberSimpleDeadLock(MyProc, lockmode, lock, proc);
- early_deadlock = true;
+ if (proclock->upgradeIntent != lockmode)
+ {
+ RememberSimpleDeadLock(MyProc, lockmode, lock, proc);
+ early_deadlock = true;
+ }
break;
}
/* I must go before this waiter. Check special case. */
@@ -1270,6 +1278,75 @@ JoinWaitQueue(LOCALLOCK *locallock, LockMethod lockMethodTable, bool dontWait)
if (early_deadlock)
return PROC_WAIT_STATUS_ERROR;
+ /*
+ * Fast-fail: if some holder's announced upgrade conflicts with a mode we
+ * already hold, and its current holdMask conflicts with our request, a
+ * future cycle is inevitable. Don't wait deadlock_timeout for the same
+ * conclusion. Skipped when we are the announced upgrader, when we hold
+ * nothing here, or when no holder has any announcement.
+ *
+ * This is only a fast-fail check for cycles already visible in the main
+ * lock table. Fast-path locks can hide edges. For example, while waiting
+ * for ShareUpdateExclusiveLock, this backend may still hold weaker fast-path
+ * locks on the same relation, since ShareUpdateExclusiveLock does not itself
+ * force fast-path transfer. Such an edge becomes visible only later, e.g.
+ * when another backend's AccessExclusiveLock upgrade transfers fast-path
+ * locks. The full deadlock detector handles that case.
+ *
+ * We could cheaply include modes this backend holds locally, via
+ * LockMethodLocalHash. That catches the direct case where this backend holds
+ * a weak fast-path lock and waits behind an announced stronger upgrade, but
+ * it cannot see hidden fast-path edges owned by third backends in longer
+ * cycles.
+ *
+ * Making upgrade intent visible to the fast-path mechanism would catch more
+ * cases, e.g. by treating upgradeIntent as conflicting for
+ * ConflictsWithRelationFastPath() and keeping FastPathStrongRelationLocks
+ * bumped until the upgrade is consumed. However, that would pessimize the
+ * whole concurrent operation: ordinary weak relation locks would stop using
+ * the fast path and contend in the main lock table.
+ *
+ * Transferring fast-path locks from DeadLockCheck() only when needed would
+ * avoid that penalty, but is lock-ordering-sensitive: DeadLockCheck() already
+ * holds all lock partition LWLocks, while fast-path transfer needs
+ * fpInfoLocks and may acquire partition LWLocks in the normal order. A safe
+ * design would need probe/retry logic, making it worthwhile only if hidden
+ * longer fast-path cycles must be preempted early.
+ */
+ if (proclock->upgradeIntent != lockmode && myHeldLocks != 0 &&
+ lock->nUpgradeIntent > 0)
+ {
+ dlist_iter iter;
+
+ dlist_foreach(iter, &lock->procLocks)
+ {
+ PROCLOCK *holder = dlist_container(PROCLOCK, lockLink, iter.cur);
+
+ if (holder->upgradeIntent == NoLock)
+ continue;
+
+ /* Skip self (we're in lock->procLocks too). */
+ if (holder == proclock)
+ continue;
+
+ /* Same lock group: not a cycle. */
+ if (leader != NULL && holder->groupLeader == leader)
+ continue;
+
+ /* Must the holder later wait for me? */
+ if ((lockMethodTable->conflictTab[holder->upgradeIntent] &
+ myHeldLocks) == 0)
+ continue;
+
+ /* Must I wait for the holder now? */
+ if ((lockMethodTable->conflictTab[lockmode] &
+ holder->holdMask) == 0)
+ continue;
+
+ return PROC_WAIT_STATUS_PREEMPTED;
+ }
+ }
+
/*
* At this point we know that we'd really need to sleep. If we've been
* commanded not to do that, bail out.
@@ -1308,8 +1385,10 @@ JoinWaitQueue(LOCALLOCK *locallock, LockMethod lockMethodTable, bool dontWait)
*
* Result is one of the following:
*
- * PROC_WAIT_STATUS_OK - lock was granted
- * PROC_WAIT_STATUS_ERROR - a deadlock was detected
+ * PROC_WAIT_STATUS_OK - lock granted
+ * PROC_WAIT_STATUS_ERROR - deadlock detected
+ * PROC_WAIT_STATUS_PREEMPTED - canceled by another backend's DeadLockCheck
+ * to break a protected-upgrade cycle
*/
ProcWaitStatus
ProcSleep(LOCALLOCK *locallock)
@@ -1562,16 +1641,19 @@ ProcSleep(LOCALLOCK *locallock)
/*
* If awoken after the deadlock check interrupt has run, increment the
- * lock statistics counters and if log_lock_waits is on, then report
- * about the wait.
+ * lock statistics counters and, if log_lock_waits is on, report about
+ * the wait. Also report a wait that another backend preempted before
+ * our own deadlock timeout fired.
*/
- if (deadlock_state != DS_NOT_YET_CHECKED)
+ if (deadlock_state != DS_NOT_YET_CHECKED ||
+ (myWaitStatus == PROC_WAIT_STATUS_PREEMPTED && log_lock_waits))
{
long secs;
int usecs;
long msecs;
- INJECTION_POINT("deadlock-timeout-fired", NULL);
+ if (deadlock_state != DS_NOT_YET_CHECKED)
+ INJECTION_POINT("deadlock-timeout-fired", NULL);
TimestampDifference(get_timeout_start_time(DEADLOCK_TIMEOUT),
GetCurrentTimestamp(),
&secs, &usecs);
@@ -1611,6 +1693,13 @@ ProcSleep(LOCALLOCK *locallock)
(errdetail_log_plural("Process holding the lock: %s. Wait queue: %s.",
"Processes holding the lock: %s. Wait queue: %s.",
lockHoldersNum, lock_holders_sbuf.data, lock_waiters_sbuf.data))));
+ else if (deadlock_state == DS_PREEMPT_DEADLOCK)
+ ereport(LOG,
+ (errmsg("process %d avoided deadlock for %s on %s by canceling a blocking lock wait after %ld.%03d ms",
+ MyProcPid, modename, buf.data, msecs, usecs),
+ (errdetail_log_plural("Process holding the lock: %s. Wait queue: %s.",
+ "Processes holding the lock: %s. Wait queue: %s.",
+ lockHoldersNum, lock_holders_sbuf.data, lock_waiters_sbuf.data))));
else if (deadlock_state == DS_HARD_DEADLOCK)
{
/*
@@ -1659,16 +1748,13 @@ ProcSleep(LOCALLOCK *locallock)
MyProcPid, modename, buf.data, msecs, usecs)));
else
{
- Assert(myWaitStatus == PROC_WAIT_STATUS_ERROR);
+ Assert(myWaitStatus == PROC_WAIT_STATUS_ERROR ||
+ myWaitStatus == PROC_WAIT_STATUS_PREEMPTED);
/*
- * Currently, the deadlock checker always kicks its own
- * process, which means that we'll only see
- * PROC_WAIT_STATUS_ERROR when deadlock_state ==
- * DS_HARD_DEADLOCK, and there's no need to print
- * redundant messages. But for completeness and
- * future-proofing, print a message if it looks like
- * someone else kicked us off the lock.
+ * Self-detected hard deadlocks log via DeadLockReport;
+ * skip the redundant LOG. Otherwise (e.g. preempted),
+ * emit it here.
*/
if (deadlock_state != DS_HARD_DEADLOCK)
ereport(LOG,
@@ -1870,14 +1956,16 @@ CheckDeadLock(void)
* Get this process out of wait state. (Note: we could do this more
* efficiently by relying on lockAwaited, but use this coding to
* preserve the flexibility to kill some other transaction than the
- * one detecting the deadlock.)
+ * one detecting the deadlock. DS_PREEMPT_DEADLOCK exercises that
+ * already.)
*
* RemoveFromWaitQueue sets MyProc->waitStatus to
* PROC_WAIT_STATUS_ERROR, so ProcSleep will report an error after we
* return.
*/
Assert(MyProc->waitLock != NULL);
- RemoveFromWaitQueue(MyProc, LockTagHashCode(&(MyProc->waitLock->tag)));
+ RemoveFromWaitQueue(MyProc, LockTagHashCode(&(MyProc->waitLock->tag)),
+ PROC_WAIT_STATUS_ERROR);
/*
* We're done here. Transaction abort caused by the error that
diff --git a/src/include/catalog/namespace.h b/src/include/catalog/namespace.h
index 9453a3e4932..f0a71184757 100644
--- a/src/include/catalog/namespace.h
+++ b/src/include/catalog/namespace.h
@@ -99,6 +99,13 @@ typedef void (*RangeVarGetRelidCallback) (const RangeVar *relation, Oid relId,
RangeVarGetRelidExtended(relation, lockmode, \
(missing_ok) ? RVR_MISSING_OK : 0, NULL, NULL)
+extern Oid RangeVarGetRelidWithUpgradeIntent(const RangeVar *relation,
+ LOCKMODE lockmode,
+ LOCKMODE upgradeMode,
+ uint32 flags,
+ RangeVarGetRelidCallback callback,
+ void *callback_arg);
+
extern Oid RangeVarGetRelidExtended(const RangeVar *relation,
LOCKMODE lockmode, uint32 flags,
RangeVarGetRelidCallback callback,
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 2a985ce5e15..57667432e02 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -42,6 +42,9 @@ extern void LockRelationId(LockRelId *relid, LOCKMODE lockmode);
extern bool ConditionalLockRelationOid(Oid relid, LOCKMODE lockmode);
extern void UnlockRelationId(LockRelId *relid, LOCKMODE lockmode);
extern void UnlockRelationOid(Oid relid, LOCKMODE lockmode);
+extern void LockRelationOidWithUpgradeIntent(Oid relid,
+ LOCKMODE lockmode,
+ LOCKMODE upgradeMode);
extern void LockRelation(Relation relation, LOCKMODE lockmode);
extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ee3cb1dc203..cff76a7aed8 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -131,6 +131,7 @@ typedef const LockMethodData *LockMethod;
* nRequested -- total requested locks of all types.
* granted -- count of each lock type currently granted on the lock.
* nGranted -- total granted locks of all types.
+ * nUpgradeIntent -- # of proclocks here with upgradeIntent set.
*
* Note: these counts count 1 for each backend. Internally to a backend,
* there may be multiple grabs on a particular lock, but this is not reflected
@@ -150,6 +151,7 @@ typedef struct LOCK
int nRequested; /* total of requested[] array */
int granted[MAX_LOCKMODES]; /* counts of granted locks */
int nGranted; /* total of granted[] array */
+ int nUpgradeIntent; /* see LOCK comment above */
} LOCK;
#define LOCK_LOCKMETHOD(lock) ((LOCKMETHODID) (lock).tag.locktag_lockmethodid)
@@ -206,6 +208,7 @@ typedef struct PROCLOCK
PGPROC *groupLeader; /* proc's lock group leader, or proc itself */
LOCKMASK holdMask; /* bitmask for lock types currently held */
LOCKMASK releaseMask; /* bitmask for lock types to be released */
+ LOCKMODE upgradeIntent; /* announced future upgrade target, or NoLock */
dlist_node lockLink; /* list link in LOCK's list of proclocks */
dlist_node procLink; /* list link in PGPROC's list of proclocks */
} PROCLOCK;
@@ -342,10 +345,22 @@ typedef enum
DS_NO_DEADLOCK, /* no deadlock detected */
DS_SOFT_DEADLOCK, /* deadlock avoided by queue rearrangement */
DS_HARD_DEADLOCK, /* deadlock, no way out but ERROR */
+ DS_PREEMPT_DEADLOCK, /* resolved by canceling a blocking waiter */
DS_BLOCKED_BY_AUTOVACUUM, /* no deadlock; queue blocked by autovacuum
* worker */
} DeadLockState;
+/*
+ * Lock-wait status set by RemoveFromWaitQueue and read by the waiter on wakeup
+ */
+typedef enum
+{
+ PROC_WAIT_STATUS_OK,
+ PROC_WAIT_STATUS_WAITING,
+ PROC_WAIT_STATUS_ERROR,
+ PROC_WAIT_STATUS_PREEMPTED,
+} ProcWaitStatus;
+
/*
* The lockmgr's shared hash tables are partitioned to reduce contention.
* To determine which partition a given locktag belongs to, compute the tag's
@@ -386,6 +401,7 @@ extern LockAcquireResult LockAcquire(const LOCKTAG *locktag,
bool dontWait);
extern LockAcquireResult LockAcquireExtended(const LOCKTAG *locktag,
LOCKMODE lockmode,
+ LOCKMODE upgradeIntent,
bool sessionLock,
bool dontWait,
bool reportMemoryError,
@@ -418,7 +434,8 @@ extern void GrantAwaitedLock(void);
extern LOCALLOCK *GetAwaitedLock(void);
extern void ResetAwaitedLock(void);
-extern void RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode);
+extern void RemoveFromWaitQueue(PGPROC *proc, uint32 hashcode,
+ ProcWaitStatus newStatus);
extern LockData *GetLockStatusData(void);
extern BlockedProcsData *GetBlockerStatusData(int blocked_pid);
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 3e1d1fad5f9..9f89e587f3a 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -146,13 +146,6 @@ extern PGDLLIMPORT int FastPathLockGroupsPerBackend;
#define DELAY_CHKPT_COMPLETE (1<<1)
#define DELAY_CHKPT_IN_COMMIT (DELAY_CHKPT_START | 1<<2)
-typedef enum
-{
- PROC_WAIT_STATUS_OK,
- PROC_WAIT_STATUS_WAITING,
- PROC_WAIT_STATUS_ERROR,
-} ProcWaitStatus;
-
/*
* Each backend has a PGPROC struct in shared memory. There is also a list of
* currently-unused PGPROC structs that will be reallocated to new backends.
diff --git a/src/test/modules/injection_points/expected/repack.out b/src/test/modules/injection_points/expected/repack.out
index b575e9052ee..9dffc603d9f 100644
--- a/src/test/modules/injection_points/expected/repack.out
+++ b/src/test/modules/injection_points/expected/repack.out
@@ -1,4 +1,4 @@
-Parsed test spec with 2 sessions
+Parsed test spec with 4 sessions
starting permutation: wait_before_lock change_existing change_new change_subxact1 change_subxact2 check2 wakeup_before_lock check1
injection_points_attach
@@ -111,3 +111,182 @@ injection_points_detach
(1 row)
+
+starting permutation: wait_before_lock begin_txn lock_table_ae s3_wakeup end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_txn:
+ BEGIN;
+
+step lock_table_ae:
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+ <waiting ...>
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step lock_table_ae: <... completed>
+step end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s2_timeout_fast wait_before_lock begin_and_read lock_table_ae s3_wakeup end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s2_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step lock_table_ae:
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+ <waiting ...>
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step lock_table_ae: <... completed>
+ERROR: deadlock detected
+step end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s2_timeout_fast wait_before_lock begin_and_read lock_table_sue s3_wakeup end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s2_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step lock_table_sue:
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+ <waiting ...>
+step s3_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step lock_table_sue: <... completed>
+ERROR: deadlock detected
+step wait_before_lock: <... completed>
+step end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+
+starting permutation: s2_timeout_fast wait_before_lock begin_and_read s3_begin_txn s3_lock_table_y_sue s3_lock_table_sue lock_table_y_sue s4_wakeup end_txn s3_end_txn
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s2_timeout_fast:
+ SET deadlock_timeout = '10ms';
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step begin_and_read:
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+
+?column?
+--------
+ 1
+(1 row)
+
+step s3_begin_txn:
+ BEGIN;
+
+step s3_lock_table_y_sue:
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+
+step s3_lock_table_sue:
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+ <waiting ...>
+step lock_table_y_sue:
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+ <waiting ...>
+step s4_wakeup:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step lock_table_y_sue: <... completed>
+step s3_lock_table_sue: <... completed>
+ERROR: deadlock detected
+step end_txn:
+ COMMIT;
+
+step wait_before_lock: <... completed>
+step s3_end_txn:
+ COMMIT;
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index d727a9b056b..36d32964f94 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -5,6 +5,7 @@ setup
CREATE TABLE repack_test(i int PRIMARY KEY, j int);
INSERT INTO repack_test(i, j) VALUES (1, 1), (2, 2), (3, 3), (4, 4);
+ CREATE TABLE repack_test_y(i int);
CREATE TABLE relfilenodes(node oid);
@@ -15,6 +16,7 @@ setup
teardown
{
DROP TABLE repack_test;
+ DROP TABLE repack_test_y;
DROP EXTENSION injection_points;
DROP TABLE relfilenodes;
@@ -61,6 +63,10 @@ teardown
}
session s2
+step s2_timeout_fast
+{
+ SET deadlock_timeout = '10ms';
+}
# Change the existing data. UPDATE changes both key and non-key columns. Also
# update one row twice to test whether tuple version generated by this session
# can be found.
@@ -128,6 +134,60 @@ step wakeup_before_lock
{
SELECT injection_points_wakeup('repack-concurrently-before-lock');
}
+# Steps used in lock contention tests.
+step begin_txn
+{
+ BEGIN;
+}
+step begin_and_read
+{
+ BEGIN;
+ SELECT 1 FROM repack_test LIMIT 1;
+}
+step lock_table_ae
+{
+ LOCK TABLE repack_test IN ACCESS EXCLUSIVE MODE;
+}
+step lock_table_sue
+{
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step lock_table_y_sue
+{
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step end_txn
+{
+ COMMIT;
+}
+
+session s3
+step s3_begin_txn
+{
+ BEGIN;
+}
+step s3_lock_table_y_sue
+{
+ LOCK TABLE repack_test_y IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step s3_lock_table_sue
+{
+ LOCK TABLE repack_test IN SHARE UPDATE EXCLUSIVE MODE;
+}
+step s3_wakeup
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
+step s3_end_txn
+{
+ COMMIT;
+}
+
+session s4
+step s4_wakeup
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
# Test if data changes introduced while one session is performing REPACK
# CONCURRENTLY find their way into the table.
@@ -140,3 +200,51 @@ permutation
check2
wakeup_before_lock
check1
+
+# A waiter that does not already hold a conflicting lock on the table is
+# waits until REPACK finishes and then acquires its lock (soft-deadlock).
+permutation
+ wait_before_lock
+ begin_txn
+ lock_table_ae
+ s3_wakeup
+ end_txn
+
+# A waiter that already holds AccessShareLock then waits for AccessExclusiveLock
+# behind REPACK's ShareUpdateExclusiveLock.
+permutation
+ s2_timeout_fast
+ wait_before_lock
+ begin_and_read
+ lock_table_ae(*)
+ s3_wakeup
+ end_txn
+
+# Same shape as above, but the waiter requests ShareUpdateExclusiveLock.
+permutation
+ s2_timeout_fast
+ wait_before_lock
+ begin_and_read
+ lock_table_sue(*)
+ s3_wakeup
+ end_txn
+
+# Three-backend future deadlock:
+#
+# - s1 holds ShareUpdateExclusiveLock on repack_test and has declared a future
+# AccessExclusiveLock on it.
+# - s2 holds AccessShareLock on repack_test.
+# - s3 holds ShareUpdateExclusiveLock on repack_test_y, then waits for
+# ShareUpdateExclusiveLock on repack_test behind s1.
+# - s2 waits for ShareUpdateExclusiveLock on repack_test_y behind s3.
+permutation
+ s2_timeout_fast
+ wait_before_lock
+ begin_and_read
+ s3_begin_txn
+ s3_lock_table_y_sue
+ s3_lock_table_sue
+ lock_table_y_sue(*)
+ s4_wakeup
+ end_txn
+ s3_end_txn
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-18 19:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-19 11:00 ` Alexander Lakhin <exclusion@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Alexander Lakhin @ 2026-04-19 11:00 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Andres Freund <andres@anarazel.de>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello Antonin and Alvaro,
I've noticed a wrong reference coined in commit 28d534e2a:
A comment in repack_worker.c says:
/*
* Override the default bgworker_die() with die() so we can use
* CHECK_FOR_INTERRUPTS().
*/
pqsignal(SIGTERM, die);
BackgroundWorkerUnblockSignals();
but bgworker_die() doesn't exist since d62dca3b2 (2026-02-18).
Maybe this is not needed anymore?
Best regards,
Alexander
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-17 14:44 ` Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Justin Pryzby @ 2026-04-17 14:44 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
I had trouble testing this at first:
postgres=# REPACK (CONCURRENTLY) users;
ERROR: 42501: permission denied to use replication slots
DETAIL: Only roles with the REPLICATION attribute may use replication slots.
CONTEXT: REPACK decoding worker
LOCATION: CheckSlotPermissions, slot.c:1697
That's surprising, since it was run as a superuser.
It turns out that repack runs as the owner of the table, and the table
*owner* needs to have REPLICATION -- regardless of who runs the command.
I imagine people have been testing with one user, that both owns the
table and invokes REPACK. Maybe this just needs to be clarified in the
documentation/error message?
--
Justin
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
@ 2026-04-20 07:44 ` Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-20 07:44 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
Justin Pryzby <pryzby@telsasoft.com> wrote:
> I had trouble testing this at first:
>
> postgres=# REPACK (CONCURRENTLY) users;
> ERROR: 42501: permission denied to use replication slots
> DETAIL: Only roles with the REPLICATION attribute may use replication slots.
> CONTEXT: REPACK decoding worker
> LOCATION: CheckSlotPermissions, slot.c:1697
>
> That's surprising, since it was run as a superuser.
>
> It turns out that repack runs as the owner of the table, and the table
> *owner* needs to have REPLICATION -- regardless of who runs the command.
>
> I imagine people have been testing with one user, that both owns the
> table and invokes REPACK. Maybe this just needs to be clarified in the
> documentation/error message?
It was discussed earlier [1] and the concerns about possibly excessive
resource consumptions were addressed by [2]. So I think it the fix was just
forgotten. Attached here.
[1] https://www.postgresql.org/message-id/202603161220.7nv6whwu33hi@alvherre.pgsql
[2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=e76d8c749c3152657711ed733f0aea61c0e36...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-20 07:47 ` Antonin Houska <ah@cybertec.at>
2026-04-20 07:54 ` Re: Adding REPACK [concurrently] Chao Li <li.evan.chao@gmail.com>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-20 07:47 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> It was discussed earlier [1] and the concerns about possibly excessive
> resource consumptions were addressed by [2]. So I think it the fix was just
> forgotten. Attached here.
Sorry, I attached wrong patch. This is what I meant.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-20 07:54 ` Chao Li <li.evan.chao@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Chao Li @ 2026-04-20 07:54 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
> On Apr 20, 2026, at 15:47, Antonin Houska <ah@cybertec.at> wrote:
>
> Antonin Houska <ah@cybertec.at> wrote:
>
>> It was discussed earlier [1] and the concerns about possibly excessive
>> resource consumptions were addressed by [2]. So I think it the fix was just
>> forgotten. Attached here.
>
> Sorry, I attached wrong patch. This is what I meant.
>
> --
> Antonin Houska
> Web: https://www.cybertec-postgresql.com
>
> <0001-Do-not-check-the-REPLICATION-attribute-when-running-.patch>
In my test, I found the exact same issue and I was about to post a fix that is the same as v1. So, v1 looks good to me.
BTW, Antonin, can you please take a look at [1] that is the other issue I found with repack.
[1] https://www.postgresql.org/message-id/10DD5E13-B45D-44F1-BE08-C63E00ABCAC0%40gmail.com
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-20 11:45 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-20 11:45 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
On 2026-Apr-20, Antonin Houska wrote:
> Antonin Houska <ah@cybertec.at> wrote:
>
> > It was discussed earlier [1] and the concerns about possibly excessive
> > resource consumptions were addressed by [2]. So I think it the fix was just
> > forgotten. Attached here.
>
> Sorry, I attached wrong patch. This is what I meant.
Yeah, I had also written the same patch a couple of days ago.
BTW I ran into a small problem after adding some tests in cluster.sql
that would exercise this -- that test would die more or less randomly
but frequently in CI (which it never did in my laptop) because of the
size of the snapshot,
ALTER TABLE ptnowner1 REPLICA IDENTITY USING INDEX ptnowner1_i_key;
REPACK (CONCURRENTLY) ptnowner1;
+ERROR: initial slot snapshot too large
+CONTEXT: REPACK decoding worker
RESET SESSION AUTHORIZATION;
I think the solution for this is to move cluster to a separate parallel
test. The one where it is now is a bit too crowded. Maybe the one for
compression is okay? I'll test and push if I see it passing CI.
Another obvious thing after adding tests is that the LOGIN privilege is
required, which is also quite bogus IMO. 0002 here should solve that.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"If you want to have good ideas, you must have many ideas. Most of them
will be wrong, and what you have to learn is which ones to throw away."
(Linus Pauling)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-20 13:46 ` Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-20 13:46 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Apr-20, Antonin Houska wrote:
>
> > Antonin Houska <ah@cybertec.at> wrote:
> >
> > > It was discussed earlier [1] and the concerns about possibly excessive
> > > resource consumptions were addressed by [2]. So I think it the fix was just
> > > forgotten. Attached here.
> >
> > Sorry, I attached wrong patch. This is what I meant.
>
> Yeah, I had also written the same patch a couple of days ago.
>
> BTW I ran into a small problem after adding some tests in cluster.sql
> that would exercise this -- that test would die more or less randomly
> but frequently in CI (which it never did in my laptop) because of the
> size of the snapshot,
>
> ALTER TABLE ptnowner1 REPLICA IDENTITY USING INDEX ptnowner1_i_key;
> REPACK (CONCURRENTLY) ptnowner1;
> +ERROR: initial slot snapshot too large
> +CONTEXT: REPACK decoding worker
> RESET SESSION AUTHORIZATION;
>
> I think the solution for this is to move cluster to a separate parallel
> test. The one where it is now is a bit too crowded. Maybe the one for
> compression is okay? I'll test and push if I see it passing CI.
That shouldn't break anything, but I have no idea why this problem was not
triggered (as far as I remember) by the stress tests we ran during
development.
I thought that it might be due to less frequent calls of
SnapBuildPurgeOlderTxn(), which might in turn be due to less frequent
checkpoints (because xl_running_xacts is typically written during checkpoint),
and that checkpoints may be deliberately less frequent to make regression
tests run faster. However I don't immediately see related configuration in the
regression test setup.
> Another obvious thing after adding tests is that the LOGIN privilege is
> required, which is also quite bogus IMO. 0002 here should solve that.
ok
> >From b3d4158356f4914d2b0cba86eef6994c0ee50ab9 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
> Date: Mon, 20 Apr 2026 11:38:48 +0200
> Subject: [PATCH 1/2] REPACK: do not require the user to have REPLICATION
> Because there are now successful concurrent repack runs in the
> regression tests, we're forced to run test_plan_advice under
> wal_level=replica.
Is this an attempt to disable REPACK (CONCURRENTLY)? That would require
wal_level=minimal (due to commit 67c20979ce). In which way does REPACK seem to
break test_plan_advice?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-20 13:54 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 14:42 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-20 13:54 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
On 2026-Apr-20, Antonin Houska wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > BTW I ran into a small problem after adding some tests in cluster.sql
> > that would exercise this -- that test would die more or less randomly
> > but frequently in CI (which it never did in my laptop) because of the
> > size of the snapshot,
> >
> > ALTER TABLE ptnowner1 REPLICA IDENTITY USING INDEX ptnowner1_i_key;
> > REPACK (CONCURRENTLY) ptnowner1;
> > +ERROR: initial slot snapshot too large
> > +CONTEXT: REPACK decoding worker
> > RESET SESSION AUTHORIZATION;
> >
> > I think the solution for this is to move cluster to a separate parallel
> > test. The one where it is now is a bit too crowded. Maybe the one for
> > compression is okay? I'll test and push if I see it passing CI.
>
> That shouldn't break anything, but I have no idea why this problem was not
> triggered (as far as I remember) by the stress tests we ran during
> development.
I took a guess that it's because some tests use minimally configured
servers -- that is, max_connections=20 or so -- and then run 20
processes. If we then try to construct a snapshot that's limited to
having only that many XIDs, we might not have enough room in the xip
array. I didn't try to trace it super carefully though.
> > >From b3d4158356f4914d2b0cba86eef6994c0ee50ab9 Mon Sep 17 00:00:00 2001
> > From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
> > Date: Mon, 20 Apr 2026 11:38:48 +0200
> > Subject: [PATCH 1/2] REPACK: do not require the user to have REPLICATION
>
> > Because there are now successful concurrent repack runs in the
> > regression tests, we're forced to run test_plan_advice under
> > wal_level=replica.
>
> Is this an attempt to disable REPACK (CONCURRENTLY)? That would require
> wal_level=minimal (due to commit 67c20979ce). In which way does REPACK seem to
> break test_plan_advice?
No, quite the contrary. That test normally runs with wal_level=minimal,
which causes REPACK to complain that it cannot start logical
decoding.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-20 14:42 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 15:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Tom Lane @ 2026-04-20 14:42 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2026-Apr-20, Antonin Houska wrote:
>> Is this an attempt to disable REPACK (CONCURRENTLY)? That would require
>> wal_level=minimal (due to commit 67c20979ce). In which way does REPACK seem to
>> break test_plan_advice?
> No, quite the contrary. That test normally runs with wal_level=minimal,
> which causes REPACK to complain that it cannot start logical
> decoding.
So what you're saying is that the core regression tests will now fail
with wal_level=minimal? I don't see how that can possibly be
considered acceptable from a global standpoint; we might as well
remove wal_level=minimal, because it will never again get tested.
I think you need to move these tests out into some other test suite
(or make a new one).
regards, tom lane
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 14:42 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-04-20 15:11 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 16:01 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 16:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-20 15:11 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Antonin Houska <ah@cybertec.at>; Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
On 2026-Apr-20, Tom Lane wrote:
> So what you're saying is that the core regression tests will now fail
> with wal_level=minimal? I don't see how that can possibly be
> considered acceptable from a global standpoint; we might as well
> remove wal_level=minimal, because it will never again get tested.
Hmm, you're right, this was brain fade on my part.
It surprising though that no buildfarm animal so far has complained.
But I remember there's at least one that runs with that, per
be142fa008ad.
> I think you need to move these tests out into some other test suite
> (or make a new one).
I'll see what I can find.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 14:42 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 15:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-20 16:01 ` Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 416+ messages in thread
From: Tom Lane @ 2026-04-20 16:01 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> It surprising though that no buildfarm animal so far has complained.
> But I remember there's at least one that runs with that, per
> be142fa008ad.
Yeah, thorntail does, but it only runs once a day.
regards, tom lane
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 14:42 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 15:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-20 16:30 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-22 09:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-20 16:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Antonin Houska <ah@cybertec.at>; Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
On 2026-Apr-20, Alvaro Herrera wrote:
> On 2026-Apr-20, Tom Lane wrote:
> > I think you need to move these tests out into some other test suite
> > (or make a new one).
>
> I'll see what I can find.
I think the simplest would be to add them to
src/test/modules/injection_points. We already have some repack tests
there (because they needed injection points), and it has an SQL suite,
so we would not be adding any extra overhead. However, the new tests
are not related to injection points, so they would be out of place.
Another possibility could be src/test/subscription, but it doesn't have
sql tests; doesn't seem good to have them just for this.
There's also contrib/test_decoding. It's somewhat vaguely adjacent, and
the tests aren't _really_ about the decoding part, but of all these
options, it seems the least bad one.
I don't like the idea of adding another suite. Too much scaffolding for
so little, I think.
I don't have any other ideas ATM.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"We're here to devour each other alive" (Hobbes)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 14:42 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 15:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 16:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-22 09:30 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-23 11:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-22 09:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Antonin Houska <ah@cybertec.at>; Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
On 2026-Apr-20, Alvaro Herrera wrote:
> There's also contrib/test_decoding. It's somewhat vaguely adjacent, and
> the tests aren't _really_ about the decoding part, but of all these
> options, it seems the least bad one.
Here's a patch that does this.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 14:42 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 15:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 16:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-22 09:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-23 11:01 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-23 11:01 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Antonin Houska <ah@cybertec.at>; Justin Pryzby <pryzby@telsasoft.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; pgsql-hackers@lists.postgresql.org; Robert Treat <rob@xzilla.net>
On 2026-Apr-22, Alvaro Herrera wrote:
> On 2026-Apr-20, Alvaro Herrera wrote:
>
> > There's also contrib/test_decoding. It's somewhat vaguely adjacent, and
> > the tests aren't _really_ about the decoding part, but of all these
> > options, it seems the least bad one.
>
> Here's a patch that does this.
Pushed, thanks. Thorntail should turn green in its next run.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Nunca se desea ardientemente lo que solo se desea por razón" (F. Alexandre)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-15 15:54 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2026-04-15 15:54 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Andres Freund <andres@anarazel.de>; Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Wed, Apr 15, 2026 at 4:50 PM Antonin Houska <ah@cybertec.at> wrote:
> The approach proposed by Mihail [1] seems the least problematic to me, and
> something like that occurred to me when I thought about the problem the first
> time. However, when we wake up the other processes in order to run the
> deadlock detection, they should do that immediately. I've got no good idea
> about implementation at the moment, since latch can be set for unrelated
> reasons. (Besides that, I have some more questions about this patch, which I
> can post separately.)
It is already possible to "deadlock" ANOTHER backend while running the
deadlock check [0].
Supported in current infra, just not used at the moment (my POC used that also).
[0]: https://github.com/postgres/postgres/blob/master/src/backend/storage/lmgr/proc.c#L1870-L1873
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-04-09 14:13 ` Andres Freund <andres@anarazel.de>
1 sibling, 0 replies; 416+ messages in thread
From: Andres Freund @ 2026-04-09 14:13 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
On 2026-04-09 01:36:00 +0200, Mihail Nikalayeu wrote:
> Hello, Andres!
>
> On Wed, Apr 8, 2026 at 7:22 PM Andres Freund <andres@anarazel.de> wrote:
> > I don't think this is a viable path. You need to prevent any further lock
> > acquisitions on the relation to be able to swap it, not just conflicting DDL.
>
> AFAIU, Amit's main idea is that we currently upgrade the lock instead
> of **releasing and re-acquiring** it because we fear DDL between those
> actions.
I got that, I just don't think it's going to work in any sort of way
reasonably well.
The fix here should be to make the system understand how to make lock upgrades
as safe as possible, not to hack around the corners. For that you need to
teach the deadlock detector about all of this properly.
> > I don't think CheckTableNotInUse() would work anyway - don't we already hold
> > locks by the point we call it?
>
> Yes, the DDL session already holds locks by the time
> CheckTableNotInUse is called - but is that really the problem? They
> will be released on error.
There's no guarantee they get there if it's done after the lock
acquisition. It can be part of a more complicated lock cycle.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-06 09:03 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-06 09:03 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> So I've been trying to understand the "Introduce an option to make
> logical replication database specific." patch and I have to confess I
> just cannot.
>
> As far as I can read, the point is that if we reach
> SnapBuildProcessRunningXacts() when db_specific is true (which means
> standby_decode is called in an output plugin that has set
> need_shared_catalogs to false), _and_ we've not reached consistent state
> yet, then we'll call LogStandbySnapshot with our DB oid to emit a new
> xl_running_xacts message.
Right.
> So the WAL-decoding process emits WAL. I don't know if in normal
> conditions logical decoding processes emit WAL. If this is exceptional,
> I think we should add a comment.
Emitting WAL in logical decoding is not exceptional: SnapBuildWaitSnapshot()
already calls LogStandbySnapshot(), in order to get to the next phase.
> Now, this additional WAL message will be processed by all other
> processes decoding WAL. Perhaps it will ignored by most of them.
Right. That's one thing that I realized late yesterday, after having posted
the latest version of the patch. In SnapBuildProcessRunningXacts(), we need
if (OidIsValid(running->dbid))
return;
rather than
if (db_specific)
return;
because other backends can also generate their database-specific records.
> But
> most importantly, it will also reach back to ourselves, at which point
> we can hopefully use it to see that we might have reached consistent
> state within our database. Then we know our snapshot is ready to be
> used.
>
> Is this correct?
Yes.
> I think the reason it's safe to skip a lot of the processing caused by
> this additional process, is that xl_running_xacts messages are also
> emitted in other places in a non-database specific manner. So all the
> other placecs that are emitting that message continue to exist and
> cause logical-decoders operate in the same way as before.
Yes. I'm still thinking though if, after having used the database-specific
record to reach CONSITENT, we sould enforce one cluster-wide record, so that
the cleanup (in "our backend") takes place sooner rather than later. Not sure
about that.
> I think we should sprinkle lots of comments in several places about
> this. For example, I propose that standby_redo() should have something
> like
>
> * If 'dbid' is valid, only gather transactions running in that database.
> + * Such records should not be the only ones emitted, because this has
> + * potentially dangerous side-effects which makes some places ignore them:
> + *
> + * 1. SnapBuildProcessRunningXacts will skip computing the xmin and restart
> + * point from its input record if the record's xmin is older that the
> + * snapbuilder's current xmin; this should normally be fine because that
> + * information will be updated from other xl_running_xacts records.
> + * 2. standby_redo will likewise skip processing such a record
> *
> (are there other things that should be mentioned?)
I added something like that, but - due to the reference to
SnapBuildProcessRunningXacts() - less verbose about snapbuild.c.
> Also, LogStandbySnapshot() should have a comment explaining that passing
> a valid dboid is a weird corner case which is to be used with care, and
> that functions X Y and Z are going to ignore snapshots carrying a valid
> dbid.
One more check added in standby_decode() (and mentioned in that function in
the comment).
> Why do we call SnapBuildFindSnapshot() to do this, instead of doing it
> directly in SnapBuildProcessRunningXacts? Seems like it would be more
> straightforward.
Yes, fixed.
One more problem I found when testing w/o background worker
(contrib/test_decoding) was that accessSharedCatalogsInDecoding was not set
back to true. Both AllocateSnapshotBuilder() FreeSnapshotBuilder() do that
now.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-02 00:19 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-04-02 00:19 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi Antonin,
On Wed, Apr 1, 2026 at 10:36 PM Antonin Houska <ah@cybertec.at> wrote:
> Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
>
> > i was fuzz testing v48 , and found a crash when REPACK (concurrently)
> itself errors out,
> > 1) while running
> >
> > create table test(id int);
> > REPACK (concurrently) test;
> >
> > TBH i didn't knew this, sometimes it's better to not know "rules" ;)
> > [NOTE: maybe we should add that we can't run
> > REPACK (concurrently) on table without identity index or primary key
> into repack.sgml]
> >
> > ERROR: cannot process relation "test"
> > 2026-04-01 19:06:31.211 IST [2495575] HINT: Relation "test" has no
> identity index.
> > 2026-04-01 19:06:31.211 IST [2495575] STATEMENT: repack (concurrently)
> test;
> > TRAP: failed Assert("proc->statusFlags ==
> ProcGlobal->statusFlags[proc->pgxactoff]"), File: "procarray.c", Line: 719,
> PID: 2495575
> > Here's the diff to solve this crash.
>
> Thanks. Attached here is v48-0006 fixed.
>
On Wed, Apr 1, 2026 at 8:25 PM Srinath Reddy Sadipiralla <
srinath2133@gmail.com> wrote:
> Here's the diff to solve this crash.
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
> index 29ba49744eb..d44092a407a 100644
> --- a/src/backend/commands/repack.c
> +++ b/src/backend/commands/repack.c
> @@ -284,7 +284,23 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool
> isTopLevel)
> * that others can conflict with.
> */
> if ((params.options & CLUOPT_CONCURRENT) != 0)
> + {
> + /*
> + * Do not let other backends wait for our completion during their
> + * setup of logical replication. Unlike logical replication publisher,
> + * we will have XID assigned, so the other backends - whether
> + * walsenders involved in logical replication or regular backends
> + * executing also REPACK (CONCURRENTLY) - would have to wait for our
> + * completion before they can build their initial snapshot. It is o.k.
+ * for any decoding backend to ignore us because we do not change
> + * tuple descriptor of any table, and the data changes we write should
> + * not be decoded by other backends.
> + */
> + LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
> MyProc->statusFlags |= PROC_IN_CONCURRENT_REPACK;
> + ProcGlobal->statusFlags[MyProc->pgxactoff] = MyProc->statusFlags;
> + LWLockRelease(ProcArrayLock);
> + }
>
> /*
> * If a single relation is specified, process it and we're done ... unless
> @@ -988,22 +1004,6 @@ rebuild_relation(Relation OldHeap, Relation index,
> bool verbose,
>
> if (concurrent)
> {
> - /*
> - * Do not let other backends wait for our completion during their
> - * setup of logical replication. Unlike logical replication publisher,
> - * we will have XID assigned, so the other backends - whether
> - * walsenders involved in logical replication or regular backends
> - * executing also REPACK (CONCURRENTLY) - would have to wait for our
> - * completion before they can build their initial snapshot. It is o.k.
> - * for any decoding backend to ignore us because we do not change
> - * tuple descriptor of any table, and the data changes we write should
> - * not be decoded by other backends.
> - */
> - LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
> - MyProc->statusFlags |= PROC_IN_CONCURRENT_REPACK;
> - ProcGlobal->statusFlags[MyProc->pgxactoff] = MyProc->statusFlags;
> - LWLockRelease(ProcArrayLock);
> -
> /*
> * The worker needs to be member of the locking group we're the leader
> * of. We ought to become the leader before the worker starts. The
i think as i did earlier in the diff, shouldn't we remove the duplicate code
from rebuild_relation, am i missing something?
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-01 08:42 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 09:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Amit Kapila @ 2026-04-01 08:42 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Tue, Mar 31, 2026 at 8:06 PM Antonin Houska <ah@cybertec.at> wrote:
>
> Amit Kapila <amit.kapila16@gmail.com> wrote:
> > 2.
> > *
> > + /*
> > + * TODO Consider a GUC to reserve certain amount of replication slots for
> > + * REPACK (CONCURRENTLY) and using it here.
> > + */
> > +#define MAX_REPACK_XIDS 16
> > +
> >
> > This sounds a bit scary as reserving replication slots for REPACK
> > (CONCURRENTLY) may not be what users expect. But it is not clear why
> > replication slots need to be reserved for this.
>
> The point is that REPACK should not block replication [1]. Thus reserving
> slots for non-REPACK users is probably more precise statement.
>
oh, so shouldn't be a separate patch than this and an important for
this functionality to get committed? I don't see why we need to make
such a GUC or knob as part of this patch if we need the same.
> > IIUC, two reasons why logical decoding can ignore REPACK
> > (CONCURRENTLY) are (a) does not perform any catalog changes relevant
> > to logical decoding, (b) neither walsenders nor backends performing
> > logical decoding needs to care sending the WAL generated by REPACK
> > (CONCURRENTLY). Is that understanding correct? If so, we may want to
> > clarify why we want to ignore this command's WAL while sending changes
> > downstream in the commit message or give some reference of the patch
> > where the same is mentioned. This can help reviewing this patch
> > independently.
>
> Correct, but in fact this diff only affects the setup of the logical decoding
> rather than the decoding itself. On the other hand, if REPACK (CONCURRENTLY)
> starts when the decoding backend's snapshot builder is already in the
> SNAPBUILD_FULL_SNAPSHOT state, reorderbuffer.c processes the transaction
> normally, and another part of the series (v46-0002) ensures that the table
> rewriting is not decoded: REPACK simply tells heap_insert(), heap_update(),
> heap_delete() not to put the extra (replication specific) information into the
> corresponding WAL records. I suppose this is what you mean in (b).
>
> Regarding (a), yes, the absence of catalog changes in the REPACK's transaction
> is the reason that even the logical decoding setup does not have to wait for
> the transaction to finish.
>
Hmm, but we don't do any catalog changes for transactions that have
DML say only INSERT but we don't have separate logic like REPACK in
snapbuild machinery. Same is probably true for dml operations on an
unlogged table which doesn't have WAL to send nor any catalog
operations involved but we don't have any special path for that in
snapbuild code path. So, why do we need special handling for this
operation?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-01 09:43 ` Antonin Houska <ah@cybertec.at>
2026-04-01 10:22 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-04-01 09:43 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Tue, Mar 31, 2026 at 8:06 PM Antonin Houska <ah@cybertec.at> wrote:
> >
> > Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > 2.
> > > *
> > > + /*
> > > + * TODO Consider a GUC to reserve certain amount of replication slots for
> > > + * REPACK (CONCURRENTLY) and using it here.
> > > + */
> > > +#define MAX_REPACK_XIDS 16
> > > +
> > >
> > > This sounds a bit scary as reserving replication slots for REPACK
> > > (CONCURRENTLY) may not be what users expect. But it is not clear why
> > > replication slots need to be reserved for this.
> >
> > The point is that REPACK should not block replication [1]. Thus reserving
> > slots for non-REPACK users is probably more precise statement.
> >
>
> oh, so shouldn't be a separate patch than this and an important for
> this functionality to get committed? I don't see why we need to make
> such a GUC or knob as part of this patch if we need the same.
REPACK is a new user of replication slots. Without that, there is no other way
to "steal" the slots from the replication users.
> > > IIUC, two reasons why logical decoding can ignore REPACK
> > > (CONCURRENTLY) are (a) does not perform any catalog changes relevant
> > > to logical decoding, (b) neither walsenders nor backends performing
> > > logical decoding needs to care sending the WAL generated by REPACK
> > > (CONCURRENTLY). Is that understanding correct? If so, we may want to
> > > clarify why we want to ignore this command's WAL while sending changes
> > > downstream in the commit message or give some reference of the patch
> > > where the same is mentioned. This can help reviewing this patch
> > > independently.
> >
> > Correct, but in fact this diff only affects the setup of the logical decoding
> > rather than the decoding itself. On the other hand, if REPACK (CONCURRENTLY)
> > starts when the decoding backend's snapshot builder is already in the
> > SNAPBUILD_FULL_SNAPSHOT state, reorderbuffer.c processes the transaction
> > normally, and another part of the series (v46-0002) ensures that the table
> > rewriting is not decoded: REPACK simply tells heap_insert(), heap_update(),
> > heap_delete() not to put the extra (replication specific) information into the
> > corresponding WAL records. I suppose this is what you mean in (b).
> >
> > Regarding (a), yes, the absence of catalog changes in the REPACK's transaction
> > is the reason that even the logical decoding setup does not have to wait for
> > the transaction to finish.
> >
>
> Hmm, but we don't do any catalog changes for transactions that have
> DML say only INSERT but we don't have separate logic like REPACK in
> snapbuild machinery. Same is probably true for dml operations on an
> unlogged table which doesn't have WAL to send nor any catalog
> operations involved but we don't have any special path for that in
> snapbuild code path. So, why do we need special handling for this
> operation?
If an "ordinary" transaction, which had started before the snapshot builder
reached the SNAPBUILD_FULL_SNAPSHOT state, runs DML, the snapshot builder does
not know if the same transaction changed something in the catalog earlier. So
it needs to wait for this transaction to finish before it advances to
SNAPBUILD_CONSISTENT. For REPACK, we know that it cannot happen because it
cannot run in transaction block for other reasons. So the snapshot builder
does not have to wait.
Nevertheless, I'm not sure it's a good idea for snapbuild.c to handle special
cases like REPACK. Instead, I'm thinking if snapbuild.c can safely ignore XIDs
of backends connected to databases other than the one we're decoding. Thus the
restriction would be one backend running REPACK per database rather than
cluster.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 09:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-04-01 10:22 ` Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 11:42 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 12:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Amit Kapila @ 2026-04-01 10:22 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, Apr 1, 2026 at 3:13 PM Antonin Houska <ah@cybertec.at> wrote:
>
> Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > On Tue, Mar 31, 2026 at 8:06 PM Antonin Houska <ah@cybertec.at> wrote:
> > >
> > > Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > > 2.
> > > > *
> > > > + /*
> > > > + * TODO Consider a GUC to reserve certain amount of replication slots for
> > > > + * REPACK (CONCURRENTLY) and using it here.
> > > > + */
> > > > +#define MAX_REPACK_XIDS 16
> > > > +
> > > >
> > > > This sounds a bit scary as reserving replication slots for REPACK
> > > > (CONCURRENTLY) may not be what users expect. But it is not clear why
> > > > replication slots need to be reserved for this.
> > >
> > > The point is that REPACK should not block replication [1]. Thus reserving
> > > slots for non-REPACK users is probably more precise statement.
> > >
> >
> > oh, so shouldn't be a separate patch than this and an important for
> > this functionality to get committed? I don't see why we need to make
> > such a GUC or knob as part of this patch if we need the same.
>
> REPACK is a new user of replication slots. Without that, there is no other way
> to "steal" the slots from the replication users.
>
> > > > IIUC, two reasons why logical decoding can ignore REPACK
> > > > (CONCURRENTLY) are (a) does not perform any catalog changes relevant
> > > > to logical decoding, (b) neither walsenders nor backends performing
> > > > logical decoding needs to care sending the WAL generated by REPACK
> > > > (CONCURRENTLY). Is that understanding correct? If so, we may want to
> > > > clarify why we want to ignore this command's WAL while sending changes
> > > > downstream in the commit message or give some reference of the patch
> > > > where the same is mentioned. This can help reviewing this patch
> > > > independently.
> > >
> > > Correct, but in fact this diff only affects the setup of the logical decoding
> > > rather than the decoding itself. On the other hand, if REPACK (CONCURRENTLY)
> > > starts when the decoding backend's snapshot builder is already in the
> > > SNAPBUILD_FULL_SNAPSHOT state, reorderbuffer.c processes the transaction
> > > normally, and another part of the series (v46-0002) ensures that the table
> > > rewriting is not decoded: REPACK simply tells heap_insert(), heap_update(),
> > > heap_delete() not to put the extra (replication specific) information into the
> > > corresponding WAL records. I suppose this is what you mean in (b).
> > >
> > > Regarding (a), yes, the absence of catalog changes in the REPACK's transaction
> > > is the reason that even the logical decoding setup does not have to wait for
> > > the transaction to finish.
> > >
> >
> > Hmm, but we don't do any catalog changes for transactions that have
> > DML say only INSERT but we don't have separate logic like REPACK in
> > snapbuild machinery. Same is probably true for dml operations on an
> > unlogged table which doesn't have WAL to send nor any catalog
> > operations involved but we don't have any special path for that in
> > snapbuild code path. So, why do we need special handling for this
> > operation?
>
> If an "ordinary" transaction, which had started before the snapshot builder
> reached the SNAPBUILD_FULL_SNAPSHOT state, runs DML, the snapshot builder does
> not know if the same transaction changed something in the catalog earlier. So
> it needs to wait for this transaction to finish before it advances to
> SNAPBUILD_CONSISTENT. For REPACK, we know that it cannot happen because it
> cannot run in transaction block for other reasons. So the snapshot builder
> does not have to wait.
>
> Nevertheless, I'm not sure it's a good idea for snapbuild.c to handle special
> cases like REPACK. Instead, I'm thinking if snapbuild.c can safely ignore XIDs
> of backends connected to databases other than the one we're decoding.
>
What if such transactions have made changes in the global catalog?
Even if that won't matter, I feel such a change would be quite
fundamental to snapbuild machinery and changing at this point would be
risky.
BTW, is the reason to skip REPACK while building a snapshot is that it
can take a long time to finish?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 09:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 10:22 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-01 11:42 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 12:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 13:20 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Alvaro Herrera @ 2026-04-01 11:42 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Apr-01, Amit Kapila wrote:
> BTW, is the reason to skip REPACK while building a snapshot is that it
> can take a long time to finish?
As I understand the issue, yes, that's precisely the problem: if you
have one REPACK running, then starting a second REPACK (which requires
building a new snapshot) would have to wait until the first REPACK is
over. In other words, you wouldn't be able to have two repacks running
concurrently. This sounds like a problematic requirement. So having
snapbuild ignore REPACK is there to allow the second REPACK to work at
all. But more generally, *all* users of snapbuild would be prevented
from starting until REPACK is done; so if you have a very very large
table that takes a long time to repack, then everything would be blocked
behind it until it completes, which sounds extremely unpleasant.
So, if we're unable to get this particular patch in, we would have to
have a big fat warning in the docs, telling people to be careful about
other load if they choose to run concurrent repack -- it could have
serious consequences. But on the other hand, it's better to *have* the
tool, even if it has problems, than not have it. Keep in mind that
pg_repack and pg_squeeze also have all these problems/limitations (and
others), and still people use them.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 09:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 10:22 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 11:42 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-01 12:35 ` Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Amit Kapila @ 2026-04-01 12:35 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Antonin Houska <ah@cybertec.at>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On Wed, Apr 1, 2026 at 5:12 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2026-Apr-01, Amit Kapila wrote:
>
> > BTW, is the reason to skip REPACK while building a snapshot is that it
> > can take a long time to finish?
>
> As I understand the issue, yes, that's precisely the problem: if you
> have one REPACK running, then starting a second REPACK (which requires
> building a new snapshot) would have to wait until the first REPACK is
> over. In other words, you wouldn't be able to have two repacks running
> concurrently. This sounds like a problematic requirement. So having
> snapbuild ignore REPACK is there to allow the second REPACK to work at
> all. But more generally, *all* users of snapbuild would be prevented
> from starting until REPACK is done; so if you have a very very large
> table that takes a long time to repack, then everything would be blocked
> behind it until it completes, which sounds extremely unpleasant.
>
> So, if we're unable to get this particular patch in, we would have to
> have a big fat warning in the docs, telling people to be careful about
> other load if they choose to run concurrent repack -- it could have
> serious consequences.
>
Right, I think during this time logical workers will keep timing out
and restarting without any progress because during this wait, we won't
be sending keep_alive messages.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 09:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 10:22 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 11:42 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-04-01 13:20 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-01 13:20 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> So, if we're unable to get this particular patch in, we would have to
> have a big fat warning in the docs, telling people to be careful about
> other load if they choose to run concurrent repack -- it could have
> serious consequences. But on the other hand, it's better to *have* the
> tool, even if it has problems, than not have it. Keep in mind that
> pg_repack and pg_squeeze also have all these problems/limitations (and
> others), and still people use them.
1. To be precise, pg_squeeze has this limitation. pg_repack does not use
logical replication.
2. I expect the limitation of PG core to be relaxed in versions > 19, as long
as we integrate the MVCC safety feature. REPACK will then run w/o XID most of
the time (XID will only be needed for catalog changes), so other decoding
backends won't need to wait for its completion.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 09:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 10:22 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
@ 2026-04-01 12:31 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-04-01 12:31 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Matthias van de Meent <boekewurm+postgres@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Wed, Apr 1, 2026 at 3:13 PM Antonin Houska <ah@cybertec.at> wrote:
> >
> > Nevertheless, I'm not sure it's a good idea for snapbuild.c to handle special
> > cases like REPACK. Instead, I'm thinking if snapbuild.c can safely ignore XIDs
> > of backends connected to databases other than the one we're decoding.
> >
>
> What if such transactions have made changes in the global catalog?
> Even if that won't matter, I feel such a change would be quite
> fundamental to snapbuild machinery and changing at this point would be
> risky.
I had thought that catalog is usually needed only to retrieve the tuple
descriptor, but regression tests with some Assert() statements prove now that
shared catalogs can be accessed too. So that idea does not seem to be useful.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-03-16 13:46 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-16 13:46 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> I offer the following rather trivial fixup diffs, which I think should
> be mostly be for 0002.
This is a proposal to fix one more problem I had on my list: index build
should not report its progress if invoked by REPACK. More info in the commit
message and in the patch itself.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-25 13:55 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-02-25 13:55 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello,
I did stress testing on v35 patches, where I did concurrency test using
pgbench with 50 concurrent clients, 4 threads with the below pgbench
script (dual_chaos.sql) on the following table setup(setup.sql).
I ran pgbench with 5M rows for 10 minutes and 50M for ~45 minutes
multiple times. REPACK (concurrently) ran successfully except "once"(see
below).
I created a shadow/clone table to use for checking the correctness after
doing
the concurrency test.I used 4 checks to verify that data is intact and
REPACK (concurrently) ran successfully.
1) table file OID(relfilenode) swapped?
2) bloat gone? victim relation size should be less than
shadow relation size.
3) using FULL JOIN logic (borrowed from repack.spec, with small change)
against the shadow table which goes under the same concurrent ops
done on the victim table , basically doing dual writes (see dual_chaos.sql)
to
verify table data integrity.
4) Physical Index Integrity (amcheck) (borrowed from Mihail's tests)
The concurrency test failed once. I tried to reproduce the below scenario
but no luck,i think the reason the assert failure happened because
after speculative insert there might be no spec CONFIRM or ABORT, thoughts?
TRAP: failed Assert("!specinsert"), File: "reorderbuffer.c", Line: 2610,
PID: 3956168
postgres: REPACK decoding worker for relation "stress_victim"
(ExceptionalCondition+0x98)[0xaaaab1251188]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x67b1cc)[0xaaaab0f4b1cc]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x67b86c)[0xaaaab0f4b86c]
postgres: REPACK decoding worker for relation "stress_victim"
(ReorderBufferCommit+0x74)[0xaaaab0f4b8f0]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x66229c)[0xaaaab0f3229c]
postgres: REPACK decoding worker for relation "stress_victim"
(xact_decode+0x1a0)[0xaaaab0f312bc]
postgres: REPACK decoding worker for relation "stress_victim"
(LogicalDecodingProcessRecord+0xd4)[0xaaaab0f30e60]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x3372e4)[0xaaaab0c072e4]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x339634)[0xaaaab0c09634]
postgres: REPACK decoding worker for relation "stress_victim"
(RepackWorkerMain+0x1ac)[0xaaaab0c094e8]
postgres: REPACK decoding worker for relation "stress_victim"
(BackgroundWorkerMain+0x2b0)[0xaaaab0efc440]
postgres: REPACK decoding worker for relation "stress_victim"
(postmaster_child_launch+0x1f0)[0xaaaab0f00398]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x639ca4)[0xaaaab0f09ca4]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x639f94)[0xaaaab0f09f94]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x638714)[0xaaaab0f08714]
postgres: REPACK decoding worker for relation "stress_victim"
(+0x635978)[0xaaaab0f05978]
postgres: REPACK decoding worker for relation "stress_victim"
(PostmasterMain+0x160c)[0xaaaab0f050c8]
postgres: REPACK decoding worker for relation "stress_victim"
(main+0x3dc)[0xaaaab0d974d4]
/lib/aarch64-linux-gnu/libc.so.6(+0x284c4)[0xffff867584c4]
/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0x98)[0xffff86758598]
postgres: REPACK decoding worker for relation "stress_victim"
(_start+0x30)[0xaaaab09bc1f0]
2026-02-19 18:20:56.088 IST [3905812] LOG: checkpoint starting: wal
2026-02-19 18:21:10.683 IST [3905808] LOG: background worker "REPACK
decoding worker" (PID 3956168) was terminated by signal 6: Aborted
Crash Test:
i did crash test using debugger using a breakpoint
inside apply_concurrent_changes
to simulate a crash while concurrent changes are being done, after few
concurrent changes
are done , i crashed the server using "pg_ctl -m immediate stop", then
restarted the server,
i observed that REPACK (concurrently) didn't completed (expected), files
were not swapped and data
on the victim table is intact checked using FULL JOIN with shadow table,
but there are
some leftovers of the transient table we used for REPACK (concurrently)
such as
1) transient table's relation files - these consume extra space , i think
this was the
case with VACUUM FULL previously, so these has to be removed manually , but
I think this time we have a "leverage" which we can use to remove the extra
space.
2) transient table's WALs - these are generated because of concurrent
changes done while
applying the logical decoded changes on the new transient table, i think
this won't be a problem
until they only will get recycled but if they get archived , they are of no
use instead they
consume more space and time during the archival process.
"Leverage" Idea:
i think we can re-use these transient table's relation files and WALs
during crash recovery,
so that user don't have to re-run the REPACK (concurrently) after server
has recovered,
for this we might need to write a WAL for REPACK (concurrently) to let
startup process
know REPACK (concurrently) occurred which sets a flag, so at the end of
startup process
all the WALs of the transient table are already applied so transient table
perfect now ,
at the end we can do swapping (finish_heap_swap) after checking the flag ,
these are
all my initial thoughts on this idea to reuse the "residue" files of the
transient table.
I could be totally wrong :) Please correct me if I am.
i think we need to update this statement in repack.sgml regarding wal_level
<listitem>
<para>
The <link
linkend="guc-wal-level"><varname>wal_level</varname></link>
configuration parameter is less than <literal>logical</literal>.
</para>
</listitem>
because of this commit POC: enable logical decoding when wal_level =
'replica' without a server restart (67c2097)
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] setup.sql (390B, ../../CAFC+b6o2yzA80YmfEhmMO9puN8qvGRvr-15BBLn3UmJxPfpr2w@mail.gmail.com/3-setup.sql)
download
[application/octet-stream] dual_chaos.sql (891B, ../../CAFC+b6o2yzA80YmfEhmMO9puN8qvGRvr-15BBLn3UmJxPfpr2w@mail.gmail.com/4-dual_chaos.sql)
download
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-02-25 19:41 ` Antonin Houska <ah@cybertec.at>
2026-02-26 12:19 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-02-25 19:41 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> I did stress testing on v35 patches, where I did concurrency test using
> pgbench with 50 concurrent clients, 4 threads with the below pgbench
> script (dual_chaos.sql) on the following table setup(setup.sql).
> I ran pgbench with 5M rows for 10 minutes and 50M for ~45 minutes
> multiple times. REPACK (concurrently) ran successfully except "once"(see below).
> I created a shadow/clone table to use for checking the correctness after doing
> the concurrency test.I used 4 checks to verify that data is intact and
> REPACK (concurrently) ran successfully.
>
> 1) table file OID(relfilenode) swapped?
> 2) bloat gone? victim relation size should be less than
> shadow relation size.
> 3) using FULL JOIN logic (borrowed from repack.spec, with small change)
> against the shadow table which goes under the same concurrent ops
> done on the victim table , basically doing dual writes (see dual_chaos.sql) to
> verify table data integrity.
> 4) Physical Index Integrity (amcheck) (borrowed from Mihail's tests)
Thanks!
> The concurrency test failed once. I tried to reproduce the below scenario
> but no luck,i think the reason the assert failure happened because
> after speculative insert there might be no spec CONFIRM or ABORT, thoughts?
Perhaps, I'll try. I'm not sure the REPACK decoding worker does anthing
special regarding decoding. If you happen to see the problem again, please try
to preserve the related WAL segments - if this is a bug in PG executor,
pg_waldump might reveal that.
> Crash Test:
> i did crash test using debugger using a breakpoint inside apply_concurrent_changes
> to simulate a crash while concurrent changes are being done, after few concurrent changes
> are done , i crashed the server using "pg_ctl -m immediate stop", then restarted the server,
> i observed that REPACK (concurrently) didn't completed (expected), files were not swapped and data
> on the victim table is intact checked using FULL JOIN with shadow table, but there are
> some leftovers of the transient table we used for REPACK (concurrently) such as
> 1) transient table's relation files - these consume extra space , i think this was the
> case with VACUUM FULL previously, so these has to be removed manually , but
> I think this time we have a "leverage" which we can use to remove the extra space.
> 2) transient table's WALs - these are generated because of concurrent changes done while
> applying the logical decoded changes on the new transient table, i think this won't be a problem
> until they only will get recycled but if they get archived , they are of no use instead they
> consume more space and time during the archival process.
VACUUM FULL / CLUSTER also produces (a lot of) WAL, so IMO there's nothing
specific about REPACK.
Regarding the transient table, I have a draft patch (for future versions) that
creates the transient table in a separate transaction and commits it. (This is
part of the effort to not block the progress of VACUUM xmin horizon. The point
is that most of the time REPACK should not have XID assigned.) With this
design, each time REPACK starts, it checks (in the pg_depend catalog) if the
transient table exists for particular table, and if it does, it drops it.
> "Leverage" Idea:
> i think we can re-use these transient table's relation files and WALs during crash recovery,
> so that user don't have to re-run the REPACK (concurrently) after server has recovered,
> for this we might need to write a WAL for REPACK (concurrently) to let startup process
> know REPACK (concurrently) occurred which sets a flag, so at the end of startup process
> all the WALs of the transient table are already applied so transient table perfect now ,
> at the end we can do swapping (finish_heap_swap) after checking the flag , these are
> all my initial thoughts on this idea to reuse the "residue" files of the transient table.
> I could be totally wrong :) Please correct me if I am.
I think it'd be quite difficult to restart REPACK exactly at the point it
crashed. Especially if the tables are unlocked between server restart and the
restart of REPACK.
> i think we need to update this statement in repack.sgml regarding wal_level
> <listitem>
> <para>
> The <link linkend="guc-wal-level"><varname>wal_level</varname></link>
> configuration parameter is less than <literal>logical</literal>.
> </para>
> </listitem>
> because of this commit POC: enable logical decoding when wal_level = 'replica' without a server restart (67c2097)
I'm aware of this commit and already updated regression tests, however forgot
to update the user documentation. Thanks for reminder.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-02-26 12:19 ` Álvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 0 replies; 416+ messages in thread
From: Álvaro Herrera @ 2026-02-26 12:19 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Feb-25, Antonin Houska wrote:
> Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> > 2) transient table's WALs - these are generated because of
> > concurrent changes done while applying the logical decoded changes
> > on the new transient table, i think this won't be a problem until
> > they only will get recycled but if they get archived , they are of
> > no use instead they consume more space and time during the archival
> > process.
>
> VACUUM FULL / CLUSTER also produces (a lot of) WAL, so IMO there's
> nothing specific about REPACK.
Yeah, I don't think there's anything we can (or should) do about this.
It's just writing to the common WAL stream, which means that any
non-repack concurrent load is going to be producing WAL messages
interspersed with what is being done for repack. So it's not possible
to skip archiving those files, or anything of the sort.
In fact, these messages being written to WAL is how REPACK is going to
be transmitted to replicas.
(Actually, now that I think about this, I realize that I don't know how
this part really works, and I should. Back to code review it is then.)
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"At least to kernel hackers, who really are human, despite occasional
rumors to the contrary" (LWN.net)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-06 07:14 ` Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-06 07:14 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
>
> > The concurrency test failed once. I tried to reproduce the below scenario
> > but no luck,i think the reason the assert failure happened because
> > after speculative insert there might be no spec CONFIRM or ABORT, thoughts?
>
> Perhaps, I'll try. I'm not sure the REPACK decoding worker does anthing
> special regarding decoding. If you happen to see the problem again, please try
> to preserve the related WAL segments - if this is a bug in PG executor,
> pg_waldump might reveal that.
I could not reproduce the failure, and have no idea how speculative insert can
stay w/o CONFIRM / ABORT record. The only problem I could imagine is that
change_useless_for_repack() filters out the CONFIRM / ABORT record
accidentally, but neither code review nor debugger proves that
theory. (Actually if this was the problem, the test failure probably wouldn't
be that rare.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-20 18:06 ` Antonin Houska <ah@cybertec.at>
2026-03-23 12:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-20 18:06 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Antonin Houska <ah@cybertec.at> wrote:
>
> > Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> >
> > > The concurrency test failed once. I tried to reproduce the below scenario
> > > but no luck,i think the reason the assert failure happened because
> > > after speculative insert there might be no spec CONFIRM or ABORT, thoughts?
> >
> > Perhaps, I'll try. I'm not sure the REPACK decoding worker does anthing
> > special regarding decoding. If you happen to see the problem again, please try
> > to preserve the related WAL segments - if this is a bug in PG executor,
> > pg_waldump might reveal that.
>
> I could not reproduce the failure, and have no idea how speculative insert can
> stay w/o CONFIRM / ABORT record. The only problem I could imagine is that
> change_useless_for_repack() filters out the CONFIRM / ABORT record
> accidentally, but neither code review nor debugger proves that
> theory. (Actually if this was the problem, the test failure probably wouldn't
> be that rare.)
I confirm that I was able to reproduce the crash using debugger and your more
recent diagnosis [1]. Indeed, filtering was the problem.
Unfortunately, I wasn't able to make the crash easily reproducible using
isolation tester. The problem is that the logical decoding is performed by a
background worker, and when the backend executing REPACK waits for the
background worker, which in turn waits on an injection point, the isolation
tester does not recognize that it's effectively the backend who is waiting on
the injection point. Therefore the isolation tester does not proceed to the
next step.
Anyway, thanks again for your testing!
[1] https://www.postgresql.org/message-id/CAFC%2Bb6qk3-DQTi43QMqvVLP%2BsudPV4vsLQm5iHfcCeObrNaVyA%40mail...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-23 12:00 ` Antonin Houska <ah@cybertec.at>
2026-03-23 16:07 ` Re: Adding REPACK [concurrently] Jim Jones <jim.jones@uni-muenster.de>
2026-03-26 13:19 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
0 siblings, 2 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-23 12:00 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Antonin Houska <ah@cybertec.at> wrote:
> Antonin Houska <ah@cybertec.at> wrote:
>
> > Antonin Houska <ah@cybertec.at> wrote:
> >
> > > Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> > >
> > > > The concurrency test failed once. I tried to reproduce the below scenario
> > > > but no luck,i think the reason the assert failure happened because
> > > > after speculative insert there might be no spec CONFIRM or ABORT, thoughts?
> > >
> > > Perhaps, I'll try. I'm not sure the REPACK decoding worker does anthing
> > > special regarding decoding. If you happen to see the problem again, please try
> > > to preserve the related WAL segments - if this is a bug in PG executor,
> > > pg_waldump might reveal that.
> >
> > I could not reproduce the failure, and have no idea how speculative insert can
> > stay w/o CONFIRM / ABORT record. The only problem I could imagine is that
> > change_useless_for_repack() filters out the CONFIRM / ABORT record
> > accidentally, but neither code review nor debugger proves that
> > theory. (Actually if this was the problem, the test failure probably wouldn't
> > be that rare.)
>
> I confirm that I was able to reproduce the crash using debugger and your more
> recent diagnosis [1]. Indeed, filtering was the problem.
>
> Unfortunately, I wasn't able to make the crash easily reproducible using
> isolation tester. The problem is that the logical decoding is performed by a
> background worker, and when the backend executing REPACK waits for the
> background worker, which in turn waits on an injection point, the isolation
> tester does not recognize that it's effectively the backend who is waiting on
> the injection point. Therefore the isolation tester does not proceed to the
> next step.
I could not resist digging in it deeper :-) Attached is a test that reproduces
the crash - it includes the isolation tester enhancement that I posted
separately [1]. It crashes reliably with v43 [2] if your fix v43-0005 is
omitted.
[1] https://www.postgresql.org/message-id/4703.1774250534%40localhost
[2] https://www.postgresql.org/message-id/202603191855.fzsgsnyzfvpt%40alvherre.pgsql
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 12:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-23 16:07 ` Jim Jones <jim.jones@uni-muenster.de>
2026-03-24 08:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Jim Jones @ 2026-03-23 16:07 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi,
while reviewing another patch I noticed that REPACK is trying to access
temp tables from other sessions.
== session 1 ==
$ psql postgres
psql (19devel)
Type "help" for help.
postgres=# SELECT pg_backend_pid();
pg_backend_pid
----------------
730392
(1 row)
postgres=# CREATE TEMP TABLE tmp AS SELECT generate_series(1, 1000) AS id;
SELECT 1000
postgres=# BEGIN;
LOCK TABLE tmp IN SHARE MODE;
BEGIN
LOCK TABLE
postgres=*#
== session 2 ==
$ psql postgres
psql (19devel)
Type "help" for help.
postgres=# REPACK;
(waits for LOCK)
== session 3 ==
$ psql postgres
psql (19devel)
Type "help" for help.
postgres=# SELECT pid, relation::regclass, mode, granted
FROM pg_locks
WHERE relation::regclass::text ~~ '%.tmp%';
pid | relation | mode | granted
--------+----------------+---------------------+---------
730608 | pg_temp_12.tmp | AccessExclusiveLock | f
730392 | pg_temp_12.tmp | ShareLock | t
(2 rows)
The same applies for REPACK USING INDEX if indisclustered is true.
I played a bit with the code and perhaps skipping temp relations in
get_tables_to_repack() before they're added to the list can do the
trick. I tried the draft attached and REPACK could run despite the LOCK
in the other session... in case it helps.
Best, Jim
Attachments:
[text/x-patch] nocfbot-0001-Fix-REPACK-to-skip-temporary-tables-of-other-sess.patch (1.7K, ../../713021dd-bf6c-407a-8c9d-f79ecbd1c2ee@uni-muenster.de/2-nocfbot-0001-Fix-REPACK-to-skip-temporary-tables-of-other-sess.patch)
download | inline diff:
From be0f2aeeaa53e775d5312be87d0ad34f6cd8c271 Mon Sep 17 00:00:00 2001
From: Jim Jones <jim.jones@uni-muenster.de>
Date: Mon, 23 Mar 2026 16:20:09 +0100
Subject: [PATCH v1] Fix REPACK to skip temporary tables of other sessions
get_tables_to_repack() was adding other sessions' temporary tables
to the work list, causing REPACK to attempt to acquire
AccessExclusiveLock on them. Fix by skipping other-session temp
relations early in get_tables_to_repack(), before they are added
to the list.
---
src/backend/commands/cluster.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 09066db095..f701d426dd 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1699,6 +1699,13 @@ get_tables_to_repack(RepackCommand cmd, bool usingindex, MemoryContext permcxt)
continue;
}
+ /* Skip temp relations belonging to other sessions */
+ if (isOtherTempNamespace(get_rel_namespace(index->indrelid)))
+ {
+ UnlockRelationOid(index->indrelid, AccessShareLock);
+ continue;
+ }
+
/* noisily skip rels which the user can't process */
if (!repack_is_permitted_for_relation(cmd, index->indrelid,
GetUserId()))
@@ -1753,6 +1760,14 @@ get_tables_to_repack(RepackCommand cmd, bool usingindex, MemoryContext permcxt)
continue;
}
+ /* Skip temp relations belonging to other sessions */
+ if (class->relpersistence == RELPERSISTENCE_TEMP &&
+ isOtherTempNamespace(class->relnamespace))
+ {
+ UnlockRelationOid(class->oid, AccessShareLock);
+ continue;
+ }
+
/* noisily skip rels which the user can't process */
if (!repack_is_permitted_for_relation(cmd, class->oid,
GetUserId()))
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 12:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 16:07 ` Re: Adding REPACK [concurrently] Jim Jones <jim.jones@uni-muenster.de>
@ 2026-03-24 08:48 ` Antonin Houska <ah@cybertec.at>
2026-03-24 12:13 ` Re: Adding REPACK [concurrently] Jim Jones <jim.jones@uni-muenster.de>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-03-24 08:48 UTC (permalink / raw)
To: Jim Jones <jim.jones@uni-muenster.de>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Jim Jones <jim.jones@uni-muenster.de> wrote:
> while reviewing another patch I noticed that REPACK is trying to access
> temp tables from other sessions.
Thanks for the report. I agree that there's no reason for REPACK to get
blocked on a lock of a table that it will not process anyway. However, as both
VACUUM FULL and CLUSTER in PG 18 appear to have the same problem, I'm not sure
your patch needs to be incorporated in the new feature. Adding a bug fix entry
to the CF seems the appropriate action to me.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 12:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 16:07 ` Re: Adding REPACK [concurrently] Jim Jones <jim.jones@uni-muenster.de>
2026-03-24 08:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-24 12:13 ` Jim Jones <jim.jones@uni-muenster.de>
0 siblings, 0 replies; 416+ messages in thread
From: Jim Jones @ 2026-03-24 12:13 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 24/03/2026 09:48, Antonin Houska wrote:
> Thanks for the report. I agree that there's no reason for REPACK to get
> blocked on a lock of a table that it will not process anyway. However, as both
> VACUUM FULL and CLUSTER in PG 18 appear to have the same problem, I'm not sure
> your patch needs to be incorporated in the new feature. Adding a bug fix entry
> to the CF seems the appropriate action to me.
Indeed. VACUUM FULL seems to have the same problem.
I'll include it in the patch and open a new thread.
Best, Jim
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 12:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-03-26 13:19 ` Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Srinath Reddy Sadipiralla @ 2026-03-26 13:19 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hi Antonin,
On Mon, Mar 23, 2026 at 5:30 PM Antonin Houska <ah@cybertec.at> wrote:
>
> I could not resist digging in it deeper :-) Attached is a test that
> reproduces
> the crash - it includes the isolation tester enhancement that I posted
> separately [1]. It crashes reliably with v43 [2] if your fix v43-0005 is
> omitted.
>
+1, i tested the same and yeah it reproduces the crash ,also reviewed
the patch LGTM, except i think you need to add this to makefile, so that a
simple
make check would be enough to run the isolation test, i think this all
makes sense
if this [1] patch goes in.
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index acbcaed2feb..201e3c84cb4 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -9,7 +9,10 @@ REGRESS = ddl xact rewrite toast permissions
decoding_in_xact \
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream
\
twophase_snapshot slot_creation_error catalog_change_snapshot \
- skip_snapshot_restore invalidation_distribution
parallel_session_origin
+ skip_snapshot_restore invalidation_distribution
parallel_session_origin \
+ filtering
+
+EXTRA_INSTALL = src/test/modules/injection_points
REGRESS_OPTS = --temp-config
$(top_srcdir)/contrib/test_decoding/logical.conf
ISOLATION_OPTS = --temp-config
$(top_srcdir)/contrib/test_decoding/logical.conf
[1] - https://www.postgresql.org/message-id/flat/4703.1774250534%40localhost
--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 12:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-26 13:19 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
@ 2026-03-26 19:36 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-03-26 19:36 UTC (permalink / raw)
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>; +Cc: alvherre@alvh.no-ip.org; Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Srinath Reddy Sadipiralla <srinath2133@gmail.com> wrote:
> On Mon, Mar 23, 2026 at 5:30 PM Antonin Houska <ah@cybertec.at> wrote:
>
>> I could not resist digging in it deeper :-) Attached is a test that reproduces
>> the crash - it includes the isolation tester enhancement that I posted
>> separately [1]. It crashes reliably with v43 [2] if your fix v43-0005 is
>> omitted.
>
> +1, i tested the same and yeah it reproduces the crash ,also reviewed
> the patch LGTM, except i think you need to add this to makefile, so that a simple
> make check would be enough to run the isolation test, i think this all makes sense
> if this [1] patch goes in.
The purpose of this test was only to reproduce the crash you encountered
during the stress test, and to confirm that your fix works. I didn't really
intend to add the test to the tree.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-20 15:39 ` Antonin Houska <ah@cybertec.at>
2 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-01-20 15:39 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > if (size >= MaxAllocSize)
> Seems like we lost that check, I think it may be executed on storing
> the data
The tuple we process in store_change was created elsewhere (I think in
reorderbuffer.c), so I wouldn't re-check the size here.
> or before "tup = (HeapTuple) palloc(HEAPTUPLESIZE + t_len);"
> in apply_concurrent_changes
It's essentially the same length that we write in store_change() so I wouldn't
bother re-checking it here. In general, I doubt the constant is
appropriate. Its meaning is much more generic than the size of memory for a
tuple and even heap_form_tuple() does not use it.
> > bool done;
> I still think it is a confusing name.
I don't. The last call of process_concurrent_changes() tells the worker "Give
me the the next batch and we are done". Your proposal "exit_after_lsn_upto"
seems to me too verbose: the worker itself is supposed to know that it has to
reach the LSN passed via another argument.
> > chgdst.file_seq = WORKER_FILE_SNAPSHOT + 1;
> I think it is better to increment it once a snapshot is received.
The 'chgdst' is only defined in rebuild_relation_finish_concurrent(), no need
to use it where the snapshot is received (in rebuild_relation()).
> And rename to last_processed/last_improrted to be aligned with
> last_exported.
While DecodingWorkerShared deals with multiple files (not all of them
necessarily available for "consumer") and therefore it makes sense to
distinguish if file is exported or not, each instance of ChangeDest is
assigned particular file and the functions using it do not care if the file is
the last in the sequence or not.
Other proposals accepted, will be reflected in the next version.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-11 20:38 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:45 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 19:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-11 20:38 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
On Tue, Dec 9, 2025 at 7:52 PM Antonin Houska <ah@cybertec.at> wrote:
> Worker makes more sense to me - the initial implementation is in 0005.
Comments for 0005, so far:
---
> export_initial_snapshot
Hm, should we use ExportSnapshot instead? And ImportSnapshort to import it.
---
> get_initial_snapshot
Should we check if a worker is still alive while waiting? Also is
"process_concurrent_changes".
And AFAIU RegisterDynamicBackgroundWorker does not guarantee new
workers to be started (in case of some fork-related issues).
---
> Assert(res = SHM_MQ_DETACHED);
==
---
> /* Wait a bit before we retry reading WAL. */
> (void) WaitLatch(MyLatch,
> WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
> 1000L,
> WAIT_EVENT_REPACK_WORKER_MAIN);
Looks like we need ResetLatch(MyLatch); here.
---
> * - decoding_ctx - logical decoding context, to capture concurrent data
Need to be removed together with parameters.
---
> hpm_context = AllocSetContextCreate(TopMemoryContext,
> "ProcessParallelMessages",
> ALLOCSET_DEFAULT_SIZES);
"ProcessRepacklMessages"
---
> if (XLogRecPtrIsInvalid(lsn_upto))
> {
> SpinLockAcquire(&shared->mutex);
> lsn_upto = shared->lsn_upto;
> /* 'done' should be set at the same time as 'lsn_upto' */
> done = shared->done;
> SpinLockRelease(&shared->mutex);
>
> /* Check if the work happens to be complete. */
> continue;
> }
May be moved to the start of the loop to avoid duplication.
---
> SpinLockAcquire(&shared->mutex);
> valid = shared->sfs_valid;
> SpinLockRelease(&shared->mutex);
Better to remember last_exported here to avoid any races/misses.
---
> shared->lsn_upto = InvalidXLogRecPtr;
I think it is better to clear it once it is read (after removing duplication).
---
> bool done;
bool exit_after_lsn_upto?
---
> bool sfs_valid;
Do we really need it? I think it is better to leave only last_exported
and in process_concurrent_changes wait add argument
(last_processed_file) and wait for last_exported to become higher.
---
What if we reverse roles of leader-worker?
Leader gets a snapshot, transfers it to workers (multiple probably for
parallel scan) using already ready mechanics - workers are processing
the scan of the table in parallel. Leader decodes the WAL.
Also, workers may be assigned with a list of indexes they need to build.
Feels like it reuses more from current infrastructure and also needs
less different synchronization logic. But I'm not sure about the
indexes phase - maybe it is not so easy to do.
---
Also, should we add some kind of back pressure between building
indexes/new heap and num of WAL we have?
But probably it is out of scope of the patch.
---
To build N indexes we need to scan table N times. What is about
building multiple indexes during a single heap scan?
--
Just a gentle reminder about the XMIN_COMMITTED flag and WAL storm
after the switch.
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-13 18:45 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:59 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-18 01:47 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 2 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-13 18:45 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, everyone.
Stress tests for REPACK concurrently in attachment.
So far I can't break anything (except MVCC of course).
A rebased version of the MVCC-safe "light" version with its own stress
test is attached also.
Best regards,
Mikhail.
From 457235c743a2dec2c1917fbdfa7f5a48d305c63e Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sat, 13 Dec 2025 19:42:52 +0100
Subject: [PATCH vnocfbot] Preserve visibility information of the concurrent
data changes.
As explained in the commit message of the preceding patch of the series, the
data changes done by applications while REPACK CONCURRENTLY is copying the
table contents to a new file are decoded from WAL and eventually also applied
to the new file. To reduce the complexity a little bit, the preceding patch
uses the current transaction (i.e. transaction opened by the REPACK command)
to execute those INSERT, UPDATE and DELETE commands.
However, REPACK is not expected to change visibility of tuples. Therefore,
this patch fixes the handling of the "concurrent data changes". It ensures
that tuples written into the new table have the same XID and command ID (CID)
as they had in the old table.
To "replay" an UPDATE or DELETE command on the new table, we use SnapshotSelf to find the last alive version of tuple and update with stamp with xid of original transaction. It is safe because:
* all transactions we replaying are committed
* apply worker working without any concurrent modifiers of the table
As long as we preserve the tuple visibility information (which includes XID),
it's important to avoid logical decoding of the WAL generated by DMLs on the
new table: the logical decoding subsystem probably does not expect that the
incoming WAL records contain XIDs of an already decoded transactions. (And of
course, repeated decoding would be wasted effort.)
Author: Antonin Houska <ah@cybertec.at> with changes from Mikhail Nikalayeu <mihailnikalayeu@gmail.com
---
contrib/amcheck/meson.build | 1 +
.../amcheck/t/009_repack_concurrently_mvcc.pl | 113 ++++++++++++++++++
doc/src/sgml/mvcc.sgml | 12 +-
doc/src/sgml/ref/repack.sgml | 9 --
src/backend/access/common/toast_internals.c | 3 +-
src/backend/access/heap/heapam.c | 29 +++--
src/backend/access/heap/heapam_handler.c | 24 ++--
src/backend/commands/cluster.c | 107 ++++++++++++-----
.../pgoutput_repack/pgoutput_repack.c | 16 ++-
src/include/access/heapam.h | 6 +-
.../injection_points/specs/repack.spec | 4 -
11 files changed, 243 insertions(+), 81 deletions(-)
create mode 100644 contrib/amcheck/t/009_repack_concurrently_mvcc.pl
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index f7c70735989..6946c684259 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -52,6 +52,7 @@ tests += {
't/006_verify_gin.pl',
't/007_repack_concurrently.pl',
't/008_repack_concurrently.pl',
+ 't/009_repack_concurrently_mvcc.pl',
],
},
}
diff --git a/contrib/amcheck/t/009_repack_concurrently_mvcc.pl b/contrib/amcheck/t/009_repack_concurrently_mvcc.pl
new file mode 100644
index 00000000000..a83fd5b8141
--- /dev/null
+++ b/contrib/amcheck/t/009_repack_concurrently_mvcc.pl
@@ -0,0 +1,113 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+));
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl1(i int PRIMARY KEY, j int)));
+$node->safe_psql('postgres', q(CREATE TABLE tbl2(i int PRIMARY KEY, j int)));
+
+
+# Insert 100 rows into tbl1
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl1 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+# Insert 100 rows into tbl2
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl2 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+
+# Insert 100 rows into tbl1
+$node->safe_psql('postgres', q(
+ CREATE OR REPLACE FUNCTION log_raise(i int, j1 int, j2 int) RETURNS VOID AS $$
+ BEGIN
+ RAISE NOTICE 'ERROR i=% j1=% j2=%', i, j1, j2;
+ END;$$ LANGUAGE plpgsql;
+));
+
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE in_row_rebuild START 1 INCREMENT 1;));
+$node->safe_psql('postgres', q(SELECT nextval('in_row_rebuild');));
+
+
+$node->pgbench(
+'--no-vacuum --client=10 --jobs=4 --exit-on-abort --transactions=2500',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY',
+{
+ 'concurrent_ops' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \if :last_value = 2
+ REPACK (CONCURRENTLY) tbl1 USING INDEX tbl1_pkey;
+ \sleep 10 ms
+ REPACK (CONCURRENTLY) tbl2 USING INDEX tbl2_pkey;
+ \sleep 10 ms
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1, 100)
+ BEGIN;
+ UPDATE tbl1 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 4 WHERE i = :num;
+ \sleep 1 ms
+
+ UPDATE tbl2 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 4 WHERE i = :num;
+
+ COMMIT;
+ SELECT setval('in_row_rebuild', 1);
+
+ BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+ SELECT COALESCE(SUM(j), 0) AS t1 FROM tbl1 WHERE i = :num \gset p_
+ \sleep 10 ms
+ SELECT COALESCE(SUM(j), 0) AS t2 FROM tbl2 WHERE i = :num \gset p_
+ \if :p_t1 != :p_t2
+ COMMIT;
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ \sleep 10 ms
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ SELECT (:p_t1 + :p_t2) / 0;
+ \endif
+
+ COMMIT;
+ \endif
+ )
+});
+
+$node->stop;
+done_testing();
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
index 0f5c34af542..049ee75a4ba 100644
--- a/doc/src/sgml/mvcc.sgml
+++ b/doc/src/sgml/mvcc.sgml
@@ -1833,17 +1833,15 @@ SELECT pg_advisory_lock(q.id) FROM
<title>Caveats</title>
<para>
- Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the
- table-rewriting forms of <link linkend="sql-altertable"><command>ALTER
- TABLE</command></link> and <command>REPACK</command> with
- the <literal>CONCURRENTLY</literal> option, are not
+ Some DDL commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link> and the
+ table-rewriting forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link>, are not
MVCC-safe. This means that after the truncation or rewrite commits, the
table will appear empty to concurrent transactions, if they are using a
- snapshot taken before the command committed. This will only be an
+ snapshot taken before the DDL command committed. This will only be an
issue for a transaction that did not access the table in question
- before the command started — any transaction that has done so
+ before the DDL command started — any transaction that has done so
would hold at least an <literal>ACCESS SHARE</literal> table lock,
- which would block the truncating or rewriting command until that transaction completes.
+ which would block the DDL command until that transaction completes.
So these commands will not cause any apparent inconsistency in the
table contents for successive queries on the target table, but they
could cause visible inconsistency between the contents of the target
diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml
index 30c43c49069..9796a923597 100644
--- a/doc/src/sgml/ref/repack.sgml
+++ b/doc/src/sgml/ref/repack.sgml
@@ -308,15 +308,6 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING
</listitem>
</itemizedlist>
</para>
-
- <warning>
- <para>
- <command>REPACK</command> with the <literal>CONCURRENTLY</literal>
- option is not MVCC-safe, see <xref linkend="mvcc-caveats"/> for
- details.
- </para>
- </warning>
-
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 63b848473f8..91119da5cd5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -311,7 +311,8 @@ toast_save_datum(Relation rel, Datum value,
toasttup = heap_form_tuple(toasttupDesc, t_values, t_isnull);
- heap_insert(toastrel, toasttup, mycid, options, NULL);
+ heap_insert(toastrel, toasttup, GetCurrentTransactionId(), mycid,
+ options, NULL);
/*
* Create the index entry. We cheat a little here by not using
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e11833f01b4..94ca07e4b55 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2085,7 +2085,7 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
/*
* heap_insert - insert tuple into a heap
*
- * The new tuple is stamped with current transaction ID and the specified
+ * The new tuple is stamped with specified transaction ID and the specified
* command ID.
*
* See table_tuple_insert for comments about most of the input flags, except
@@ -2101,15 +2101,16 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
* reflected into *tup.
*/
void
-heap_insert(Relation relation, HeapTuple tup, CommandId cid,
- int options, BulkInsertState bistate)
+heap_insert(Relation relation, HeapTuple tup, TransactionId xid,
+ CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
bool all_visible_cleared = false;
+ Assert(TransactionIdIsValid(xid));
+
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(tup->t_data) <=
RelationGetNumberOfAttributes(relation));
@@ -2375,7 +2376,6 @@ void
heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple *heaptuples;
int i;
int ndone;
@@ -2408,7 +2408,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
tuple = ExecFetchSlotHeapTuple(slots[i], true, NULL);
slots[i]->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slots[i]->tts_tableOid;
- heaptuples[i] = heap_prepare_insert(relation, tuple, xid, cid,
+ heaptuples[i] = heap_prepare_insert(relation, tuple, GetCurrentTransactionId(), cid,
options);
}
@@ -2746,7 +2746,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
void
simple_heap_insert(Relation relation, HeapTuple tup)
{
- heap_insert(relation, tup, GetCurrentCommandId(true), 0, NULL);
+ heap_insert(relation, tup, GetCurrentTransactionId(),
+ GetCurrentCommandId(true), 0, NULL);
}
/*
@@ -2803,11 +2804,10 @@ xmax_infomask_changed(uint16 new_infomask, uint16 old_infomask)
*/
TM_Result
heap_delete(Relation relation, const ItemPointerData *tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, bool changingPart, bool walLogical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
ItemId lp;
HeapTupleData tp;
Page page;
@@ -2824,6 +2824,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
bool old_key_copied = false;
Assert(ItemPointerIsValid(tid));
+ Assert(TransactionIdIsValid(xid));
AssertHasSnapshotForToast(relation);
@@ -3240,7 +3241,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result result;
TM_FailureData tmfd;
- result = heap_delete(relation, tid,
+ result = heap_delete(relation, tid, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, false, /* changingPart */
@@ -3283,12 +3284,11 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
*/
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool walLogical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
Bitmapset *key_attrs;
@@ -3328,6 +3328,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask2_new_tuple;
Assert(ItemPointerIsValid(otid));
+ Assert(TransactionIdIsValid(xid));
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
@@ -4534,7 +4535,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
TM_FailureData tmfd;
LockTupleMode lockmode;
- result = heap_update(relation, otid, tup,
+ result = heap_update(relation, otid, tup, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode, update_indexes,
@@ -5373,8 +5374,6 @@ compute_new_xmax_infomask(TransactionId xmax, uint16 old_infomask,
uint16 new_infomask,
new_infomask2;
- Assert(TransactionIdIsCurrentTransactionId(add_to_xmax));
-
l5:
new_infomask = 0;
new_infomask2 = 0;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e6d630fa2f7..b49f9add5bb 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -252,7 +252,8 @@ heapam_tuple_insert(Relation relation, TupleTableSlot *slot, CommandId cid,
tuple->t_tableOid = slot->tts_tableOid;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -275,7 +276,8 @@ heapam_tuple_insert_speculative(Relation relation, TupleTableSlot *slot,
options |= HEAP_INSERT_SPECULATIVE;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -309,8 +311,8 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
* the storage itself is cleaning the dead tuples by itself, it is the
* time to call the index tuple deletion also.
*/
- return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart,
- true);
+ return heap_delete(relation, tid, GetCurrentTransactionId(), cid,
+ crosscheck, wait, tmfd, changingPart, true);
}
@@ -328,7 +330,8 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
+ result = heap_update(relation, otid, tuple, GetCurrentTransactionId(),
+ cid, crosscheck, wait,
tmfd, lockmode, update_indexes, true);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
@@ -2441,9 +2444,16 @@ reform_and_rewrite_tuple(HeapTuple tuple,
* flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps
* the relation files, it drops this relation, so no logical
* replication subscription should need the data.
+ *
+ * It is also crucial to stamp the new record with the exact same xid
+ * and cid, because the tuple must be visible to the snapshots of the
+ * concurrent transactions later.
*/
- heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true),
- HEAP_INSERT_NO_LOGICAL, NULL);
+ // TODO: looks like cid is not required
+ CommandId cid = HeapTupleHeaderGetRawCommandId(tuple->t_data);
+ TransactionId xid = HeapTupleHeaderGetXmin(tuple->t_data);
+
+ heap_insert(NewHeap, copiedTuple, xid, cid, HEAP_INSERT_NO_LOGICAL, NULL);
}
heap_freetuple(copiedTuple);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index f2a2ec6d3e5..1b1928ce300 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -58,6 +58,7 @@
#include "storage/ipc.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
+#include "storage/procarray.h"
#include "storage/procsignal.h"
#include "tcop/tcopprot.h"
#include "utils/acl.h"
@@ -249,15 +250,20 @@ static bool decode_concurrent_changes(LogicalDecodingContext *ctx,
DecodingWorkerShared *shared);
static void apply_concurrent_changes(BufFile *file, ChangeDest *dest);
static void apply_concurrent_insert(Relation rel, HeapTuple tup,
+ TransactionId xid,
IndexInsertState *iistate,
TupleTableSlot *index_slot);
static void apply_concurrent_update(Relation rel, HeapTuple tup,
HeapTuple tup_target,
+ TransactionId xid,
IndexInsertState *iistate,
TupleTableSlot *index_slot);
-static void apply_concurrent_delete(Relation rel, HeapTuple tup_target);
+static void apply_concurrent_delete(Relation rel,
+ TransactionId xid,
+ HeapTuple tup_target);
static HeapTuple find_target_tuple(Relation rel, ChangeDest *dest,
HeapTuple tup_key,
+ Snapshot snapshot,
TupleTableSlot *ident_slot);
static void process_concurrent_changes(XLogRecPtr end_of_wal,
ChangeDest *dest,
@@ -1091,7 +1097,14 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose, bool concurrent
/* The historic snapshot won't be needed anymore. */
if (snapshot)
+ {
+ TransactionId xmin = snapshot->xmin;
PopActiveSnapshot();
+ Assert(concurrent);
+ // TODO: seems like it not required: need to check SnapBuildInitialSnapshotForRepack
+ WaitForOlderSnapshots(xmin, false);
+ }
+
if (concurrent)
{
@@ -1382,30 +1395,35 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
* not to be aggressive about this.
*/
memset(¶ms, 0, sizeof(VacuumParams));
- vacuum_get_cutoffs(OldHeap, params, &cutoffs);
-
- /*
- * FreezeXid will become the table's new relfrozenxid, and that mustn't go
- * backwards, so take the max.
- */
+ if (!concurrent)
{
TransactionId relfrozenxid = OldHeap->rd_rel->relfrozenxid;
+ MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
+ vacuum_get_cutoffs(OldHeap, params, &cutoffs);
+ /*
+ * FreezeXid will become the table's new relfrozenxid, and that mustn't go
+ * backwards, so take the max.
+ */
if (TransactionIdIsValid(relfrozenxid) &&
TransactionIdPrecedes(cutoffs.FreezeLimit, relfrozenxid))
cutoffs.FreezeLimit = relfrozenxid;
- }
-
- /*
- * MultiXactCutoff, similarly, shouldn't go backwards either.
- */
- {
- MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
-
+ /*
+ * MultiXactCutoff, similarly, shouldn't go backwards either.
+ */
if (MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(cutoffs.MultiXactCutoff, relminmxid))
cutoffs.MultiXactCutoff = relminmxid;
}
+ else
+ {
+ /*
+ * In concurrent mode we reuse all the xmin/xmax,
+ * so just use current values for simplicity.
+ */
+ cutoffs.FreezeLimit = OldHeap->rd_rel->relfrozenxid;
+ cutoffs.MultiXactCutoff = OldHeap->rd_rel->relminmxid;
+ }
/*
* Decide whether to use an indexscan or seqscan-and-optional-sort to scan
@@ -2745,6 +2763,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
size_t nread;
HeapTuple tup,
tup_exist;
+ TransactionId xid;
CHECK_FOR_INTERRUPTS();
@@ -2761,6 +2780,17 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
tup->t_len = t_len;
ItemPointerSetInvalid(&tup->t_self);
tup->t_tableOid = RelationGetRelid(dest->rel);
+ BufFileReadExact(file, &xid, sizeof(TransactionId));
+
+ if (TransactionIdIsValid(xid && TransactionIdIsInProgress(xid)))
+ {
+ /* xmin is committed for sure because we got that update from reorderbuffer.
+ * but there is a possibility procarray is not yet updated and current backend still see it as
+ * in-progress. Let's wait for procarray to be updated. */
+ XactLockTableWait(xid, NULL, NULL, XLTW_None);
+ Assert(!TransactionIdIsInProgress(xid));
+ Assert(TransactionIdDidCommit(xid));
+ }
if (kind == CHANGE_UPDATE_OLD)
{
@@ -2771,7 +2801,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
{
Assert(tup_old == NULL);
- apply_concurrent_insert(rel, tup, dest->iistate, index_slot);
+ apply_concurrent_insert(rel, tup, xid, dest->iistate, index_slot);
pfree(tup);
}
@@ -2790,17 +2820,21 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
}
/*
- * Find the tuple to be updated or deleted.
+ * Find the tuple to be updated or deleted using SnapshotSelf.
+ * That way we receive the last alive version in case of HOT chain.
+ * It is guaranteed there is no any non-yet committed, but updated version
+ * because we here replaying all-committed transactions without any concurrency
+ * involved.
*/
- tup_exist = find_target_tuple(rel, dest, tup_key, ident_slot);
+ tup_exist = find_target_tuple(rel, dest, tup_key, SnapshotSelf, ident_slot);
if (tup_exist == NULL)
elog(ERROR, "failed to find target tuple");
if (kind == CHANGE_UPDATE_NEW)
- apply_concurrent_update(rel, tup, tup_exist, dest->iistate,
+ apply_concurrent_update(rel, tup, tup_exist, xid, dest->iistate,
index_slot);
else
- apply_concurrent_delete(rel, tup_exist);
+ apply_concurrent_delete(rel, xid, tup_exist);
if (tup_old != NULL)
{
@@ -2819,6 +2853,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
*/
if (kind != CHANGE_UPDATE_OLD)
{
+ // TODO: not sure it is required at all: we are replaying committed transactions stamping them with committed XID
CommandCounterIncrement();
UpdateActiveSnapshotCommandId();
}
@@ -2830,7 +2865,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
}
static void
-apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
+apply_concurrent_insert(Relation rel, HeapTuple tup, TransactionId xid, IndexInsertState *iistate,
TupleTableSlot *index_slot)
{
List *recheck;
@@ -2840,9 +2875,12 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
* Like simple_heap_insert(), but make sure that the INSERT is not
* logically decoded - see reform_and_rewrite_tuple() for more
* information.
+ *
+ * Use already committed xid to stamp the tuple.
*/
- heap_insert(rel, tup, GetCurrentCommandId(true), HEAP_INSERT_NO_LOGICAL,
- NULL);
+ Assert(TransactionIdIsValid(xid));
+ heap_insert(rel, tup, xid, GetCurrentCommandId(true),
+ HEAP_INSERT_NO_LOGICAL, NULL);
/*
* Update indexes.
@@ -2850,6 +2888,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
* In case functions in the index need the active snapshot and caller
* hasn't set one.
*/
+ PushActiveSnapshot(GetLatestSnapshot());
ExecStoreHeapTuple(tup, index_slot, false);
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
@@ -2860,6 +2899,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
NIL, /* arbiterIndexes */
false /* onlySummarizing */
);
+ PopActiveSnapshot();
/*
* If recheck is required, it must have been preformed on the source
@@ -2873,6 +2913,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
static void
apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
+ TransactionId xid,
IndexInsertState *iistate, TupleTableSlot *index_slot)
{
LockTupleMode lockmode;
@@ -2887,9 +2928,12 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
*
* Do it like in simple_heap_update(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
+ Assert(TransactionIdIsValid(xid));
res = heap_update(rel, &tup_target->t_self, tup,
- GetCurrentCommandId(true),
+ xid, GetCurrentCommandId(true),
InvalidSnapshot,
false, /* no wait - only we are doing changes */
&tmfd, &lockmode, &update_indexes,
@@ -2901,6 +2945,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
if (update_indexes != TU_None)
{
+ PushActiveSnapshot(GetLatestSnapshot());
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
iistate->estate,
@@ -2910,6 +2955,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
NIL, /* arbiterIndexes */
/* onlySummarizing */
update_indexes == TU_Summarizing);
+ PopActiveSnapshot();
list_free(recheck);
}
@@ -2917,7 +2963,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
}
static void
-apply_concurrent_delete(Relation rel, HeapTuple tup_target)
+apply_concurrent_delete(Relation rel, TransactionId xid, HeapTuple tup_target)
{
TM_Result res;
TM_FailureData tmfd;
@@ -2927,9 +2973,12 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target)
*
* Do it like in simple_heap_delete(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
- res = heap_delete(rel, &tup_target->t_self, GetCurrentCommandId(true),
- InvalidSnapshot, false,
+ Assert(TransactionIdIsValid(xid));
+ res = heap_delete(rel, &tup_target->t_self, xid,
+ GetCurrentCommandId(true), InvalidSnapshot, false,
&tmfd,
false, /* no wait - only we are doing changes */
false /* wal_logical */ );
@@ -2950,7 +2999,7 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target)
*/
static HeapTuple
find_target_tuple(Relation rel, ChangeDest *dest, HeapTuple tup_key,
- TupleTableSlot *ident_slot)
+ Snapshot snapshot, TupleTableSlot *ident_slot)
{
Relation ident_index = dest->ident_index;
IndexScanDesc scan;
@@ -2959,7 +3008,7 @@ find_target_tuple(Relation rel, ChangeDest *dest, HeapTuple tup_key,
HeapTuple result = NULL;
/* XXX no instrumentation for now */
- scan = index_beginscan(rel, ident_index, GetActiveSnapshot(),
+ scan = index_beginscan(rel, ident_index, snapshot,
NULL, dest->ident_key_nentries, 0);
/*
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
index fb9956d392d..8d796e0a684 100644
--- a/src/backend/replication/pgoutput_repack/pgoutput_repack.c
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -29,7 +29,8 @@ static void plugin_commit_txn(LogicalDecodingContext *ctx,
static void plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Relation rel, ReorderBufferChange *change);
static void store_change(LogicalDecodingContext *ctx,
- ConcurrentChangeKind kind, HeapTuple tuple);
+ ConcurrentChangeKind kind, HeapTuple tuple,
+ TransactionId xid);
void
_PG_output_plugin_init(OutputPluginCallbacks *cb)
@@ -120,7 +121,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (newtuple == NULL)
elog(ERROR, "Incomplete insert info.");
- store_change(ctx, CHANGE_INSERT, newtuple);
+ store_change(ctx, CHANGE_INSERT, newtuple, change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -137,9 +138,11 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
elog(ERROR, "Incomplete update info.");
if (oldtuple != NULL)
- store_change(ctx, CHANGE_UPDATE_OLD, oldtuple);
+ store_change(ctx, CHANGE_UPDATE_OLD, oldtuple,
+ change->txn->xid);
- store_change(ctx, CHANGE_UPDATE_NEW, newtuple);
+ store_change(ctx, CHANGE_UPDATE_NEW, newtuple,
+ change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_DELETE:
@@ -152,7 +155,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (oldtuple == NULL)
elog(ERROR, "Incomplete delete info.");
- store_change(ctx, CHANGE_DELETE, oldtuple);
+ store_change(ctx, CHANGE_DELETE, oldtuple, change->txn->xid);
}
break;
default:
@@ -165,7 +168,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
/* Store concurrent data change. */
static void
store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
- HeapTuple tuple)
+ HeapTuple tuple, TransactionId xid)
{
RepackDecodingState *dstate;
char kind_byte = (char) kind;
@@ -195,6 +198,7 @@ store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
BufFileWrite(dstate->file, &tuple->t_len, sizeof(tuple->t_len));
/* ... and the tuple itself. */
BufFileWrite(dstate->file, tuple->t_data, tuple->t_len);
+ BufFileWrite(dstate->file, &xid, sizeof(TransactionId));
/* Free the flat copy if created above. */
if (flattened)
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index b7cd25896f6..d9776f61a0d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -354,20 +354,20 @@ extern BulkInsertState GetBulkInsertState(void);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
-extern void heap_insert(Relation relation, HeapTuple tup, CommandId cid,
+extern void heap_insert(Relation relation, HeapTuple tup, TransactionId xid, CommandId cid,
int options, BulkInsertState bistate);
extern void heap_multi_insert(Relation relation, TupleTableSlot **slots,
int ntuples, CommandId cid, int options,
BulkInsertState bistate);
extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, bool changingPart,
bool wal_logical);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool wal_logical);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index d727a9b056b..accd42d78aa 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -85,9 +85,6 @@ step change_new
# When applying concurrent data changes, we should see the effects of an
# in-progress subtransaction.
#
-# XXX Not sure this test is useful now - it was designed for the patch that
-# preserves tuple visibility and which therefore modifies
-# TransactionIdIsCurrentTransactionId().
step change_subxact1
{
BEGIN;
@@ -102,7 +99,6 @@ step change_subxact1
# When applying concurrent data changes, we should not see the effects of a
# rolled back subtransaction.
#
-# XXX Is this test useful? See above.
step change_subxact2
{
BEGIN;
--
2.43.0
Attachments:
[text/plain] nocfbot-0006-Preserve-visibility-information-of-the-conc.patch (31.8K, ../../CADzfLwUitd5J17O9FUxNGrZBurOpL6n+tnS6dgArXi-i9DNxhg@mail.gmail.com/2-nocfbot-0006-Preserve-visibility-information-of-the-conc.patch)
download | inline diff:
From 457235c743a2dec2c1917fbdfa7f5a48d305c63e Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sat, 13 Dec 2025 19:42:52 +0100
Subject: [PATCH vnocfbot] Preserve visibility information of the concurrent
data changes.
As explained in the commit message of the preceding patch of the series, the
data changes done by applications while REPACK CONCURRENTLY is copying the
table contents to a new file are decoded from WAL and eventually also applied
to the new file. To reduce the complexity a little bit, the preceding patch
uses the current transaction (i.e. transaction opened by the REPACK command)
to execute those INSERT, UPDATE and DELETE commands.
However, REPACK is not expected to change visibility of tuples. Therefore,
this patch fixes the handling of the "concurrent data changes". It ensures
that tuples written into the new table have the same XID and command ID (CID)
as they had in the old table.
To "replay" an UPDATE or DELETE command on the new table, we use SnapshotSelf to find the last alive version of tuple and update with stamp with xid of original transaction. It is safe because:
* all transactions we replaying are committed
* apply worker working without any concurrent modifiers of the table
As long as we preserve the tuple visibility information (which includes XID),
it's important to avoid logical decoding of the WAL generated by DMLs on the
new table: the logical decoding subsystem probably does not expect that the
incoming WAL records contain XIDs of an already decoded transactions. (And of
course, repeated decoding would be wasted effort.)
Author: Antonin Houska <ah@cybertec.at> with changes from Mikhail Nikalayeu <mihailnikalayeu@gmail.com
---
contrib/amcheck/meson.build | 1 +
.../amcheck/t/009_repack_concurrently_mvcc.pl | 113 ++++++++++++++++++
doc/src/sgml/mvcc.sgml | 12 +-
doc/src/sgml/ref/repack.sgml | 9 --
src/backend/access/common/toast_internals.c | 3 +-
src/backend/access/heap/heapam.c | 29 +++--
src/backend/access/heap/heapam_handler.c | 24 ++--
src/backend/commands/cluster.c | 107 ++++++++++++-----
.../pgoutput_repack/pgoutput_repack.c | 16 ++-
src/include/access/heapam.h | 6 +-
.../injection_points/specs/repack.spec | 4 -
11 files changed, 243 insertions(+), 81 deletions(-)
create mode 100644 contrib/amcheck/t/009_repack_concurrently_mvcc.pl
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index f7c70735989..6946c684259 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -52,6 +52,7 @@ tests += {
't/006_verify_gin.pl',
't/007_repack_concurrently.pl',
't/008_repack_concurrently.pl',
+ 't/009_repack_concurrently_mvcc.pl',
],
},
}
diff --git a/contrib/amcheck/t/009_repack_concurrently_mvcc.pl b/contrib/amcheck/t/009_repack_concurrently_mvcc.pl
new file mode 100644
index 00000000000..a83fd5b8141
--- /dev/null
+++ b/contrib/amcheck/t/009_repack_concurrently_mvcc.pl
@@ -0,0 +1,113 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+));
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl1(i int PRIMARY KEY, j int)));
+$node->safe_psql('postgres', q(CREATE TABLE tbl2(i int PRIMARY KEY, j int)));
+
+
+# Insert 100 rows into tbl1
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl1 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+# Insert 100 rows into tbl2
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl2 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+
+# Insert 100 rows into tbl1
+$node->safe_psql('postgres', q(
+ CREATE OR REPLACE FUNCTION log_raise(i int, j1 int, j2 int) RETURNS VOID AS $$
+ BEGIN
+ RAISE NOTICE 'ERROR i=% j1=% j2=%', i, j1, j2;
+ END;$$ LANGUAGE plpgsql;
+));
+
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE in_row_rebuild START 1 INCREMENT 1;));
+$node->safe_psql('postgres', q(SELECT nextval('in_row_rebuild');));
+
+
+$node->pgbench(
+'--no-vacuum --client=10 --jobs=4 --exit-on-abort --transactions=2500',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY',
+{
+ 'concurrent_ops' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \if :last_value = 2
+ REPACK (CONCURRENTLY) tbl1 USING INDEX tbl1_pkey;
+ \sleep 10 ms
+ REPACK (CONCURRENTLY) tbl2 USING INDEX tbl2_pkey;
+ \sleep 10 ms
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1, 100)
+ BEGIN;
+ UPDATE tbl1 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 4 WHERE i = :num;
+ \sleep 1 ms
+
+ UPDATE tbl2 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 4 WHERE i = :num;
+
+ COMMIT;
+ SELECT setval('in_row_rebuild', 1);
+
+ BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+ SELECT COALESCE(SUM(j), 0) AS t1 FROM tbl1 WHERE i = :num \gset p_
+ \sleep 10 ms
+ SELECT COALESCE(SUM(j), 0) AS t2 FROM tbl2 WHERE i = :num \gset p_
+ \if :p_t1 != :p_t2
+ COMMIT;
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ \sleep 10 ms
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ SELECT (:p_t1 + :p_t2) / 0;
+ \endif
+
+ COMMIT;
+ \endif
+ )
+});
+
+$node->stop;
+done_testing();
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
index 0f5c34af542..049ee75a4ba 100644
--- a/doc/src/sgml/mvcc.sgml
+++ b/doc/src/sgml/mvcc.sgml
@@ -1833,17 +1833,15 @@ SELECT pg_advisory_lock(q.id) FROM
<title>Caveats</title>
<para>
- Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the
- table-rewriting forms of <link linkend="sql-altertable"><command>ALTER
- TABLE</command></link> and <command>REPACK</command> with
- the <literal>CONCURRENTLY</literal> option, are not
+ Some DDL commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link> and the
+ table-rewriting forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link>, are not
MVCC-safe. This means that after the truncation or rewrite commits, the
table will appear empty to concurrent transactions, if they are using a
- snapshot taken before the command committed. This will only be an
+ snapshot taken before the DDL command committed. This will only be an
issue for a transaction that did not access the table in question
- before the command started — any transaction that has done so
+ before the DDL command started — any transaction that has done so
would hold at least an <literal>ACCESS SHARE</literal> table lock,
- which would block the truncating or rewriting command until that transaction completes.
+ which would block the DDL command until that transaction completes.
So these commands will not cause any apparent inconsistency in the
table contents for successive queries on the target table, but they
could cause visible inconsistency between the contents of the target
diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml
index 30c43c49069..9796a923597 100644
--- a/doc/src/sgml/ref/repack.sgml
+++ b/doc/src/sgml/ref/repack.sgml
@@ -308,15 +308,6 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING
</listitem>
</itemizedlist>
</para>
-
- <warning>
- <para>
- <command>REPACK</command> with the <literal>CONCURRENTLY</literal>
- option is not MVCC-safe, see <xref linkend="mvcc-caveats"/> for
- details.
- </para>
- </warning>
-
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 63b848473f8..91119da5cd5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -311,7 +311,8 @@ toast_save_datum(Relation rel, Datum value,
toasttup = heap_form_tuple(toasttupDesc, t_values, t_isnull);
- heap_insert(toastrel, toasttup, mycid, options, NULL);
+ heap_insert(toastrel, toasttup, GetCurrentTransactionId(), mycid,
+ options, NULL);
/*
* Create the index entry. We cheat a little here by not using
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e11833f01b4..94ca07e4b55 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2085,7 +2085,7 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
/*
* heap_insert - insert tuple into a heap
*
- * The new tuple is stamped with current transaction ID and the specified
+ * The new tuple is stamped with specified transaction ID and the specified
* command ID.
*
* See table_tuple_insert for comments about most of the input flags, except
@@ -2101,15 +2101,16 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
* reflected into *tup.
*/
void
-heap_insert(Relation relation, HeapTuple tup, CommandId cid,
- int options, BulkInsertState bistate)
+heap_insert(Relation relation, HeapTuple tup, TransactionId xid,
+ CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
bool all_visible_cleared = false;
+ Assert(TransactionIdIsValid(xid));
+
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(tup->t_data) <=
RelationGetNumberOfAttributes(relation));
@@ -2375,7 +2376,6 @@ void
heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple *heaptuples;
int i;
int ndone;
@@ -2408,7 +2408,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
tuple = ExecFetchSlotHeapTuple(slots[i], true, NULL);
slots[i]->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slots[i]->tts_tableOid;
- heaptuples[i] = heap_prepare_insert(relation, tuple, xid, cid,
+ heaptuples[i] = heap_prepare_insert(relation, tuple, GetCurrentTransactionId(), cid,
options);
}
@@ -2746,7 +2746,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
void
simple_heap_insert(Relation relation, HeapTuple tup)
{
- heap_insert(relation, tup, GetCurrentCommandId(true), 0, NULL);
+ heap_insert(relation, tup, GetCurrentTransactionId(),
+ GetCurrentCommandId(true), 0, NULL);
}
/*
@@ -2803,11 +2804,10 @@ xmax_infomask_changed(uint16 new_infomask, uint16 old_infomask)
*/
TM_Result
heap_delete(Relation relation, const ItemPointerData *tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, bool changingPart, bool walLogical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
ItemId lp;
HeapTupleData tp;
Page page;
@@ -2824,6 +2824,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
bool old_key_copied = false;
Assert(ItemPointerIsValid(tid));
+ Assert(TransactionIdIsValid(xid));
AssertHasSnapshotForToast(relation);
@@ -3240,7 +3241,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result result;
TM_FailureData tmfd;
- result = heap_delete(relation, tid,
+ result = heap_delete(relation, tid, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, false, /* changingPart */
@@ -3283,12 +3284,11 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
*/
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool walLogical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
Bitmapset *key_attrs;
@@ -3328,6 +3328,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask2_new_tuple;
Assert(ItemPointerIsValid(otid));
+ Assert(TransactionIdIsValid(xid));
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
@@ -4534,7 +4535,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
TM_FailureData tmfd;
LockTupleMode lockmode;
- result = heap_update(relation, otid, tup,
+ result = heap_update(relation, otid, tup, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode, update_indexes,
@@ -5373,8 +5374,6 @@ compute_new_xmax_infomask(TransactionId xmax, uint16 old_infomask,
uint16 new_infomask,
new_infomask2;
- Assert(TransactionIdIsCurrentTransactionId(add_to_xmax));
-
l5:
new_infomask = 0;
new_infomask2 = 0;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e6d630fa2f7..b49f9add5bb 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -252,7 +252,8 @@ heapam_tuple_insert(Relation relation, TupleTableSlot *slot, CommandId cid,
tuple->t_tableOid = slot->tts_tableOid;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -275,7 +276,8 @@ heapam_tuple_insert_speculative(Relation relation, TupleTableSlot *slot,
options |= HEAP_INSERT_SPECULATIVE;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -309,8 +311,8 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
* the storage itself is cleaning the dead tuples by itself, it is the
* time to call the index tuple deletion also.
*/
- return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart,
- true);
+ return heap_delete(relation, tid, GetCurrentTransactionId(), cid,
+ crosscheck, wait, tmfd, changingPart, true);
}
@@ -328,7 +330,8 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
+ result = heap_update(relation, otid, tuple, GetCurrentTransactionId(),
+ cid, crosscheck, wait,
tmfd, lockmode, update_indexes, true);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
@@ -2441,9 +2444,16 @@ reform_and_rewrite_tuple(HeapTuple tuple,
* flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps
* the relation files, it drops this relation, so no logical
* replication subscription should need the data.
+ *
+ * It is also crucial to stamp the new record with the exact same xid
+ * and cid, because the tuple must be visible to the snapshots of the
+ * concurrent transactions later.
*/
- heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true),
- HEAP_INSERT_NO_LOGICAL, NULL);
+ // TODO: looks like cid is not required
+ CommandId cid = HeapTupleHeaderGetRawCommandId(tuple->t_data);
+ TransactionId xid = HeapTupleHeaderGetXmin(tuple->t_data);
+
+ heap_insert(NewHeap, copiedTuple, xid, cid, HEAP_INSERT_NO_LOGICAL, NULL);
}
heap_freetuple(copiedTuple);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index f2a2ec6d3e5..1b1928ce300 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -58,6 +58,7 @@
#include "storage/ipc.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
+#include "storage/procarray.h"
#include "storage/procsignal.h"
#include "tcop/tcopprot.h"
#include "utils/acl.h"
@@ -249,15 +250,20 @@ static bool decode_concurrent_changes(LogicalDecodingContext *ctx,
DecodingWorkerShared *shared);
static void apply_concurrent_changes(BufFile *file, ChangeDest *dest);
static void apply_concurrent_insert(Relation rel, HeapTuple tup,
+ TransactionId xid,
IndexInsertState *iistate,
TupleTableSlot *index_slot);
static void apply_concurrent_update(Relation rel, HeapTuple tup,
HeapTuple tup_target,
+ TransactionId xid,
IndexInsertState *iistate,
TupleTableSlot *index_slot);
-static void apply_concurrent_delete(Relation rel, HeapTuple tup_target);
+static void apply_concurrent_delete(Relation rel,
+ TransactionId xid,
+ HeapTuple tup_target);
static HeapTuple find_target_tuple(Relation rel, ChangeDest *dest,
HeapTuple tup_key,
+ Snapshot snapshot,
TupleTableSlot *ident_slot);
static void process_concurrent_changes(XLogRecPtr end_of_wal,
ChangeDest *dest,
@@ -1091,7 +1097,14 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose, bool concurrent
/* The historic snapshot won't be needed anymore. */
if (snapshot)
+ {
+ TransactionId xmin = snapshot->xmin;
PopActiveSnapshot();
+ Assert(concurrent);
+ // TODO: seems like it not required: need to check SnapBuildInitialSnapshotForRepack
+ WaitForOlderSnapshots(xmin, false);
+ }
+
if (concurrent)
{
@@ -1382,30 +1395,35 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
* not to be aggressive about this.
*/
memset(¶ms, 0, sizeof(VacuumParams));
- vacuum_get_cutoffs(OldHeap, params, &cutoffs);
-
- /*
- * FreezeXid will become the table's new relfrozenxid, and that mustn't go
- * backwards, so take the max.
- */
+ if (!concurrent)
{
TransactionId relfrozenxid = OldHeap->rd_rel->relfrozenxid;
+ MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
+ vacuum_get_cutoffs(OldHeap, params, &cutoffs);
+ /*
+ * FreezeXid will become the table's new relfrozenxid, and that mustn't go
+ * backwards, so take the max.
+ */
if (TransactionIdIsValid(relfrozenxid) &&
TransactionIdPrecedes(cutoffs.FreezeLimit, relfrozenxid))
cutoffs.FreezeLimit = relfrozenxid;
- }
-
- /*
- * MultiXactCutoff, similarly, shouldn't go backwards either.
- */
- {
- MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
-
+ /*
+ * MultiXactCutoff, similarly, shouldn't go backwards either.
+ */
if (MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(cutoffs.MultiXactCutoff, relminmxid))
cutoffs.MultiXactCutoff = relminmxid;
}
+ else
+ {
+ /*
+ * In concurrent mode we reuse all the xmin/xmax,
+ * so just use current values for simplicity.
+ */
+ cutoffs.FreezeLimit = OldHeap->rd_rel->relfrozenxid;
+ cutoffs.MultiXactCutoff = OldHeap->rd_rel->relminmxid;
+ }
/*
* Decide whether to use an indexscan or seqscan-and-optional-sort to scan
@@ -2745,6 +2763,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
size_t nread;
HeapTuple tup,
tup_exist;
+ TransactionId xid;
CHECK_FOR_INTERRUPTS();
@@ -2761,6 +2780,17 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
tup->t_len = t_len;
ItemPointerSetInvalid(&tup->t_self);
tup->t_tableOid = RelationGetRelid(dest->rel);
+ BufFileReadExact(file, &xid, sizeof(TransactionId));
+
+ if (TransactionIdIsValid(xid && TransactionIdIsInProgress(xid)))
+ {
+ /* xmin is committed for sure because we got that update from reorderbuffer.
+ * but there is a possibility procarray is not yet updated and current backend still see it as
+ * in-progress. Let's wait for procarray to be updated. */
+ XactLockTableWait(xid, NULL, NULL, XLTW_None);
+ Assert(!TransactionIdIsInProgress(xid));
+ Assert(TransactionIdDidCommit(xid));
+ }
if (kind == CHANGE_UPDATE_OLD)
{
@@ -2771,7 +2801,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
{
Assert(tup_old == NULL);
- apply_concurrent_insert(rel, tup, dest->iistate, index_slot);
+ apply_concurrent_insert(rel, tup, xid, dest->iistate, index_slot);
pfree(tup);
}
@@ -2790,17 +2820,21 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
}
/*
- * Find the tuple to be updated or deleted.
+ * Find the tuple to be updated or deleted using SnapshotSelf.
+ * That way we receive the last alive version in case of HOT chain.
+ * It is guaranteed there is no any non-yet committed, but updated version
+ * because we here replaying all-committed transactions without any concurrency
+ * involved.
*/
- tup_exist = find_target_tuple(rel, dest, tup_key, ident_slot);
+ tup_exist = find_target_tuple(rel, dest, tup_key, SnapshotSelf, ident_slot);
if (tup_exist == NULL)
elog(ERROR, "failed to find target tuple");
if (kind == CHANGE_UPDATE_NEW)
- apply_concurrent_update(rel, tup, tup_exist, dest->iistate,
+ apply_concurrent_update(rel, tup, tup_exist, xid, dest->iistate,
index_slot);
else
- apply_concurrent_delete(rel, tup_exist);
+ apply_concurrent_delete(rel, xid, tup_exist);
if (tup_old != NULL)
{
@@ -2819,6 +2853,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
*/
if (kind != CHANGE_UPDATE_OLD)
{
+ // TODO: not sure it is required at all: we are replaying committed transactions stamping them with committed XID
CommandCounterIncrement();
UpdateActiveSnapshotCommandId();
}
@@ -2830,7 +2865,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
}
static void
-apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
+apply_concurrent_insert(Relation rel, HeapTuple tup, TransactionId xid, IndexInsertState *iistate,
TupleTableSlot *index_slot)
{
List *recheck;
@@ -2840,9 +2875,12 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
* Like simple_heap_insert(), but make sure that the INSERT is not
* logically decoded - see reform_and_rewrite_tuple() for more
* information.
+ *
+ * Use already committed xid to stamp the tuple.
*/
- heap_insert(rel, tup, GetCurrentCommandId(true), HEAP_INSERT_NO_LOGICAL,
- NULL);
+ Assert(TransactionIdIsValid(xid));
+ heap_insert(rel, tup, xid, GetCurrentCommandId(true),
+ HEAP_INSERT_NO_LOGICAL, NULL);
/*
* Update indexes.
@@ -2850,6 +2888,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
* In case functions in the index need the active snapshot and caller
* hasn't set one.
*/
+ PushActiveSnapshot(GetLatestSnapshot());
ExecStoreHeapTuple(tup, index_slot, false);
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
@@ -2860,6 +2899,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
NIL, /* arbiterIndexes */
false /* onlySummarizing */
);
+ PopActiveSnapshot();
/*
* If recheck is required, it must have been preformed on the source
@@ -2873,6 +2913,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
static void
apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
+ TransactionId xid,
IndexInsertState *iistate, TupleTableSlot *index_slot)
{
LockTupleMode lockmode;
@@ -2887,9 +2928,12 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
*
* Do it like in simple_heap_update(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
+ Assert(TransactionIdIsValid(xid));
res = heap_update(rel, &tup_target->t_self, tup,
- GetCurrentCommandId(true),
+ xid, GetCurrentCommandId(true),
InvalidSnapshot,
false, /* no wait - only we are doing changes */
&tmfd, &lockmode, &update_indexes,
@@ -2901,6 +2945,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
if (update_indexes != TU_None)
{
+ PushActiveSnapshot(GetLatestSnapshot());
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
iistate->estate,
@@ -2910,6 +2955,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
NIL, /* arbiterIndexes */
/* onlySummarizing */
update_indexes == TU_Summarizing);
+ PopActiveSnapshot();
list_free(recheck);
}
@@ -2917,7 +2963,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
}
static void
-apply_concurrent_delete(Relation rel, HeapTuple tup_target)
+apply_concurrent_delete(Relation rel, TransactionId xid, HeapTuple tup_target)
{
TM_Result res;
TM_FailureData tmfd;
@@ -2927,9 +2973,12 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target)
*
* Do it like in simple_heap_delete(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
- res = heap_delete(rel, &tup_target->t_self, GetCurrentCommandId(true),
- InvalidSnapshot, false,
+ Assert(TransactionIdIsValid(xid));
+ res = heap_delete(rel, &tup_target->t_self, xid,
+ GetCurrentCommandId(true), InvalidSnapshot, false,
&tmfd,
false, /* no wait - only we are doing changes */
false /* wal_logical */ );
@@ -2950,7 +2999,7 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target)
*/
static HeapTuple
find_target_tuple(Relation rel, ChangeDest *dest, HeapTuple tup_key,
- TupleTableSlot *ident_slot)
+ Snapshot snapshot, TupleTableSlot *ident_slot)
{
Relation ident_index = dest->ident_index;
IndexScanDesc scan;
@@ -2959,7 +3008,7 @@ find_target_tuple(Relation rel, ChangeDest *dest, HeapTuple tup_key,
HeapTuple result = NULL;
/* XXX no instrumentation for now */
- scan = index_beginscan(rel, ident_index, GetActiveSnapshot(),
+ scan = index_beginscan(rel, ident_index, snapshot,
NULL, dest->ident_key_nentries, 0);
/*
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
index fb9956d392d..8d796e0a684 100644
--- a/src/backend/replication/pgoutput_repack/pgoutput_repack.c
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -29,7 +29,8 @@ static void plugin_commit_txn(LogicalDecodingContext *ctx,
static void plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Relation rel, ReorderBufferChange *change);
static void store_change(LogicalDecodingContext *ctx,
- ConcurrentChangeKind kind, HeapTuple tuple);
+ ConcurrentChangeKind kind, HeapTuple tuple,
+ TransactionId xid);
void
_PG_output_plugin_init(OutputPluginCallbacks *cb)
@@ -120,7 +121,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (newtuple == NULL)
elog(ERROR, "Incomplete insert info.");
- store_change(ctx, CHANGE_INSERT, newtuple);
+ store_change(ctx, CHANGE_INSERT, newtuple, change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -137,9 +138,11 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
elog(ERROR, "Incomplete update info.");
if (oldtuple != NULL)
- store_change(ctx, CHANGE_UPDATE_OLD, oldtuple);
+ store_change(ctx, CHANGE_UPDATE_OLD, oldtuple,
+ change->txn->xid);
- store_change(ctx, CHANGE_UPDATE_NEW, newtuple);
+ store_change(ctx, CHANGE_UPDATE_NEW, newtuple,
+ change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_DELETE:
@@ -152,7 +155,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (oldtuple == NULL)
elog(ERROR, "Incomplete delete info.");
- store_change(ctx, CHANGE_DELETE, oldtuple);
+ store_change(ctx, CHANGE_DELETE, oldtuple, change->txn->xid);
}
break;
default:
@@ -165,7 +168,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
/* Store concurrent data change. */
static void
store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
- HeapTuple tuple)
+ HeapTuple tuple, TransactionId xid)
{
RepackDecodingState *dstate;
char kind_byte = (char) kind;
@@ -195,6 +198,7 @@ store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
BufFileWrite(dstate->file, &tuple->t_len, sizeof(tuple->t_len));
/* ... and the tuple itself. */
BufFileWrite(dstate->file, tuple->t_data, tuple->t_len);
+ BufFileWrite(dstate->file, &xid, sizeof(TransactionId));
/* Free the flat copy if created above. */
if (flattened)
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index b7cd25896f6..d9776f61a0d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -354,20 +354,20 @@ extern BulkInsertState GetBulkInsertState(void);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
-extern void heap_insert(Relation relation, HeapTuple tup, CommandId cid,
+extern void heap_insert(Relation relation, HeapTuple tup, TransactionId xid, CommandId cid,
int options, BulkInsertState bistate);
extern void heap_multi_insert(Relation relation, TupleTableSlot **slots,
int ntuples, CommandId cid, int options,
BulkInsertState bistate);
extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, bool changingPart,
bool wal_logical);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool wal_logical);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index d727a9b056b..accd42d78aa 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -85,9 +85,6 @@ step change_new
# When applying concurrent data changes, we should see the effects of an
# in-progress subtransaction.
#
-# XXX Not sure this test is useful now - it was designed for the patch that
-# preserves tuple visibility and which therefore modifies
-# TransactionIdIsCurrentTransactionId().
step change_subxact1
{
BEGIN;
@@ -102,7 +99,6 @@ step change_subxact1
# When applying concurrent data changes, we should not see the effects of a
# rolled back subtransaction.
#
-# XXX Is this test useful? See above.
step change_subxact2
{
BEGIN;
--
2.43.0
[application/x-patch] nocfbot-0002-one-more-stress-test-for-repack-concurrentl.patch (3.5K, ../../CADzfLwUitd5J17O9FUxNGrZBurOpL6n+tnS6dgArXi-i9DNxhg@mail.gmail.com/3-nocfbot-0002-one-more-stress-test-for-repack-concurrentl.patch)
download | inline diff:
From 25fc848068b28e5b2ae099bdecae35fdf8cb6240 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sat, 13 Dec 2025 18:46:46 +0100
Subject: [PATCH vnocfbot 2/2] one more stress test for repack concurrently
---
contrib/amcheck/meson.build | 1 +
contrib/amcheck/t/008_repack_concurrently.pl | 101 +++++++++++++++++++
2 files changed, 102 insertions(+)
create mode 100644 contrib/amcheck/t/008_repack_concurrently.pl
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index 2b69081d3bf..f7c70735989 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -51,6 +51,7 @@ tests += {
't/005_pitr.pl',
't/006_verify_gin.pl',
't/007_repack_concurrently.pl',
+ 't/008_repack_concurrently.pl',
],
},
}
diff --git a/contrib/amcheck/t/008_repack_concurrently.pl b/contrib/amcheck/t/008_repack_concurrently.pl
new file mode 100644
index 00000000000..220524d41b3
--- /dev/null
+++ b/contrib/amcheck/t/008_repack_concurrently.pl
@@ -0,0 +1,101 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+));
+
+my $no_hot = int(rand(2));
+
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl(i SERIAL PRIMARY KEY, j int)));
+if ($no_hot)
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(j);));
+}
+else
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(i);));
+}
+
+# Load amcheck
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+
+my $sum = $node->safe_psql('postgres', q(
+ SELECT SUM(j) AS sum FROM tbl
+));
+
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE last_j START 1 INCREMENT 1;));
+
+
+$node->pgbench(
+'--no-vacuum --client=30 --jobs=4 --exit-on-abort --transactions=1000',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY',
+{
+ 'concurrent_ops' => qq(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \\gset
+ \\if :gotlock
+ REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl USING INDEX test_idx;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ SELECT pg_advisory_unlock(42);
+ \\else
+ SELECT pg_advisory_lock(43);
+ BEGIN;
+ INSERT INTO tbl(j) VALUES (nextval('last_j')) RETURNING j \\gset p_
+ COMMIT;
+ SELECT pg_advisory_unlock(43);
+ \\sleep 1 ms
+
+ BEGIN
+ --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ ;
+ SELECT 1;
+ \\sleep 1 ms
+ SELECT COUNT(*) AS count FROM tbl WHERE j <= :p_j \\gset p_
+ \\if :p_count != :p_j
+ COMMIT;
+ SELECT (:p_count) / 0;
+ \\endif
+
+ COMMIT;
+ \\endif
+ )
+});
+
+$node->stop;
+done_testing();
--
2.43.0
[application/x-patch] nocfbot-0001-stress-test-for-repack-concurrently.patch (3.6K, ../../CADzfLwUitd5J17O9FUxNGrZBurOpL6n+tnS6dgArXi-i9DNxhg@mail.gmail.com/4-nocfbot-0001-stress-test-for-repack-concurrently.patch)
download | inline diff:
From db84bbad9d10ffacffc763dbf0ed4bb481f42399 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sat, 13 Dec 2025 18:13:37 +0100
Subject: [PATCH vnocfbot 1/2] stress test for repack concurrently
---
contrib/amcheck/meson.build | 1 +
contrib/amcheck/t/007_repack_concurrently.pl | 110 +++++++++++++++++++
2 files changed, 111 insertions(+)
create mode 100644 contrib/amcheck/t/007_repack_concurrently.pl
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index 1f0c347ed54..2b69081d3bf 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -50,6 +50,7 @@ tests += {
't/004_verify_nbtree_unique.pl',
't/005_pitr.pl',
't/006_verify_gin.pl',
+ 't/007_repack_concurrently.pl',
],
},
}
diff --git a/contrib/amcheck/t/007_repack_concurrently.pl b/contrib/amcheck/t/007_repack_concurrently.pl
new file mode 100644
index 00000000000..a47cebb347b
--- /dev/null
+++ b/contrib/amcheck/t/007_repack_concurrently.pl
@@ -0,0 +1,110 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+));
+
+my $n=1000;
+my $no_hot = int(rand(2));
+
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl(i int PRIMARY KEY, j int)));
+
+if ($no_hot)
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(j);));
+}
+else
+{
+ $node->safe_psql('postgres', q(CREATE INDEX test_idx ON tbl(i);));
+}
+
+
+# Load amcheck
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+
+# Insert $n rows into tbl
+$node->safe_psql('postgres', qq(
+ INSERT INTO tbl SELECT i, i FROM generate_series(1,$n) i
+));
+
+my $sum = $node->safe_psql('postgres', q(
+ SELECT SUM(j) AS sum FROM tbl
+));
+
+
+$node->pgbench(
+'--no-vacuum --client=30 --jobs=4 --exit-on-abort --transactions=5000',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY',
+{
+ 'concurrent_ops' => qq(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \\gset
+ \\if :gotlock
+ REPACK (CONCURRENTLY) tbl USING INDEX tbl_pkey;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl USING INDEX test_idx;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ REPACK (CONCURRENTLY) tbl;
+ SELECT bt_index_parent_check('tbl_pkey', heapallindexed => true);
+ SELECT bt_index_parent_check('test_idx', heapallindexed => true);
+ \\sleep 10 ms
+
+ SELECT pg_advisory_unlock(42);
+ \\else
+ \\set num_a random(1, $n)
+ \\set num_b random(1, $n)
+ \\set diff random(1, 10000)
+ BEGIN;
+ UPDATE tbl SET j = j + :diff WHERE i = :num_a;
+ \\sleep 1 ms
+ UPDATE tbl SET j = j - :diff WHERE i = :num_b;
+ \\sleep 1 ms
+ COMMIT;
+
+ BEGIN
+ --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ ;
+ SELECT 1;
+ \\sleep 1 ms
+ SELECT COALESCE(SUM(j), 0) AS sum FROM tbl \\gset p_
+ \\if :p_sum != $sum
+ COMMIT;
+ SELECT (:p_sum) / 0;
+ \\endif
+
+ COMMIT;
+ \\endif
+ )
+});
+
+$node->stop;
+done_testing();
--
2.43.0
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:45 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-13 18:59 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-13 18:59 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Once it was sent, I realized MVCC-safe fails with
007_repack_concurrently.pl with TRANSACTION ISOLATION LEVEL REPEATABLE
READ uncommented.
Don't know why it fails - but happy it fails :)
On Sat, Dec 13, 2025 at 7:45 PM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
>
> Hello, everyone.
>
> Stress tests for REPACK concurrently in attachment.
> So far I can't break anything (except MVCC of course).
>
> A rebased version of the MVCC-safe "light" version with its own stress
> test is attached also.
>
> Best regards,
> Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:45 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-18 01:47 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-05 10:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-18 01:47 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello!
On Sat, Dec 13, 2025 at 7:45 PM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
> Stress tests for REPACK concurrently in attachment.
To run:
ninja && meson test --suite setup && meson test --print-errorlogs
--suite amcheck *007*
ninja && meson test --suite setup && meson test --print-errorlogs
--suite amcheck *008*
Results for v28:
Up to " v28-0005-Use-background-worker-to-do-logical-decoding.patch":
Technically it passes, but sometimes I saw 0% CPU usage for long
periods with such stacks (looks like it happens for 0008 more often):
epoll_wait 0x000078b99512a037
WaitEventSetWaitBlock waiteventset.c:1192
WaitEventSetWait waiteventset.c:1140
WaitLatch latch.c:196
decode_concurrent_changes cluster.c:2702
repack_worker_internal cluster.c:3777
RepackWorkerMain cluster.c:3725
BackgroundWorkerMain bgworker.c:850
postmaster_child_launch launch_backend.c:268
StartBackgroundWorker postmaster.c:4168
maybe_start_bgworkers postmaster.c:4334
LaunchMissingBackgroundProcesses postmaster.c:3408
ServerLoop postmaster.c:1728
PostmasterMain postmaster.c:1403
main main.c:231
epoll_wait 0x000078b99512a037
WaitEventSetWaitBlock waiteventset.c:1192
WaitEventSetWait waiteventset.c:1140
WaitLatch latch.c:196
ConditionVariableTimedSleep condition_variable.c:165
ConditionVariableSleep condition_variable.c:100
process_concurrent_changes cluster.c:3042
rebuild_relation_finish_concurrent cluster.c:3303
rebuild_relation cluster.c:1121
cluster_rel cluster.c:731
process_single_relation cluster.c:2405
ExecRepack cluster.c:391
standard_ProcessUtility utility.c:864
ProcessUtility utility.c:525
PortalRunUtility pquery.c:1148
PortalRunMulti pquery.c:1306
PortalRun pquery.c:783
exec_simple_query postgres.c:1280
PostgresMain postgres.c:4779
BackendMain backend_startup.c:124
postmaster_child_launch launch_backend.c:268
BackendStartup postmaster.c:3598
ServerLoop postmaster.c:1713
PostmasterMain postmaster.c:1403
main main.c:231
Probably it is because
> 100000L, /* XXX Tune the delay. */
100 seconds is clearly too much.
For "v28-0006-Use-multiple-snapshots-to-copy-the-data.patch":
0007: crash with
TRAP: failed Assert("portal->portalSnapshot == GetActiveSnapshot()"),
File: "../src/backend/tcop/pquery.c", Line: 1169, PID: 178414
postgres: CIC_test: nkey postgres [local]
REPACK(ExceptionalCondition+0xbe)[0x5743f9a955bb]
postgres: CIC_test: nkey postgres [local] REPACK(+0x67fac4)[0x5743f98a7ac4]
postgres: CIC_test: nkey postgres [local] REPACK(+0x67fced)[0x5743f98a7ced]
postgres: CIC_test: nkey postgres [local]
REPACK(PortalRun+0x346)[0x5743f98a7107]
postgres: CIC_test: nkey postgres [local] REPACK(+0x6773bb)[0x5743f989f3bb]
postgres: CIC_test: nkey postgres [local]
REPACK(PostgresMain+0xc1c)[0x5743f98a4f58]
postgres: CIC_test: nkey postgres [local] REPACK(+0x6726c6)[0x5743f989a6c6]
postgres: CIC_test: nkey postgres [local]
REPACK(postmaster_child_launch+0x191)[0x5743f979678c]
postgres: CIC_test: nkey postgres [local] REPACK(+0x5755ca)[0x5743f979d5ca]
postgres: CIC_test: nkey postgres [local] REPACK(+0x572972)[0x5743f979a972]
postgres: CIC_test: nkey postgres [local]
REPACK(PostmasterMain+0x168a)[0x5743f979a225]
postgres: CIC_test: nkey postgres [local] REPACK(main+0x3a1)[0x5743f9662176]
/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x77f80402a1ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x77f80402a28b]
postgres: CIC_test: nkey postgres [local] REPACK(_start+0x25)[0x5743f9311eb5]
0008: pass
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:45 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-18 01:47 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2026-01-05 10:54 ` Antonin Houska <ah@cybertec.at>
2026-01-05 14:07 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2026-01-05 10:54 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Hello!
>
> On Sat, Dec 13, 2025 at 7:45 PM Mihail Nikalayeu
> <mihailnikalayeu@gmail.com> wrote:
> > Stress tests for REPACK concurrently in attachment.
>
> To run:
> ninja && meson test --suite setup && meson test --print-errorlogs
> --suite amcheck *007*
> ninja && meson test --suite setup && meson test --print-errorlogs
> --suite amcheck *008*
Thanks for running the tests!
> Results for v28:
>
> Up to " v28-0005-Use-background-worker-to-do-logical-decoding.patch":
>
> Technically it passes, but sometimes I saw 0% CPU usage for long
> periods with such stacks (looks like it happens for 0008 more often):
...
> Probably it is because
> > 100000L, /* XXX Tune the delay. */
>
> 100 seconds is clearly too much.
I confused milliseconds with microseconds. Since I was only running the code
with debugger, the long delays didn't appear to be a problem.
Instead of tuning the timeout, I'm thinking of introducing a condition
variable that signals WAL flushing.
> For "v28-0006-Use-multiple-snapshots-to-copy-the-data.patch":
>
> 0007: crash with
>
> TRAP: failed Assert("portal->portalSnapshot == GetActiveSnapshot()"),
> File: "../src/backend/tcop/pquery.c", Line: 1169, PID: 178414
Thanks. I'll check when I have time for this part.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:45 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-18 01:47 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-05 10:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2026-01-05 14:07 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 14:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 416+ messages in thread
From: Alvaro Herrera @ 2026-01-05 14:07 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
On 2026-Jan-05, Antonin Houska wrote:
> Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> > Probably it is because
> > > 100000L, /* XXX Tune the delay. */
> >
> > 100 seconds is clearly too much.
>
> I confused milliseconds with microseconds. Since I was only running the code
> with debugger, the long delays didn't appear to be a problem.
>
> Instead of tuning the timeout, I'm thinking of introducing a condition
> variable that signals WAL flushing.
I think there is a patch that adds support for this in the queue already
-- see this message:
https://postgr.es/m/CAPpHfds-KiZRuCruc0jHxLSxLqzKcHJGwOFFA0b_RgaJvtUOEQ@mail.gmail.com
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"¿Qué importan los años? Lo que realmente importa es comprobar que
a fin de cuentas la mejor edad de la vida es estar vivo" (Mafalda)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:45 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-18 01:47 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-05 10:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 14:07 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2026-01-05 14:59 ` Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2026-01-05 14:59 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> On 2026-Jan-05, Antonin Houska wrote:
>
> > Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
>
> > > Probably it is because
> > > > 100000L, /* XXX Tune the delay. */
> > >
> > > 100 seconds is clearly too much.
> >
> > I confused milliseconds with microseconds. Since I was only running the code
> > with debugger, the long delays didn't appear to be a problem.
> >
> > Instead of tuning the timeout, I'm thinking of introducing a condition
> > variable that signals WAL flushing.
>
> I think there is a patch that adds support for this in the queue already
> -- see this message:
> https://postgr.es/m/CAPpHfds-KiZRuCruc0jHxLSxLqzKcHJGwOFFA0b_RgaJvtUOEQ@mail.gmail.com
Thanks for the hint! It seems that the already committed WaitForLSN() function
does what I need.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-13 19:39 ` Antonin Houska <ah@cybertec.at>
2025-12-18 02:05 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
1 sibling, 1 reply; 416+ messages in thread
From: Antonin Houska @ 2025-12-13 19:39 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> On Tue, Dec 9, 2025 at 7:52 PM Antonin Houska <ah@cybertec.at> wrote:
> > Worker makes more sense to me - the initial implementation is in 0005.
>
> Comments for 0005, so far:
Thanks!
> ---
> > export_initial_snapshot
>
> Hm, should we use ExportSnapshot instead? And ImportSnapshort to import it.
There is at least one thing that I don't want: ImportSnapshot calls
SetTransactionSnapshot() at the end. I chose the way leader process uses to
serialize and pass snapshot to background workers.
> ---
> > get_initial_snapshot
>
> Should we check if a worker is still alive while waiting? Also is
> "process_concurrent_changes".
ConditionVariableSleep() should handle that - see the WL_EXIT_ON_PM_DEATH flag
in ConditionVariableTimedSleep().
> And AFAIU RegisterDynamicBackgroundWorker does not guarantee new
> workers to be started (in case of some fork-related issues).
Yes, user will get ERROR in such a case. This is different from parallel
workers in query processing: if parallel worker cannot be started, the leader
(AFAICS) still executes the query. I'm not sure though if we should implement
REPACK (CONCURRENTLY) in such a way that it works even w/o the worker. The
code would be more complex and the behaviour quite different (I mean the
possibly huge amount of unprocessed WAL that you pointed out earlier.)
> ---
> > Assert(res = SHM_MQ_DETACHED);
>
> ==
Thanks!
> ---
> > /* Wait a bit before we retry reading WAL. */
> > (void) WaitLatch(MyLatch,
> > WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
> > 1000L,
> > WAIT_EVENT_REPACK_WORKER_MAIN);
>
> Looks like we need ResetLatch(MyLatch); here.
You seem to be right.
> ---
> > * - decoding_ctx - logical decoding context, to capture concurrent data
>
> Need to be removed together with parameters.
Do you mean in 0005? (It'd help if you pasted the hunk headers.) This should
be fixed in v28 [1]
> ---
> > hpm_context = AllocSetContextCreate(TopMemoryContext,
> > "ProcessParallelMessages",
> > ALLOCSET_DEFAULT_SIZES);
>
> "ProcessRepacklMessages"
ok, the copy and pasting is a problem that needs to be addressed (mentioned in
the last paragraph of the commit message of 0005).
> ---
> > if (XLogRecPtrIsInvalid(lsn_upto))
> > {
> > SpinLockAcquire(&shared->mutex);
> > lsn_upto = shared->lsn_upto;
> > /* 'done' should be set at the same time as 'lsn_upto' */
> > done = shared->done;
> > SpinLockRelease(&shared->mutex);
> >
> > /* Check if the work happens to be complete. */
> > continue;
> > }
>
> May be moved to the start of the loop to avoid duplication.
I found more problems in this part when working on v28, maybe check that.
> ---
> > SpinLockAcquire(&shared->mutex);
> > valid = shared->sfs_valid;
> > SpinLockRelease(&shared->mutex);
>
> Better to remember last_exported here to avoid any races/misses.
What races/misses exactly?
> ---
> > shared->lsn_upto = InvalidXLogRecPtr;
>
> I think it is better to clear it once it is read (after removing duplication).
Maybe, I'll think about it.
> ---
> > bool done;
>
> bool exit_after_lsn_upto?
Not sure.
> ---
> > bool sfs_valid;
>
> Do we really need it? I think it is better to leave only last_exported
> and in process_concurrent_changes wait add argument
> (last_processed_file) and wait for last_exported to become higher.
I'll consider that (The variable is replaced in the 0006 part of v28, but the
idea should still be applicable.)
> ---
> What if we reverse roles of leader-worker?
>
> Leader gets a snapshot, transfers it to workers (multiple probably for
> parallel scan) using already ready mechanics - workers are processing
> the scan of the table in parallel. Leader decodes the WAL.
Insertion into a table by multiple workers is a special thing, but maybe it'd
be doable in this case, but ...
> Also, workers may be assigned with a list of indexes they need to build.
>
> Feels like it reuses more from current infrastructure and also needs
> less different synchronization logic. But I'm not sure about the
> indexes phase - maybe it is not so easy to do.
... my feelings were the opposite, i.e. I thought require higher amount of
code rearrangement. Moreover, the part 0006 of v28 (snapshot switching) would
be trickier. It processes one range of blocks after another, and parallelism
would make it more difficult.
> ---
> Also, should we add some kind of back pressure between building
> indexes/new heap and num of WAL we have?
> But probably it is out of scope of the patch.
Do you mean that the decoding worker should be less active if the amount of
WAL doesn't grow too fast?
> ---
> To build N indexes we need to scan table N times. What is about
> building multiple indexes during a single heap scan?
That sounds like a separate feature, and similarly difficult as enhancing
CREATE INDEX so it can create multiple indexes at a time.
> --
> Just a gentle reminder about the XMIN_COMMITTED flag and WAL storm
> after the switch.
ok, I have it in my notes, moved it more to the top :-)
[1] https://www.postgresql.org/message-id/210036.1765651719%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 19:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
@ 2025-12-18 02:05 ` Mihail Nikalayeu <mihailnikalayeu@gmail.com>
0 siblings, 0 replies; 416+ messages in thread
From: Mihail Nikalayeu @ 2025-12-18 02:05 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Hello, Antonin!
On Sat, Dec 13, 2025 at 8:39 PM Antonin Houska <ah@cybertec.at> wrote:
> > ---
> > > SpinLockAcquire(&shared->mutex);
> > > valid = shared->sfs_valid;
> > > SpinLockRelease(&shared->mutex);
> >
> > Better to remember last_exported here to avoid any races/misses.
>
> What races/misses exactly?
Just as some way to reduce a number of potential scenarios/states
between parallel actors.
> > ---
> > > bool done;
> >
> > bool exit_after_lsn_upto?
>
> Not sure.
I think it should be named in some way to signal it is a request, not a report.
> > Also, should we add some kind of back pressure between building
> > indexes/new heap and num of WAL we have?
> > But probably it is out of scope of the patch.
>
> Do you mean that the decoding worker should be less active if the amount of
> WAL doesn't grow too fast?
In the previous version (without background) we have some kind of
back-pressure during the scan part (if we have too muchWAL delayed
because of us - we process it).
But it is not more true with a background worker. At the same time -
it never was during the index building phase...
Best regards,
Mikhail.
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-08 09:51 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-12-08 09:51 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> Some comments for 0003:
>
> > /* allocate in transaction context */
> It may be any context now, because it is a function now.
Inaccuracy not introduced by REPACK, but I think it's o.k. if the next version
of this patch will remove the comment.
> > result = CopySnapshot(snapshot);
>
> > /* Restore the original values so the source is intact. */
> > snapshot->xip = oldxip;
> > snapshot->xcnt = oldxcnt;
>
> I think it is worth to call pfree(newxip) here.
ok
> > "This difference does has no impact"
>
> should be "This difference has no impact"?
Right, thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
@ 2025-12-08 07:35 ` Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 416+ messages in thread
From: Antonin Houska @ 2025-12-08 07:35 UTC (permalink / raw)
To: Mihail Nikalayeu <mihailnikalayeu@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Pg Hackers <pgsql-hackers@lists.postgresql.org>; Robert Treat <rob@xzilla.net>
Mihail Nikalayeu <mihailnikalayeu@gmail.com> wrote:
> On Thu, Dec 4, 2025 at 6:43 PM Antonin Houska <ah@cybertec.at> wrote:
> > v26 attached here. It's been rebased and reflects most of the feedback.
>
> Some comments on 0001-0002:
> 1)
>
> > cluster_rel(stmt->command, rel, indexOid, params);
> cluster_rel closes relation, and after it is dereferenced a few lines after.
> Technically it may be correct, but feels a little bit strange.
ok, will be fixed in the next version (supposedly later today).
> 2)
>
> > if (vacopts->mode == MODE_VACUUM)
> I think for better compatibility it is better to handle new value in
> if - (vacopts->mode == MODE_REPACK) to keep old cases unchanged
I suppose you mean vacuuming.c. We're considering removal of pg_repackdb from
the patchset, so let's decide on this later.
> 3)
>
> > case T_RepackStmt:
> > tag = CMDTAG_REPACK;
> > break;
>
> should we use instead:
>
> case T_RepackStmt:
> if (((RepackStmt *) parsetree)->command == REPACK_COMMAND_CLUSTER)
> tag = CMDTAG_CLUSTER;
> else
> tag = CMDTAG_REPACK;
> break;
>
> or delete CMDTAG_CLUSTER - since it not used anymore
LGTM, will include it in the next version.
> 4)
> "has been superceded by"
> typo
ok. (This may also be removed, as it's specific to pg_repackdb.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
@ 2025-12-04 15:17 David Klika <david.klika@atlas.cz>
2025-12-04 15:43 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 416+ messages in thread
From: David Klika @ 2025-12-04 15:17 UTC (permalink / raw)
To: alvherre@alvh.no-ip.org; ah@cybertec.at; +Cc: jian.universality@gmail.com; pgsql-hackers@lists.postgresql.org; mihailnikalayeu@gmail.com; rob@xzilla.net
Hello
Great to hear about this feature.
You speak about table rewrite (suppose a whole-table rewrite). I would
like to share idea of an alternative approach that also takes into
account amount of WAL generated during the operation. Applicable to
non-clustered case only.
Let's consider a large table where 80% blocks are fine (filled enough by
live tuples). The table could be scanned from the beginning (left side)
to identify "not enough filled" blocks and also from the end (right
side) to process live tuples by moving them to the blocks identified
by the left side scan. The work is over when both scan reaches the same
position.
Example:
_ stands for filled enough blocks
D stands for blocks with (many) dead tuples
123456789
___DD____
Left scan identifies page #4 and tuples from the right scan (page #9)
are moved here. The same with tuples from #8 to #5. Two pages from the
data file are trimmed and (only) pages #4 and #5 are written in WAL,
others are untouched.
Regards
David
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-12-04 15:17 Re: Adding REPACK [concurrently] David Klika <david.klika@atlas.cz>
@ 2025-12-04 15:43 ` Álvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:47 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
2025-12-08 09:13 ` Re: Adding REPACK [concurrently] David Klika <david.klika@atlas.cz>
0 siblings, 2 replies; 416+ messages in thread
From: Álvaro Herrera @ 2025-12-04 15:43 UTC (permalink / raw)
To: David Klika <david.klika@atlas.cz>; +Cc: ah@cybertec.at; jian.universality@gmail.com; pgsql-hackers@lists.postgresql.org; mihailnikalayeu@gmail.com; rob@xzilla.net
Hello David,
Thanks for your interest in this.
On 2025-Dec-04, David Klika wrote:
> Let's consider a large table where 80% blocks are fine (filled enough by
> live tuples). The table could be scanned from the beginning (left side) to
> identify "not enough filled" blocks and also from the end (right side) to
> process live tuples by moving them to the blocks identified by the left side
> scan. The work is over when both scan reaches the same position.
If you only have a small number of pages that have this problem, then
you don't actually need to do anything -- the pages will be marked free
by regular vacuuming, and future inserts or updates can make use of
those pages. It's not a problem to have a small number of pages in
empty state for some time.
So if you're trying to do this, the number of problematic pages must be
large.
Now, the issue with what you propose is that you need to make either the
old tuples or the new tuples visible to concurrent transactions. If at
any point they are both visible, or none of them is visible, then you
have potentially corrupted the results that would be obtained by a query
that's scanning the table and halfway through.
The other point is that you need to keep indexes updated. That is, you
need to make the indexes point to both the old and new, until you remove
the old tuples from the table, then remove those index pointers.
This process bloats the indexes, which is not insignificant, considering
that the number of tuples to process is large. If there are several
indexes, this makes your process take even longer.
You can fix the concurrency problem by holding a lock on the table that
ensures nobody is reading the table until you've finished. But we don't
want to have to hold such a lock for long! And we already established
that the number of pages to check is large, which means you're going to
work for a long time.
So, I'm not really sure that it's practical to implement what you
suggest.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-12-04 15:17 Re: Adding REPACK [concurrently] David Klika <david.klika@atlas.cz>
2025-12-04 15:43 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-12-04 17:47 ` Marcos Pegoraro <marcos@f10.com.br>
2025-12-06 10:58 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 416+ messages in thread
From: Marcos Pegoraro @ 2025-12-04 17:47 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: David Klika <david.klika@atlas.cz>; ah@cybertec.at; jian.universality@gmail.com; pgsql-hackers@lists.postgresql.org; mihailnikalayeu@gmail.com; rob@xzilla.net
Em qui., 4 de dez. de 2025 às 12:43, Álvaro Herrera <alvherre@alvh.no-ip.org>
escreveu:
> If you only have a small number of pages that have this problem, then
> you don't actually need to do anything -- the pages will be marked free
> by regular vacuuming, and future inserts or updates can make use of
> those pages. It's not a problem to have a small number of pages in
> empty state for some time.
>
> So if you're trying to do this, the number of problematic pages must be
> large.
Not necessarily. I have some tables where I like to use CLUSTER
every 2 or 3 months, to reorganize the data based on an index
and consequently load fewer pages with each call. These tables
don't have more than 2 or 3% of dead records, but they are quite
disorganized from the point of view of that index, since the
inserted and updated records don't follow the order I determined.
regards
Marcos
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-12-04 15:17 Re: Adding REPACK [concurrently] David Klika <david.klika@atlas.cz>
2025-12-04 15:43 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:47 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
@ 2025-12-06 10:58 ` Álvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 0 replies; 416+ messages in thread
From: Álvaro Herrera @ 2025-12-06 10:58 UTC (permalink / raw)
To: Marcos Pegoraro <marcos@f10.com.br>; +Cc: David Klika <david.klika@atlas.cz>; ah@cybertec.at; jian.universality@gmail.com; pgsql-hackers@lists.postgresql.org; mihailnikalayeu@gmail.com; rob@xzilla.net
Hello,
On 2025-Dec-04, Marcos Pegoraro wrote:
> Em qui., 4 de dez. de 2025 às 12:43, Álvaro Herrera <alvherre@alvh.no-ip.org>
> escreveu:
>
> > So if you're trying to do this, the number of problematic pages must
> > be large.
>
> Not necessarily. I have some tables where I like to use CLUSTER every
> 2 or 3 months, to reorganize the data based on an index and
> consequently load fewer pages with each call. These tables don't have
> more than 2 or 3% of dead records, but they are quite disorganized
> from the point of view of that index, since the inserted and updated
> records don't follow the order I determined.
I don't understand what does this have to do with what David was
proposing. I mean, you're right: if all you want is to CLUSTER, you may
not have an enormous number of pages to get rid of. But how can you use
the technique he proposes to deal with reordering tuples? If you just
move the tuples from the end of the table to where some random hole has
appeared, you've not clustered the table at all.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"People get annoyed when you try to debug them." (Larry Wall)
^ permalink raw reply [nested|flat] 416+ messages in thread
* Re: Adding REPACK [concurrently]
2025-12-04 15:17 Re: Adding REPACK [concurrently] David Klika <david.klika@atlas.cz>
2025-12-04 15:43 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
@ 2025-12-08 09:13 ` David Klika <david.klika@atlas.cz>
1 sibling, 0 replies; 416+ messages in thread
From: David Klika @ 2025-12-08 09:13 UTC (permalink / raw)
To: Álvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: ah@cybertec.at; jian.universality@gmail.com; pgsql-hackers@lists.postgresql.org; mihailnikalayeu@gmail.com; rob@xzilla.net
Hello Alvaro
Thank you for the detailed analysis.
Dne 04.12.2025 v 16:43 Álvaro Herrera napsal(a):
> Hello David,
>
> Thanks for your interest in this.
>
> On 2025-Dec-04, David Klika wrote:
>
>> Let's consider a large table where 80% blocks are fine (filled enough by
>> live tuples). The table could be scanned from the beginning (left side) to
>> identify "not enough filled" blocks and also from the end (right side) to
>> process live tuples by moving them to the blocks identified by the left side
>> scan. The work is over when both scan reaches the same position.
> If you only have a small number of pages that have this problem, then
> you don't actually need to do anything -- the pages will be marked free
> by regular vacuuming, and future inserts or updates can make use of
> those pages. It's not a problem to have a small number of pages in
> empty state for some time.
>
> So if you're trying to do this, the number of problematic pages must be
> large.
I agree, I had in mind about 20-40% of the table that could have tenths
of GB.
> Now, the issue with what you propose is that you need to make either the
> old tuples or the new tuples visible to concurrent transactions. If at
> any point they are both visible, or none of them is visible, then you
> have potentially corrupted the results that would be obtained by a query
> that's scanning the table and halfway through.
When performing a tuple movement from a (right) page to a (left) page,
both of pages must be hold in shared buffers. I suppose the other
processes scanning the table also access the table data through the
shared buffers so the movement could be handled at this level. If the
tuple movement does not change its xid, it wouldn't even have to be in
conflict with other transactions that locked/modified the tuple (in
buffer cache again, just changing the physical location). Looks like
something dirty...
> The other point is that you need to keep indexes updated. That is, you
> need to make the indexes point to both the old and new, until you remove
> the old tuples from the table, then remove those index pointers.
> This process bloats the indexes, which is not insignificant, considering
> that the number of tuples to process is large. If there are several
> indexes, this makes your process take even longer.
>
> You can fix the concurrency problem by holding a lock on the table that
> ensures nobody is reading the table until you've finished. But we don't
> want to have to hold such a lock for long! And we already established
> that the number of pages to check is large, which means you're going to
> work for a long time.
> So, I'm not really sure that it's practical to implement what you
> suggest.
I agree. Proposed tuple shuffle might work better compared to the
current VACUUM FULL (i.e. blocking non-clustered maintenance) but I
understand that you prefer an universal method of data files maintenance
(the concurrent variant will be amazing).
Regards David
^ permalink raw reply [nested|flat] 416+ messages in thread
end of thread, other threads:[~2026-07-01 14:43 UTC | newest]
Thread overview: 416+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-07-05 14:24 [PATCH 03/17] Teach postmaster to accept encryption key Antonin Houska <ah@cybertec.at>
2020-02-27 01:22 [PATCH v32 3/3] Avoid some calls to RelationGetRelationName Justin Pryzby <pryzbyj@telsasoft.com>
2025-07-26 21:56 Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 01:59 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-07-31 16:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-01 11:07 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-04 23:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 12:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-09 13:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 23:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-21 18:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-24 16:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 13:09 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 14:15 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 15:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 16:23 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-25 17:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-25 18:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 08:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-26 09:02 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-26 13:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 00:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 06:16 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 08:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-27 10:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-27 10:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 00:16 ` Re: Adding REPACK [concurrently] Michael Paquier <michael@paquier.xyz>
2025-08-25 16:36 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-25 16:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-28 21:39 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-29 00:32 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-29 07:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-11 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-15 12:32 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-15 12:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-07-27 06:00 ` Re: Adding REPACK [concurrently] Fujii Masao <masao.fujii@gmail.com>
2025-08-05 08:58 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-16 13:41 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-19 12:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-20 08:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-19 12:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-19 18:53 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 08:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 12:07 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-20 14:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-20 16:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2025-08-21 18:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-08-21 18:16 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2025-08-21 22:06 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-22 09:40 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-22 20:32 ` Re: Adding REPACK [concurrently] Euler Taveira <euler@eulerto.com>
2025-08-23 06:56 ` Re: Adding REPACK [concurrently] Michael Banck <mbanck@gmx.net>
2025-08-23 14:22 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-08-25 16:03 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-08-30 17:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 12:09 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-08-31 15:29 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-08-31 17:43 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-01 13:00 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 05:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-01 09:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-01 15:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-02 10:44 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-09-03 09:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-09-23 15:51 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-09-25 18:12 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-09-25 20:20 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
2025-09-25 21:31 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-09-25 21:46 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
2025-09-26 14:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-10-07 14:05 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-10-09 06:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-10-09 11:49 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@kurilemu.de>
2025-10-13 00:03 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-09-26 17:30 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-10-10 14:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-10-30 23:17 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-11-01 12:42 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-11-01 12:53 ` Re: Adding REPACK [concurrently] Sergei Kornilov <sk@zsrv.org>
2025-12-04 13:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-11-01 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-11-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-02 00:50 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-02 16:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-02 16:22 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-03 07:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-11-05 02:48 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-11-05 05:10 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-11-05 07:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-11-09 22:13 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2025-11-05 08:46 ` Re: Adding REPACK [concurrently] jian he <jian.universality@gmail.com>
2025-12-04 17:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-05 00:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-06 18:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-07 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-09 18:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-09 19:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-13 18:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-13 19:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-15 14:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 10:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 10:40 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-08 16:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-10 17:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 16:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-12 18:20 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-12 18:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-15 16:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-15 17:05 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-16 18:18 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-19 16:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-18 21:52 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-22 08:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-22 11:30 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-25 16:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-26 07:34 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-27 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-28 02:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-30 19:33 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-01 19:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-01 22:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-01 22:37 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:17 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 10:04 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-02 19:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-06 16:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-07 14:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-09 11:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 07:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-02 09:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-06 15:08 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-15 16:03 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-16 11:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-02-16 15:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-16 15:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-16 19:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-23 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-24 18:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 08:55 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 16:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 16:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-02-25 19:04 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-27 18:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-28 15:16 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-02 14:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-02 17:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-05 19:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-07 14:28 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-09 10:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-10 19:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-10 20:05 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 01:13 ` RE: Adding REPACK [concurrently] Shinoda, Noriyoshi (PSD Japan FSI) <noriyoshi.shinoda@hpe.com>
2026-03-11 16:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 18:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-12 19:15 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-03-12 19:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-11 15:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-11 17:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-12 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-13 08:11 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 09:13 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 12:21 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 14:21 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 15:10 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 15:49 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-03-16 16:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 17:55 ` Re: Adding REPACK [concurrently] Mahendra Singh Thalor <mahi6run@gmail.com>
2026-03-16 19:26 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 19:24 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 19:34 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-16 21:45 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 20:15 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 21:50 ` Re: Adding REPACK [concurrently] Matthias van de Meent <boekewurm+postgres@gmail.com>
2026-03-16 22:03 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-17 10:53 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-17 19:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-18 19:12 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-18 20:07 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-19 19:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-19 20:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 07:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-23 10:20 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 12:35 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-03-20 15:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-24 22:32 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-25 02:57 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 20:12 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-26 09:51 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-26 11:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-26 23:14 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-26 14:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 17:37 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-31 18:22 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 18:23 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 09:39 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 11:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 12:21 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-25 09:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-25 15:52 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 15:59 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:15 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-27 18:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-02 00:35 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-02 08:27 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 15:58 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-25 16:52 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-26 17:28 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-26 08:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 16:12 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-27 17:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-29 22:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 10:47 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-03-31 14:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-31 15:35 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-31 19:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 14:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 17:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 21:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 12:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-03 14:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 19:37 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 06:20 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-04 10:19 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-06 05:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 21:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-10 07:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
2026-04-10 10:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-12 14:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
2026-04-13 09:42 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-10 10:53 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-10 13:21 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-25 06:26 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-26 15:31 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-27 08:08 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-28 00:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-28 03:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-28 05:18 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-05-29 15:39 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-29 22:25 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-01 02:25 ` RE: Adding REPACK [concurrently] Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2026-04-21 07:24 ` Re: Adding REPACK [concurrently] Chao Li <li.evan.chao@gmail.com>
2026-04-04 13:16 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-04 13:55 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-03 17:24 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 08:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 09:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-04 15:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-04 23:53 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 07:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 11:50 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 16:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-05 18:56 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-05 20:41 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 09:15 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 09:38 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 10:01 ` Re: Adding REPACK [concurrently] vignesh C <vignesh21@gmail.com>
2026-04-06 11:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-06 11:56 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-06 21:11 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-06 21:59 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 09:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-06-19 06:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 01:10 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 01:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 02:42 ` Re: Adding REPACK [concurrently] Noah Misch <noah@leadboat.com>
2026-04-07 04:44 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-07 08:40 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 08:42 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 08:57 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 09:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 09:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 10:08 ` Re: Adding REPACK [concurrently] John Naylor <johncnaylorls@gmail.com>
2026-04-07 10:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 09:29 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 09:41 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-07 11:19 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-08 10:20 ` Re: Adding REPACK [concurrently] Tomas Vondra <tomas@vondra.me>
2026-04-08 10:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-08 16:47 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-09 06:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-06 10:14 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 10:48 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 11:21 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-06 22:22 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-06 23:53 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-07 12:15 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-07 12:33 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 14:10 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 15:38 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 15:48 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-07 16:01 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-27 04:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-27 04:46 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-30 11:24 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-05-04 13:24 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 12:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-05 15:02 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-06 08:25 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-08 12:25 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-08 13:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-08 23:22 ` Re: Adding REPACK [concurrently] Masahiko Sawada <sawada.mshk@gmail.com>
2026-05-10 11:24 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-10 11:31 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-11 15:17 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-11 19:30 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-12 07:57 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-12 11:08 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-05-13 12:34 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-13 16:58 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-14 07:02 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-19 18:52 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-05-23 15:29 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-24 11:29 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-03 17:27 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-06-17 18:19 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-06-17 20:10 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-07-01 14:43 ` Re: Adding REPACK [concurrently] Heikki Linnakangas <hlinnaka@iki.fi>
2026-05-12 02:25 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-05-05 15:04 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-07 12:32 ` RE: Adding REPACK [concurrently] Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
2026-04-07 14:19 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-08 03:49 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-05-08 23:22 ` Re: Adding REPACK [concurrently] Masahiko Sawada <sawada.mshk@gmail.com>
2026-05-01 07:30 ` Re: Adding REPACK [concurrently] 'Alvaro Herrera' <alvherre@alvh.no-ip.org>
2026-04-07 19:43 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-04-08 07:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-07 20:24 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 08:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-08 17:22 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-08 23:36 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 04:54 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-09 08:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 09:11 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 09:26 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-09 14:06 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 14:26 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-09 14:34 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-09 14:20 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-10 16:14 ` Re: Adding REPACK [concurrently] Robert Treat <rob@xzilla.net>
2026-04-10 18:56 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-12 14:02 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 13:31 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-12 14:05 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-12 14:58 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-14 13:37 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-14 13:58 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-14 16:55 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-15 14:50 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-15 14:59 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-15 18:28 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-16 11:18 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-17 02:01 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-18 19:23 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-18 22:46 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-20 17:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-23 11:43 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-26 13:34 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-19 11:00 ` Re: Adding REPACK [concurrently] Alexander Lakhin <exclusion@gmail.com>
2026-04-17 14:44 ` Re: Adding REPACK [concurrently] Justin Pryzby <pryzby@telsasoft.com>
2026-04-20 07:44 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:47 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 07:54 ` Re: Adding REPACK [concurrently] Chao Li <li.evan.chao@gmail.com>
2026-04-20 11:45 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-20 13:54 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 14:42 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 15:11 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-20 16:01 ` Re: Adding REPACK [concurrently] Tom Lane <tgl@sss.pgh.pa.us>
2026-04-20 16:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-22 09:30 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-23 11:01 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-15 15:54 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-04-09 14:13 ` Re: Adding REPACK [concurrently] Andres Freund <andres@anarazel.de>
2026-04-06 09:03 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-02 00:19 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-04-01 08:42 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 09:43 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 10:22 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 11:42 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-04-01 12:35 ` Re: Adding REPACK [concurrently] Amit Kapila <amit.kapila16@gmail.com>
2026-04-01 13:20 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-04-01 12:31 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-16 13:46 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-25 13:55 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-02-25 19:41 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-02-26 12:19 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
2026-03-06 07:14 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-20 18:06 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 12:00 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-23 16:07 ` Re: Adding REPACK [concurrently] Jim Jones <jim.jones@uni-muenster.de>
2026-03-24 08:48 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-03-24 12:13 ` Re: Adding REPACK [concurrently] Jim Jones <jim.jones@uni-muenster.de>
2026-03-26 13:19 ` Re: Adding REPACK [concurrently] Srinath Reddy Sadipiralla <srinath2133@gmail.com>
2026-03-26 19:36 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-20 15:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-11 20:38 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:45 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-13 18:59 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-18 01:47 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2026-01-05 10:54 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2026-01-05 14:07 ` Re: Adding REPACK [concurrently] Alvaro Herrera <alvherre@alvh.no-ip.org>
2026-01-05 14:59 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-13 19:39 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-18 02:05 ` Re: Adding REPACK [concurrently] Mihail Nikalayeu <mihailnikalayeu@gmail.com>
2025-12-08 09:51 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-08 07:35 ` Re: Adding REPACK [concurrently] Antonin Houska <ah@cybertec.at>
2025-12-04 15:17 Re: Adding REPACK [concurrently] David Klika <david.klika@atlas.cz>
2025-12-04 15:43 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-04 17:47 ` Re: Adding REPACK [concurrently] Marcos Pegoraro <marcos@f10.com.br>
2025-12-06 10:58 ` Re: Adding REPACK [concurrently] Álvaro Herrera <alvherre@alvh.no-ip.org>
2025-12-08 09:13 ` Re: Adding REPACK [concurrently] David Klika <david.klika@atlas.cz>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox