Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vQ1UM-00669d-1Z for pgsql-hackers@arkaria.postgresql.org; Mon, 01 Dec 2025 10:50:38 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vQ1UK-00272u-2h for pgsql-hackers@arkaria.postgresql.org; Mon, 01 Dec 2025 10:50:37 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vQ1UK-00272l-1g for pgsql-hackers@lists.postgresql.org; Mon, 01 Dec 2025 10:50:36 +0000 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1vQ1UI-002V8k-2x for pgsql-hackers@postgresql.org; Mon, 01 Dec 2025 10:50:36 +0000 Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-5957db5bdedso4580735e87.2 for ; Mon, 01 Dec 2025 02:50:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764586228; x=1765191028; darn=postgresql.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=1muemoyr8jPr/IfFwQQi03zCEdHeh+Fm983g75ng+LE=; b=mByL7o/ZhgHa3iiZe1S+ZEwOGXkmtWWxnNMs24CYfSZRX9WwH1l3lcWq+I5pEcqIlo HezpZgYOi3eEvpOCQFxwwd3uWAuX3PlfU7VQ7Gs9QrBYf3ZNRShIl/Z5aIQSXibpeqWc v0dI9vWXGwecu6ARbNk2p2Tw1L8O+0RrhWKUT4nNoHhAuxeQ0ISF1/kCGAQ537Wbr+7l uBZhs4BcjAhGMYIuDQA2si605V9HibmTgICyhbrVrAZdVm64MZLcaOytdKSAYutIyP1k G/2fZ/e9yDc/dETDN/JmzVrLeeXCTXzUVN87UpISn0pLucdX95dyLmgvnh05J5Bp23NN EjMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764586228; x=1765191028; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=1muemoyr8jPr/IfFwQQi03zCEdHeh+Fm983g75ng+LE=; b=O25MTJ7wTY0o1pLQkla7fW/aPATszsmnKpyBVZ/BoucgoXRrxiIgmkpqMIlKgRU1cb +8x00lamA5Vabd3TDVdW1eBhsueaaUN+KJ0Gc5Wep8OPi0iiAJuQrRt03wqs2Q9j40IL 5MnYRWu3FxqnF/R3N0ADjUz5WmBwOvWxwS5OJsAQQDGyhu463KqOJYXEpTwJbxwDs7nQ MbLXN5tCrwMJ1NBTynPjvlOkry/qQ/hXWFzytHAQjzo9sdnzxpr7Rrc/uGD7V9puuXOw HtnypVzLcNlfD8YCPMk3a+i/cebXCm0idkrNVG/HqFCHHQwdTHJwJyxctIS7QtbiY2ZZ 7O+g== X-Forwarded-Encrypted: i=1; AJvYcCW2rYCPdCd8hcvEYlUOsP/84byBNxpqzq3qawTP+ruc2xdmp2QKECgGBeX8F9ryIvHHo+TY+R4179ROSLRx@postgresql.org X-Gm-Message-State: AOJu0YwgBolBwxPRNPB6d1BzfmVNqQFKuefAJkaV7BplzvTAGieANLtM 1YTATOkiO4ty9VrnracwEdMDSg1glkQmwoPCMlRe6qHkfKDrAzYfSoWQc0B6h36yA6YmR6eWG9H +4CZ5j26XX14rEXjECasJgOcZVwE8Kq0= X-Gm-Gg: ASbGncsPWO0qJ7JGxxwFQB9iILg41MNWlsCQ1nKJYmXw9naD4VRig8wbd24Yher9n7y acRQlCqG3dvie0IfVoLz6NIDbaXl3GIxVcswHaBPDBwtVr560Dovgm5LyUZjLFmD2zmohRKW5w2 Xc0mQdjzaFK4sMgNbp/0PRQlY7+mhYQg0HIJ7HMPT3R6iYaddkOns5hPUOpb4Fc9JQCNkTnWAiV sDEJeIXv2oXKVyC72R2MQqD+XbEZq5E4BpbOkGqWohDyfDAkFBXjNsO0uV8Cm6/Rb0VN8k6wenm 6dOJJ1N/q2YT4sFz+SZPYvX2jvz8TQ== X-Google-Smtp-Source: AGHT+IGU8/Zn1g7pbxYFXmehPiCuZ93St/8BQ7KvELrQWv3AWUDWwi1AUtcxs4eY61NP12leoFO7WuqQIwteHdSYSAA= X-Received: by 2002:a05:6512:3b29:b0:594:2e8a:1663 with SMTP id 2adb3069b0e04-596a3eb26b7mr12696247e87.17.1764586228215; Mon, 01 Dec 2025 02:50:28 -0800 (PST) MIME-Version: 1.0 References: <202505181556.3n6oiowvntyr@alvherre.pgsql> <8010.1764584989@localhost> In-Reply-To: <8010.1764584989@localhost> From: Mihail Nikalayeu Date: Mon, 1 Dec 2025 11:49:37 +0100 X-Gm-Features: AWmQ_bkTiMqQ87_ZTaJT9d20FLvJeYOq2FiIR3Kf4eC8HOEY48oIMGaSYmYKHrs Message-ID: Subject: Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements To: Antonin Houska Cc: Hannu Krosing , Matthias van de Meent , Sergey Sargsyan , alvherre@kurilemu.de, Andres Freund , Michael Paquier , PostgreSQL Hackers , Andrey Borodin , Melanie Plageman Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hello, Antonin! On Mon, Dec 1, 2025 at 11:29=E2=80=AFAM Antonin Houska wro= te: > With logical replication, we cannot really use multiple snapshots as Miha= il is > proposing elsewhere in the thread, because the logical decoding system on= ly > generates the snapshot for non-catalog tables once (LR uses that snapshot= for > the initial table synchronization). Only snapshots for system catalog tab= les > are then built as the WAL decoding progresses. It can be worked around by > considering regular table as catalog during the processing, but it curren= tly > introduces quite some overhead: My idea related to REPACK is a little bit different. I am not talking about snapshots generated by LR - just GetLatestSnapshot. > The core problem here is that the snapshot you need for the first pass > restricts VACUUM on all tables in the database We might use it only for a few seconds - it is required only to *start* the scan (to ensure we will not miss anything in the table). After we may throw it away and ask GetLatestSnapshot a fresh one for next N pages. We just need to synchronize scan position in the table and logical decoding. The same is possible for CIC too. In that case we should do the same and just store all incoming tuples the same way as STIR does it. Best regards, Mikhail.