Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rBeeD-00BkwP-Fu for pgsql-hackers@arkaria.postgresql.org; Fri, 08 Dec 2023 17:28:21 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rBeeB-009y6K-Qg for pgsql-hackers@arkaria.postgresql.org; Fri, 08 Dec 2023 17:28:19 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rBeeB-009y6C-C6 for pgsql-hackers@lists.postgresql.org; Fri, 08 Dec 2023 17:28:19 +0000 Received: from mail-lf1-x12c.google.com ([2a00:1450:4864:20::12c]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rBee5-009Raz-12 for pgsql-hackers@lists.postgresql.org; Fri, 08 Dec 2023 17:28:17 +0000 Received: by mail-lf1-x12c.google.com with SMTP id 2adb3069b0e04-50be9e6427dso2459467e87.1 for ; Fri, 08 Dec 2023 09:28:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb.com; s=google; t=1702056491; x=1702661291; darn=lists.postgresql.org; h=in-reply-to:references:cc:to:from:content-language:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=4wnLo5fK5pTErwTjrlaSSqQPKY1OdoUJ+aUSiVvxW6Q=; b=QCbXWLdnkI1StwyVgSwdOa/vKtC0ninejNzNKYSVUIK7/W6W8A99a0Lp8v6t1XbhrQ qLXVFdBa0AhoevOc/MDr6856+ae3T3F0feVwZpeXgfcS6WfFcSN+agTKikCSuN62PVAz UQAKkTVRF1Gd7mQSkwCQGQUL/E0Q8VFoe6kCZZveU3evZAVfxNCxVWsV6JHMmDjOS2Nv rC0Y0QRrJBxuEMcp/+hVR50JBJC6YsAQixF1Y1q96hk8YsocNyjFqTPM4cIvJftVKby/ jeCc7DofyitxjhgUGdwvc98S2UBpNWFGhhxp5642+RhNTmaijJEPYu4BgZ+0awcC/LcM 9MXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702056491; x=1702661291; h=in-reply-to:references:cc:to:from:content-language:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=4wnLo5fK5pTErwTjrlaSSqQPKY1OdoUJ+aUSiVvxW6Q=; b=Y0qqLeeBquxYZfo/nBOggrzt20CPeGERLB7aNlIsI4HFrADGuSlVs4e6lqRZXeYafb TZ4vrnR0m88O6RXSg45s138LM0WQSpeqv8ovUFrNqjjNwZnnipVD491f0ab2s5XukAbG HXDCOGUEqIJwTuQ3f4OELI38fZyAaAQQm/Vzj9MN78+at1/cmCZQVtpkldryTfWacvcm 1c50045EvKUgamDhSHIkN1u+ECrkXe1D0yYrxR/nViWhp5omFhCjT+b9kAPcm2Wl7k0H bIIMytWXw2XtRnKHuVGTyS7DrtdRizIYsM7ALHpaCGcKmeN6aey4jvuNQ8kjcPcHkHoR D0DQ== X-Gm-Message-State: AOJu0Yy0akakriDjC8egxQaxvENRZAiR2tF0RGNjVsFrV/n++IrYySs5 PX5C6qcBHWrqKyJltcv9+fkJaW6+m9tCIhxJOjQ= X-Google-Smtp-Source: AGHT+IGnm4L9qQpW8lpFlULuNt4r/nZT7DSMH5ecCk0dbsLDUV6FLCgejk1AVAbPHySRN7Yr13/Iyw== X-Received: by 2002:ac2:5935:0:b0:50b:f82e:6d5a with SMTP id v21-20020ac25935000000b0050bf82e6d5amr121236lfi.73.1702056490948; Fri, 08 Dec 2023 09:28:10 -0800 (PST) Received: from [10.137.0.17] (ip-86-49-229-30.bb.vodafone.cz. [86.49.229.30]) by smtp.gmail.com with ESMTPSA id k12-20020a50c8cc000000b0054f54cc50d1sm601325edh.22.2023.12.08.09.28.10 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 Dec 2023 09:28:10 -0800 (PST) Content-Type: multipart/mixed; boundary="------------b33pcwSN8vzgHMPIN25AcNz0" Message-ID: Date: Fri, 8 Dec 2023 18:28:09 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Subject: Re: Parallel CREATE INDEX for BRIN indexes Content-Language: en-US From: Tomas Vondra To: Matthias van de Meent Cc: PostgreSQL Hackers References: <5384d42e-1961-c2fd-1955-1a5a99fa1234@enterprisedb.com> <9a646c51-a4c3-ce34-cccd-99d00fd1dc6a@enterprisedb.com> <64adb9cb-3474-e0ff-7a38-9bed4f5170e6@enterprisedb.com> <99f9ecbe-0976-8012-67b5-dbff58c7dbec@enterprisedb.com> <32e3f645-389d-87f9-2ce7-33d0087f47d4@enterprisedb.com> In-Reply-To: <32e3f645-389d-87f9-2ce7-33d0087f47d4@enterprisedb.com> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------b33pcwSN8vzgHMPIN25AcNz0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hi, I've pushed the first two parts (backfill of empty ranges for serial builds, allowing parallelism) after a bit more cleanup, adding a simple pageinspect test to 0001, improving comments and some minor adjustments. I ended up removing the protections against BlockNumber overflows, and moved them into a separate WIP patch. I still think we should probably reconsider the position that we don't need to worry about issues so close to the 32TB boundary, but it seemed rather weird to fix only the new bits and leave the existing issues in place. I'm attaching that as a WIP patch, but I don't know if/when I'll get back to this. Thanks for the reviews/reworks/ideas! -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --------------b33pcwSN8vzgHMPIN25AcNz0 Content-Type: text/x-patch; charset=UTF-8; name="0001-WIP-Prevent-overflows-for-BRIN-page-ranges.patch" Content-Disposition: attachment; filename="0001-WIP-Prevent-overflows-for-BRIN-page-ranges.patch" Content-Transfer-Encoding: base64 RnJvbSBjZjUzYzEwOWM3M2NmYTkyNjRkZjcxNzYzZTllYzU3MTJmMWMxZjdmIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBUb21hcyBWb25kcmEgPHRvbWFzQDJuZHF1YWRyYW50 LmNvbT4KRGF0ZTogVGh1LCA3IERlYyAyMDIzIDE1OjA3OjA0ICswMTAwClN1YmplY3Q6IFtQ QVRDSF0gV0lQOiBQcmV2ZW50IG92ZXJmbG93cyBmb3IgQlJJTiBwYWdlIHJhbmdlcwoKSW50 cm9kdWNlcyByZW1vdmUgYnJpbl9uZXh0X3JhbmdlIHRoYXQgY2hlY2tzIGlmIGJsa25vIG92 ZXJmbG93cywgYW5kCmluc3RlYWQgc2V0cyBpdCB0byBJbnZhbGlkQmxvY2tOdW1iZXIuCi0t LQogc3JjL2JhY2tlbmQvYWNjZXNzL2JyaW4vYnJpbi5jIHwgMTcgKysrKysrKysrKysrKysr LS0KIDEgZmlsZSBjaGFuZ2VkLCAxNSBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQoK ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2FjY2Vzcy9icmluL2JyaW4uYyBiL3NyYy9iYWNr ZW5kL2FjY2Vzcy9icmluL2JyaW4uYwppbmRleCAyM2YwODEzODliMi4uMTk1MjE0MmQwNTAg MTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL2FjY2Vzcy9icmluL2JyaW4uYworKysgYi9zcmMv YmFja2VuZC9hY2Nlc3MvYnJpbi9icmluLmMKQEAgLTIyNyw2ICsyMjcsNyBAQCBzdGF0aWMg Ym9vbCBhZGRfdmFsdWVzX3RvX3JhbmdlKFJlbGF0aW9uIGlkeFJlbCwgQnJpbkRlc2MgKmJk ZXNjLAogc3RhdGljIGJvb2wgY2hlY2tfbnVsbF9rZXlzKEJyaW5WYWx1ZXMgKmJ2YWwsIFNj YW5LZXkgKm51bGxrZXlzLCBpbnQgbm51bGxrZXlzKTsKIHN0YXRpYyB2b2lkIGJyaW5fZmls bF9lbXB0eV9yYW5nZXMoQnJpbkJ1aWxkU3RhdGUgKnN0YXRlLAogCQkJCQkJCQkgICBCbG9j a051bWJlciBwcmV2UmFuZ2UsIEJsb2NrTnVtYmVyIG1heFJhbmdlKTsKK3N0YXRpYyBCbG9j a051bWJlciBicmluX25leHRfcmFuZ2UoQnJpbkJ1aWxkU3RhdGUgKnN0YXRlLCBCbG9ja051 bWJlciBibGtubyk7CiAKIC8qIHBhcmFsbGVsIGluZGV4IGJ1aWxkcyAqLwogc3RhdGljIHZv aWQgX2JyaW5fYmVnaW5fcGFyYWxsZWwoQnJpbkJ1aWxkU3RhdGUgKmJ1aWxkc3RhdGUsIFJl bGF0aW9uIGhlYXAsIFJlbGF0aW9uIGluZGV4LApAQCAtMjkzNSw3ICsyOTM2LDcgQEAgYnJp bl9maWxsX2VtcHR5X3JhbmdlcyhCcmluQnVpbGRTdGF0ZSAqc3RhdGUsCiAJICogSWYgd2Ug YWxyZWFkeSBzdW1tYXJpemVkIHNvbWUgcmFuZ2VzLCB3ZSBuZWVkIHRvIHN0YXJ0IHdpdGgg dGhlIG5leHQKIAkgKiBvbmUuIE90aGVyd2lzZSBzdGFydCBmcm9tIHRoZSBmaXJzdCByYW5n ZSBvZiB0aGUgdGFibGUuCiAJICovCi0JYmxrbm8gPSAocHJldlJhbmdlID09IEludmFsaWRC bG9ja051bWJlcikgPyAwIDogKHByZXZSYW5nZSArIHN0YXRlLT5ic19wYWdlc1BlclJhbmdl KTsKKwlibGtubyA9IChwcmV2UmFuZ2UgPT0gSW52YWxpZEJsb2NrTnVtYmVyKSA/IDAgOiBi cmluX25leHRfcmFuZ2Uoc3RhdGUsIHByZXZSYW5nZSk7CiAKIAkvKiBHZW5lcmF0ZSBlbXB0 eSByYW5nZXMgdW50aWwgd2UgaGl0IHRoZSBuZXh0IG5vbi1lbXB0eSByYW5nZS4gKi8KIAl3 aGlsZSAoYmxrbm8gPCBuZXh0UmFuZ2UpCkBAIC0yOTQ4LDYgKzI5NDksMTggQEAgYnJpbl9m aWxsX2VtcHR5X3JhbmdlcyhCcmluQnVpbGRTdGF0ZSAqc3RhdGUsCiAJCQkJCSAgYmxrbm8s IHN0YXRlLT5ic19lbXB0eVR1cGxlLCBzdGF0ZS0+YnNfZW1wdHlUdXBsZUxlbik7CiAKIAkJ LyogdHJ5IG5leHQgcGFnZSByYW5nZSAqLwotCQlibGtubyArPSBzdGF0ZS0+YnNfcGFnZXNQ ZXJSYW5nZTsKKwkJYmxrbm8gPSBicmluX25leHRfcmFuZ2Uoc3RhdGUsIGJsa25vKTsKIAl9 CiB9CisKK3N0YXRpYyBCbG9ja051bWJlcgorYnJpbl9uZXh0X3JhbmdlKEJyaW5CdWlsZFN0 YXRlICpzdGF0ZSwgQmxvY2tOdW1iZXIgYmxrbm8pCit7CisJQmxvY2tOdW1iZXIgcmV0ID0g KGJsa25vICsgc3RhdGUtPmJzX3BhZ2VzUGVyUmFuZ2UpOworCisJLyogb3ZlcmZsb3cgKi8K KwlpZiAocmV0IDwgYmxrbm8pCisJCXJldCA9IEludmFsaWRCbG9ja051bWJlcjsKKworCXJl dHVybiByZXQ7Cit9Ci0tIAoyLjQxLjAKCg== --------------b33pcwSN8vzgHMPIN25AcNz0--