Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h0koa-0006OE-9P for pgsql-hackers@arkaria.postgresql.org; Mon, 04 Mar 2019 10:27:20 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1h0koY-0008CD-3Z for pgsql-hackers@arkaria.postgresql.org; Mon, 04 Mar 2019 10:27:18 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h0koX-0008An-P0 for pgsql-hackers@lists.postgresql.org; Mon, 04 Mar 2019 10:27:17 +0000 Received: from mail-lf1-f44.google.com ([209.85.167.44]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h0koP-0000Sj-Be for pgsql-hackers@postgresql.org; Mon, 04 Mar 2019 10:27:16 +0000 Received: by mail-lf1-f44.google.com with SMTP id p73so2512237lfe.10 for ; Mon, 04 Mar 2019 02:27:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=42On3G+Ytrh/VIislaHna/8MKma/vFYeE6BXbVL53PQ=; b=AIuzJ4AtxRgmTcXmvJmauNJLPnlwS9fCHuBHBHfmvIZiUtkFqpTJICbeAOIzG1S8rx UfvLDR4bzfE7KvLpHJ4sYkAyEX2DzkaCZ3S9WlNe2nHBxpRCewrT+HFbOdW7s3TJrXok 7jKjoOXJwpZfCZxEQfsMvQlZGq7kxUh8x3J880HQ78zdxzG0CPW3iqb5GPGjrz7oOK8g H7kyJayhCKJNgLlFD20Fot4/6b/vIBfWp7769UuncJSE3VZx0ZJfURc6CqO0MBbP0zkA GhoBM51zLPeLuO456udwNyjt96WpNcNoZYtb83Soxvensghqo6G6nOQ1iZYg6SSi+Skw Npsg== X-Gm-Message-State: APjAAAWPf3OY+WThILNT1t0fg7sxZBSYcyg0nJ+lRglKlMRk9aN5u88u g1avgnMAj5IEq+F1IiICv/M= X-Google-Smtp-Source: APXvYqxHdOea8iUMVI8BShW8Hhl1lpUNChFxs2kdmdY2cKhPHa0ihAW2wcZnIgoYgU62r77XPoSi+A== X-Received: by 2002:ac2:52ae:: with SMTP id r14mr9056124lfm.66.1551695225587; Mon, 04 Mar 2019 02:27:05 -0800 (PST) Received: from [192.168.1.10] (dsl-hkibng32-54fbf8-78.dhcp.inet.fi. [84.251.248.78]) by smtp.gmail.com with ESMTPSA id y2sm1444652lji.46.2019.03.04.02.27.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Mar 2019 02:27:04 -0800 (PST) Subject: Re: GiST VACUUM To: Andrey Borodin Cc: Michael Paquier , Thomas Munro , Robert Haas , pgsql-hackers , =?UTF-8?B?0JrQvtGB0YLRjyDQmtGD0LfQvdC10YbQvtCy?= References: <479336FE-B713-4DFB-9661-9BFCCA70F4EF@yandex-team.ru> <51675201-3F1C-4796-ADB4-28D0B35E9CF0@yandex-team.ru> <651572F0-65C3-4285-85FC-50E81B1199F1@yandex-team.ru> <2AF4B7F1-ABCF-4602-8AF8-06EA45191BFC@yandex-team.ru> <844f3373-486e-a34d-d5a1-6a048e19355c@iki.fi> <4A1FF988-A69D-4607-8A6E-B3EC0FD6C2CF@yandex-team.ru> <43f15ab5-dd95-97d4-658b-e9954ca986a6@iki.fi> <20181002011404.GU11712@paquier.xyz> From: Heikki Linnakangas Message-ID: Date: Mon, 4 Mar 2019 12:27:04 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 04/01/2019 21:26, Andrey Borodin wrote: > 3 янв. 2019 г., в 23:47, Andrey Borodin > написал(а): >> >>> * Bitmapset stores 32 bit signed integers, but BlockNumber is >>> unsigned. So this will fail with an index larger than 2^31 >>> blocks. That's perhaps academical, I don't think anyone will try >>> to create a 16 TB GiST index any time soon. But it feels wrong to >>> introduce an arbitrary limitation like that. >> >> Looks like bitmapset is unsuitable for collecting block numbers due >> to the interface. Let's just create custom container for this? Or >> there is one other option: for each block number throw away sign >> bit and consider page potentially internal and potentially empty >> leaf if (blkno & 0x7FFFFFF) is in bitmapset. And then we will have >> to create at least one 17Tb GiST to check it actually works. > > Heikki, how do you think, is implementing our own radix tree for this > is viable solution? I've written working implementation with 4-level > statically typed tree. If we follow this route, probably, there must > be tests for this data structure. Yeah, seems reasonable. - Heikki