Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rZVwP-004q5e-1d for pgsql-hackers@arkaria.postgresql.org; Mon, 12 Feb 2024 13:01:45 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rZVwN-009eaq-NQ for pgsql-hackers@arkaria.postgresql.org; Mon, 12 Feb 2024 13:01:43 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rZVwN-009eai-AO for pgsql-hackers@lists.postgresql.org; Mon, 12 Feb 2024 13:01:43 +0000 Received: from mail-oi1-x22a.google.com ([2607:f8b0:4864:20::22a]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rZVwJ-006oNw-QK for pgsql-hackers@lists.postgresql.org; Mon, 12 Feb 2024 13:01:42 +0000 Received: by mail-oi1-x22a.google.com with SMTP id 5614622812f47-3bd4e6a7cb0so2337863b6e.3 for ; Mon, 12 Feb 2024 05:01:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707742897; x=1708347697; darn=lists.postgresql.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8XjswYx86+mVyxqtRPsVCtGfQSxQ+I7yy+ZmWdMAVqE=; b=SVC+m2+/BD/VWcqaf8MDu6IUlBpL7F9o8+zfbV84g7yHLHB+WjM7ItXT1FqWnWiK9x 1zN4wbfGUwLlb8eCzLazfdw72Jcm2Mwooszp2GjZ2TwJoQpLgdWsHgLpbTGCHQagZ85l O5WDdr9iIYzXuz4zGPJDGAK5ZNSMN1Vk3665ODfyEv908wT4r9LpbKVjCIvC4dJEmNIi aAkKbuRb+uL08iy/9Iv0B+8lhJ1TkcXVlWVEegzFm7O8rgAJ4lvKCwThEQdrRSl6r0M2 6iT6qQ1bT8HOdus/BmPKyaTmLDBDxCJSGT0E2xW6hfAKYTozKC4ebIgz6G+iVXqatl2x hOkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707742897; x=1708347697; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8XjswYx86+mVyxqtRPsVCtGfQSxQ+I7yy+ZmWdMAVqE=; b=oBDn5nJ71UOIhBnzY+u2b+1XhPdryoKCo4Q1aEnZLyek68dZxEGFDrs5nHPukGfRh0 lZdy1DzXIER4PXPh4n/xij8lT1SaB30pjCaYhT7JajrmbCufDxK/DxAjimeZDJx23KqV NrJ0Wkn+jEVZ3U0xerhJp/o6TiLOY+q7bDyhdoU7x3Fhb5b5Jfp4WM1i7RoFzNJoAzRC BXcFlW+xjAFe9lhwY/vtLk0EWhTQcE4vCIzpoCsX23MwJkfad5ZgiWc2S+CBt0hxPn5x 3dYcmKrTh3sDE2SwejK30mO8yjg5vEpN70G8gESk50ehNtrr2ZMUCcwL0P4/NxgHApgV 7xdw== X-Gm-Message-State: AOJu0Yw6sulIgAHLHso12hpMdZs0Ik7zXpCQfwSMEEvTl2Xudu8UtnAP cbqp0WbwvonDl+C9ebZNn0+dzwG2baqS0BoSw8Kz3pjUZ3IYju6XxbE22bblAyY8ixJBpT2hNEK Bbx1A5Xf41jgNqGsjCcCocc/xYGwaOVlKenA1Zg== X-Google-Smtp-Source: AGHT+IFFbsq1ZBJ1IXPo03MqErQ33A2Dj5hf931wuM2Gpquq7qriRGz9vgoyh7pAXXiXv1MgYUCegbSnuy3rufQ2c8Q= X-Received: by 2002:a05:6808:1803:b0:3bd:c805:b7b1 with SMTP id bh3-20020a056808180300b003bdc805b7b1mr6435886oib.59.1707742896819; Mon, 12 Feb 2024 05:01:36 -0800 (PST) MIME-Version: 1.0 References: <437ccbcf-8f80-2919-411d-a3af88becf6c@cybertec.at> <77f4bacbeb9d223f90b3a0bd55ef7d65407bfddd.camel@oopsware.de> <2f2e88bf1d44d06c1baf27bbf3db880f42a5cb87.camel@oopsware.de> In-Reply-To: <2f2e88bf1d44d06c1baf27bbf3db880f42a5cb87.camel@oopsware.de> From: jian he Date: Mon, 12 Feb 2024 21:00:00 +0800 Message-ID: Subject: Re: [PATCH] Add sortsupport for range types and btree_gist To: Bernd Helmle Cc: pgsql-hackers@lists.postgresql.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Fri, Feb 9, 2024 at 2:14=E2=80=AFAM Bernd Helmle = wrote: > > Am Mittwoch, dem 10.01.2024 um 22:18 +0800 schrieb jian he: > > > > I split the original author's patch into 2. > > 1. Add GiST sortsupport function for all the btree-gist module data > > types except anyrange data type (which actually does not in this > > module) > > 2. Add GiST sortsupport function for anyrange data type. > > > > > what I am confused: > > In fmgr.h > > > > /* > > * Support for cleaning up detoasted copies of inputs. This must > > only > > * be used for pass-by-ref datatypes, and normally would only be used > > * for toastable types. If the given pointer is different from the > > * original argument, assume it's a palloc'd detoasted copy, and > > pfree it. > > * NOTE: most functions on toastable types do not have to worry about > > this, > > * but we currently require that support functions for indexes not > > leak > > * memory. > > */ > > #define PG_FREE_IF_COPY(ptr,n) \ > > do { \ > > if ((Pointer) (ptr) !=3D PG_GETARG_POINTER(n)) \ > > pfree(ptr); \ > > } while (0) > > > > but the doc > > (https://www.postgresql.org/docs/current/gist-extensibility.html) > > says: > > All the GiST support methods are normally called in short-lived > > memory > > contexts; that is, CurrentMemoryContext will get reset after each > > tuple is processed. It is therefore not very important to worry about > > pfree'ing everything you palloc. > > ENDOF_QUOTE > > > > so I am not sure in range_gist_cmp, we need the following: > > ` > > if ((Datum) range_a !=3D a) > > pfree(range_a); > > if ((Datum) range_b !=3D b) > > pfree(range_b); > > ` > > Turns out this is not true for sortsupport: the comparison function is > called for each tuple during sorting, which will leak the detoasted > (and probably copied datum) in the sort memory context. See the same > for e.g. numeric and text, which i needed to change to pass the key > values correctly to the text_cmp() or numeric_cmp() function in these > cases. > + + Per default btree_gist builts GiST indexe with + sortsupport in sorted mode. This usually results in a + much better index quality and smaller index sizes by much faster index built speed. It is still + possible to revert to buffered built strategy by using the buffering parameter + when creating the index. + + I believe `built` |`builts` should be `build`. Also maybe we can simply copy some texts from https://www.postgresql.org/docs/current/gist-implementation.html. how about the following: The sorted method is only available if each of the opclasses used by the index provides a sortsupport function, as described in . If they do, this method is usually the best, so it is used by default. It is still possible to change to a buffered build strategy by using the buffering parameter to the CREATE INDEX command. you've changed contrib/btree_gist/meson.build, seems we also need to change contrib/btree_gist/Makefile gist_point_sortsupport have `if (ssup->abbreviate)`, does range_gist_sortsupport also this part? I think the `if(ssup->abbreviate)` part is optional? Can we add some comments on it?