public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 6/6] Ignore correlation for new BRIN opclasses
38+ messages / 15 participants
[nested] [flat]
* [PATCH 6/6] Ignore correlation for new BRIN opclasses
@ 2020-09-12 13:07 Tomas Vondra <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Tomas Vondra @ 2020-09-12 13:07 UTC (permalink / raw)
The new BRIN opclasses (bloom and minmax-multi) are less sensitive to
poorly correlated data, so just assume the data is perfectly correlated
during costing.
---
src/backend/access/brin/brin_bloom.c | 1 +
src/backend/access/brin/brin_minmax_multi.c | 1 +
src/backend/utils/adt/selfuncs.c | 19 ++++++++++++++++++-
src/include/access/brin_internal.h | 3 +++
4 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c
index c2cbbd9400..03e9d4b713 100644
--- a/src/backend/access/brin/brin_bloom.c
+++ b/src/backend/access/brin/brin_bloom.c
@@ -681,6 +681,7 @@ brin_bloom_opcinfo(PG_FUNCTION_ARGS)
result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) +
sizeof(BloomOpaque));
+ result->oi_ignore_correlation = true;
result->oi_nstored = 1;
result->oi_regular_nulls = true;
result->oi_opaque = (BloomOpaque *)
diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c
index 227927fcf5..c8f36b0c8a 100644
--- a/src/backend/access/brin/brin_minmax_multi.c
+++ b/src/backend/access/brin/brin_minmax_multi.c
@@ -1306,6 +1306,7 @@ brin_minmax_multi_opcinfo(PG_FUNCTION_ARGS)
result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) +
sizeof(MinmaxMultiOpaque));
+ result->oi_ignore_correlation = true;
result->oi_nstored = 1;
result->oi_regular_nulls = true;
result->oi_opaque = (MinmaxMultiOpaque *)
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 00c7afc66f..c00265a66d 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -98,6 +98,7 @@
#include <math.h>
#include "access/brin.h"
+#include "access/brin_internal.h"
#include "access/brin_page.h"
#include "access/gin.h"
#include "access/table.h"
@@ -7350,7 +7351,8 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
double minimalRanges;
double estimatedRanges;
double selec;
- Relation indexRel;
+ Relation indexRel = NULL;
+ TupleDesc tupdesc = NULL;
ListCell *l;
VariableStatData vardata;
@@ -7372,6 +7374,7 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
*/
indexRel = index_open(index->indexoid, NoLock);
brinGetStats(indexRel, &statsData);
+ tupdesc = RelationGetDescr(indexRel);
index_close(indexRel, NoLock);
/* work out the actual number of ranges in the index */
@@ -7405,6 +7408,17 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
{
IndexClause *iclause = lfirst_node(IndexClause, l);
AttrNumber attnum = index->indexkeys[iclause->indexcol];
+ FmgrInfo *opcInfoFn;
+ BrinOpcInfo *opcInfo;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, iclause->indexcol);
+ bool ignore_correlation;
+
+ opcInfoFn = index_getprocinfo(indexRel, iclause->indexcol + 1, BRIN_PROCNUM_OPCINFO);
+
+ opcInfo = (BrinOpcInfo *)
+ DatumGetPointer(FunctionCall1(opcInfoFn, attr->atttypid));
+
+ ignore_correlation = opcInfo->oi_ignore_correlation;
/* attempt to lookup stats in relation for this index column */
if (attnum != 0)
@@ -7475,6 +7489,9 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
if (sslot.nnumbers > 0)
varCorrelation = Abs(sslot.numbers[0]);
+ if (ignore_correlation)
+ varCorrelation = 1.0;
+
if (varCorrelation > *indexCorrelation)
*indexCorrelation = varCorrelation;
diff --git a/src/include/access/brin_internal.h b/src/include/access/brin_internal.h
index ee4d0706df..67aea62a02 100644
--- a/src/include/access/brin_internal.h
+++ b/src/include/access/brin_internal.h
@@ -30,6 +30,9 @@ typedef struct BrinOpcInfo
/* Regular processing of NULLs in BrinValues? */
bool oi_regular_nulls;
+ /* Ignore correlation during cost estimation */
+ bool oi_ignore_correlation;
+
/* Opaque pointer for the opclass' private use */
void *oi_opaque;
--
2.25.4
--7kiiugxvbzvpnhzt--
^ permalink raw reply [nested|flat] 38+ messages in thread
* documentation structure
@ 2024-03-18 14:11 Robert Haas <[email protected]>
0 siblings, 4 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-18 14:11 UTC (permalink / raw)
To: pgsql-hackers
I was looking at the documentation index this morning[1], and I can't
help feeling like there are some parts of it that are over-emphasized
and some parts that are under-emphasized. I'm not sure what we can do
about this exactly, but I thought it worth writing an email and seeing
what other people think.
The two sections of the documentation that seem really
under-emphasized to me are the GUC documentation and the SQL
reference. The GUC documentation is all buried under "20. Server
Configuration" and the SQL command reference is under "I. SQL
commands". For reasons that I don't understand, all chapters except
for those in "VI. Reference" are numbered, but the chapters in that
section have Roman numerals instead.
I don't know what other people's experience is, but for me, wanting to
know what a command does or what a setting does is extremely common.
Therefore, I think these chapters are disproportionately important and
should be emphasized more. In the case of the GUC reference, one idea
I have is to split up "III. Server Administration". My proposal is
that we divide it into three sections. The first would be called "III.
Server Installation" and would cover chapters 16 (installation from
binaries) through 19 (server setup and operation). The second would be
called "IV. Server Configuration" -- so every section that's currently
a subsection of "server configuration" would become a top-level
chapter. The third division would be "V. Server Administration," and
would cover the current chapters 21-33. This is probably far from
perfect, but it seems like a relatively simple change and better than
what we have now.
I don't know what to do about "I. SQL commands". It's obviously
impractical to promote that to a top-level section, because it's got a
zillion sub-pages which I don't think we want in the top-level
documentation index. But having it as one of several unnumbered
chapters interposed between 51 and 52 doesn't seem great either.
The stuff that I think is over-emphasized is as follows: (a) chapters
1-3, the tutorial; (b) chapters 4-6, which are essentially a
continuation of the tutorial, and not at all similar to chapters 8-11
which are chalk-full of detailed technical information; (c) chapters
43-46, one per procedural language; perhaps these could just be
demoted to sub-sections of chapter 42 on procedural languages; (d)
chapters 47 (server programming interface), 50 (replication progress
tracking), and 51 (archive modules), all of which are important to
document but none of which seem important enough to put them in the
top-level documentation index; and (e) large parts of section "VII.
Internals," which again contain tons of stuff of very marginal
interest. The first ~4 chapters of the internals section seem like
they might be mainstream enough to justify the level of prominence
that we give them, but the rest has got to be of interest to a tiny
minority of readers.
I think it might be possible to consolidate the internals section by
grouping a bunch of existing entries together by category. Basically,
after the first few chapters, you've got stuff that is of interest to
C programmers writing core or extension code; and you've got
explainers on things like GEQO and index op-classes and support
functions which might be of interest even to non-programmers. I think
for example that we don't need separate top-level chapters on writing
procedural language handlers, FDWs, tablesample methods, custom scan
providers, table access methods, index access methods, and WAL
resource managers. Some or all of those could be grouped under a
single chapter, perhaps, e.g. Using PostgreSQL Extensibility
Interfaces.
Thoughts? I realize that this topic is HIGHLY prone to ENDLESS
bikeshedding, and it's inevitable that not everybody is going to
agree. But I hope we can agree that it's completely silly that it's
vastly easier to find the documentation about the backup manifest
format than it is to find the documentation on CREATE TABLE or
shared_buffers, and if we can agree on that, then perhaps we can agree
on some way to make things better.
--
Robert Haas
EDB: http://www.enterprisedb.com
[1] https://www.postgresql.org/docs/16/index.html
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-18 14:54 Matthias van de Meent <[email protected]>
parent: Robert Haas <[email protected]>
3 siblings, 1 reply; 38+ messages in thread
From: Matthias van de Meent @ 2024-03-18 14:54 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
On Mon, 18 Mar 2024 at 15:12, Robert Haas <[email protected]> wrote:
I'm not going into detail about the other docs comments, I don't have
much of an opinion either way on the mentioned sections. You make good
arguments; yet I don't usually use those sections of the docs but
rather do code searches.
> I don't know what to do about "I. SQL commands". It's obviously
> impractical to promote that to a top-level section, because it's got a
> zillion sub-pages which I don't think we want in the top-level
> documentation index. But having it as one of several unnumbered
> chapters interposed between 51 and 52 doesn't seem great either.
Could "SQL Commands" be a top-level construct, with subsections for
SQL/DML, SQL/DDL, SQL/Transaction management, and PG's
extensions/administrative/misc features? I sometimes find myself
trying to mentally organize what SQL commands users can use vs those
accessible to database owners and administrators, which is not
currently organized as such in the SQL Commands section.
Kind regards,
Matthias van de Meent
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-18 17:21 Robert Haas <[email protected]>
parent: Matthias van de Meent <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-18 17:21 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; +Cc: pgsql-hackers
On Mon, Mar 18, 2024 at 10:55 AM Matthias van de Meent
<[email protected]> wrote:
> > I don't know what to do about "I. SQL commands". It's obviously
> > impractical to promote that to a top-level section, because it's got a
> > zillion sub-pages which I don't think we want in the top-level
> > documentation index. But having it as one of several unnumbered
> > chapters interposed between 51 and 52 doesn't seem great either.
>
> Could "SQL Commands" be a top-level construct, with subsections for
> SQL/DML, SQL/DDL, SQL/Transaction management, and PG's
> extensions/administrative/misc features? I sometimes find myself
> trying to mentally organize what SQL commands users can use vs those
> accessible to database owners and administrators, which is not
> currently organized as such in the SQL Commands section.
Yeah, I wondered about that, too. Or for example you could group all
CREATE commands together, all ALTER commands together, all DROP
commands together, etc. But I can't really see a future in such
schemes, because having a single page that links to the reference
documentation for every single command we have in alphabetical order
is incredibly handy, or at least I have found it so. So my feeling -
at least at present - is that it's more fruitful to look into cutting
down the amount of clutter that appears in the top-level documentation
index, and maybe finding ways to make important sections like the SQL
reference more prominent.
Given how much documentation we have, it's just not going to be
possible to make everything that matters conveniently visible at the
top level. I think if people have to click down a level for the SQL
reference, that's fine, as long as the link they need to click on is
reasonably visible. What annoys me about the present structure is that
it isn't. You don't get any visual clue that the "SQL Commands" page
with ~100 subpages is more important than "51. Archive Modules" or
"33. Regression Tests" or "58. Writing a Procedural Language Handler,"
but it totally is.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-18 19:06 Roberto Mello <[email protected]>
parent: Robert Haas <[email protected]>
3 siblings, 0 replies; 38+ messages in thread
From: Roberto Mello @ 2024-03-18 19:06 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
On Mon, Mar 18, 2024 at 10:12 AM Robert Haas <[email protected]> wrote:
> I was looking at the documentation index this morning[1], and I can't
> help feeling like there are some parts of it that are over-emphasized
> and some parts that are under-emphasized. I'm not sure what we can do
> about this exactly, but I thought it worth writing an email and seeing
> what other people think.
>
I agree, and my usage patterns of the docs are similar.
As the project progresses and more features are added and tacked on to
existing docs, things can get
murky or buried. I imagine that web access and search logs could paint a
picture of documentation usage.
I don't know what other people's experience is, but for me, wanting to
> know what a command does or what a setting does is extremely common.
> Therefore, I think these chapters are disproportionately important and
> should be emphasized more. In the case of the GUC reference, one idea
>
+1
I have is to split up "III. Server Administration". My proposal is
> that we divide it into three sections. The first would be called "III.
> Server Installation" and would cover chapters 16 (installation from
> binaries) through 19 (server setup and operation). The second would be
> called "IV. Server Configuration" -- so every section that's currently
> a subsection of "server configuration" would become a top-level
>
chapter. The third division would be "V. Server Administration," and
> would cover the current chapters 21-33. This is probably far from
I like all of those.
> I don't know what to do about "I. SQL commands". It's obviously
> impractical to promote that to a top-level section, because it's got a
> zillion sub-pages which I don't think we want in the top-level
> documentation index. But having it as one of several unnumbered
> chapters interposed between 51 and 52 doesn't seem great either.
>
I think it'd be easier to read if current "VI. Reference" came right after
"Server Administration",
ahead of "Client Interfaces" and "Server Programming", which are of
interest to a much smaller
subset of users.
Also if the subchapters were numbered like the rest of them. I don't think
the roman numerals are
particularly helpful.
The stuff that I think is over-emphasized is as follows: (a) chapters
> 1-3, the tutorial; (b) chapters 4-6, which are essentially a
...
Also +1
Thoughts? I realize that this topic is HIGHLY prone to ENDLESS
> bikeshedding, and it's inevitable that not everybody is going to
> agree. But I hope we can agree that it's completely silly that it's
> vastly easier to find the documentation about the backup manifest
> format than it is to find the documentation on CREATE TABLE or
> shared_buffers, and if we can agree on that, then perhaps we can agree
> on some way to make things better.
>
Impossible to please everyone, but I'm sure we can improve things.
I've contributed to different parts of the docs over the years, and would
be happy
to help with this work.
Roberto
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-18 21:40 Laurenz Albe <[email protected]>
parent: Robert Haas <[email protected]>
3 siblings, 3 replies; 38+ messages in thread
From: Laurenz Albe @ 2024-03-18 21:40 UTC (permalink / raw)
To: Robert Haas <[email protected]>; pgsql-hackers
On Mon, 2024-03-18 at 10:11 -0400, Robert Haas wrote:
> The two sections of the documentation that seem really
> under-emphasized to me are the GUC documentation and the SQL
> reference. The GUC documentation is all buried under "20. Server
> Configuration" and the SQL command reference is under "I. SQL
> commands". For reasons that I don't understand, all chapters except
> for those in "VI. Reference" are numbered, but the chapters in that
> section have Roman numerals instead.
That last fact is very odd indeed and could be easily fixed.
> I don't know what other people's experience is, but for me, wanting to
> know what a command does or what a setting does is extremely common.
> Therefore, I think these chapters are disproportionately important and
> should be emphasized more. In the case of the GUC reference, one idea
> I have is to split up "III. Server Administration". My proposal is
> that we divide it into three sections. The first would be called "III.
> Server Installation" and would cover chapters 16 (installation from
> binaries) through 19 (server setup and operation). The second would be
> called "IV. Server Configuration" -- so every section that's currently
> a subsection of "server configuration" would become a top-level
> chapter. The third division would be "V. Server Administration," and
> would cover the current chapters 21-33. This is probably far from
> perfect, but it seems like a relatively simple change and better than
> what we have now.
I'm fine with splitting up "Server Administration" into three sections
like you propose.
> I don't know what to do about "I. SQL commands". It's obviously
> impractical to promote that to a top-level section, because it's got a
> zillion sub-pages which I don't think we want in the top-level
> documentation index. But having it as one of several unnumbered
> chapters interposed between 51 and 52 doesn't seem great either.
I think that both the GUCs and the SQL reference could be top-level
sections. For the GUCs there is an obvious split in sub-chapters,
and the SQL reference could be a top-level section without any chapters
under it.
> The stuff that I think is over-emphasized is as follows: (a) chapters
> 1-3, the tutorial; (b) chapters 4-6, which are essentially a
> continuation of the tutorial, and not at all similar to chapters 8-11
> which are chalk-full of detailed technical information; (c) chapters
> 43-46, one per procedural language; perhaps these could just be
> demoted to sub-sections of chapter 42 on procedural languages; (d)
> chapters 47 (server programming interface), 50 (replication progress
> tracking), and 51 (archive modules), all of which are important to
> document but none of which seem important enough to put them in the
> top-level documentation index; and (e) large parts of section "VII.
> Internals," which again contain tons of stuff of very marginal
> interest. The first ~4 chapters of the internals section seem like
> they might be mainstream enough to justify the level of prominence
> that we give them, but the rest has got to be of interest to a tiny
> minority of readers.
I disagree that the tutorial is over-emphasized.
I also disagree that chapters 4 to 6 are a continuation of the tutorial.
Or at least, they shouldn't be.
When I am looking for a documentation reference on something like
security considerations of SECURITY DEFINER functions, my first
impulse is to look in chapter 5 (Data Definition) or in chapter 38
(Extending SQL), and I am surprised to find it discussed in the
SQL reference of CREATE FUNCTION.
Another case in point is the "Notes" section for CREATE VIEW. Why is
that not somewhere under "Data Definition"?
For me, the reference should be terse and focused on the syntax.
Changing that is probably a lost cause by now, but I feel that we need
not encourage that development any more by playing down the earlier
chapters.
> I think it might be possible to consolidate the internals section by
> grouping a bunch of existing entries together by category. Basically,
> after the first few chapters, you've got stuff that is of interest to
> C programmers writing core or extension code; and you've got
> explainers on things like GEQO and index op-classes and support
> functions which might be of interest even to non-programmers. I think
> for example that we don't need separate top-level chapters on writing
> procedural language handlers, FDWs, tablesample methods, custom scan
> providers, table access methods, index access methods, and WAL
> resource managers. Some or all of those could be grouped under a
> single chapter, perhaps, e.g. Using PostgreSQL Extensibility
> Interfaces.
I have no strong feelings about that.
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-18 22:51 Tom Lane <[email protected]>
parent: Laurenz Albe <[email protected]>
2 siblings, 1 reply; 38+ messages in thread
From: Tom Lane @ 2024-03-18 22:51 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; +Cc: Robert Haas <[email protected]>; pgsql-hackers
Laurenz Albe <[email protected]> writes:
> On Mon, 2024-03-18 at 10:11 -0400, Robert Haas wrote:
>> I don't know what to do about "I. SQL commands". It's obviously
>> impractical to promote that to a top-level section, because it's got a
>> zillion sub-pages which I don't think we want in the top-level
>> documentation index. But having it as one of several unnumbered
>> chapters interposed between 51 and 52 doesn't seem great either.
> I think that both the GUCs and the SQL reference could be top-level
> sections. For the GUCs there is an obvious split in sub-chapters,
> and the SQL reference could be a top-level section without any chapters
> under it.
I'd be in favor of promoting all three of the "Reference" things to
the top level, except that as Robert says, it seems likely that that
would end in having a hundred individual command reference pages
visible in the topmost table of contents. Also, if we manage to
suppress that, did we really make it any more prominent? Not sure.
Making "SQL commands" top-level with half a dozen subsections would
solve the visibility problem, but I'm not real eager to go there,
because I foresee endless arguments about which subsection a given
command goes in. Robert's point about wanting a single alphabetized
list is valid too (although you could imagine that being a list in an
introductory section, similar to what we have for system catalogs).
This might be a silly suggestion, but: could we just render the
"most important" chapter titles in a larger font?
regards, tom lane
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-18 23:34 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-18 23:34 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Laurenz Albe <[email protected]>; pgsql-hackers
On Mon, Mar 18, 2024 at 6:51 PM Tom Lane <[email protected]> wrote:
> This might be a silly suggestion, but: could we just render the
> "most important" chapter titles in a larger font?
It's not the silliest suggestion ever -- you could have proposed
<blink>! -- but I also suspect it's not the right answer. Of course,
varying the font size can be a great way of emphasizing some things
more than others, but it doesn't usually work out well to just take a
document that was designed to be displayed in a uniform font size and
enlarge bits of text here and there. You usually want to have some
kind of overall plan of which font size is a single component.
For example, on a corporate home page, it's quite common to have two
nav bars, the larger of which has entries that correspond to the
company's product offerings and/or marketing materials, and the
smaller of which has "utility functions" like "login", "contact us",
and "search". Font size can be an effective tool for emphasizing the
relative importance of one nav bar versus the other, but you don't
start by deciding which things are going to get displayed in a larger
font. You start with an overall idea of the layout and then the font
size flows out of that.
Just riffing a bit, you could imagine adding a nav bar to our
documentation, either across the top or along the side, that is always
there on every page of the documentation and contains those links that
we want to make sure are always visible. Necessarily, these must be
limited in number. Then on the home page you could have the whole
table of contents as we do today, and you use that to navigate to
everything that isn't one of the quick links.
Or you can imagine that the home page of our documentation isn't just
a tree view like it is today; it might instead be written in paragraph
form. "Welcome to the PostgreSQL documentation! If you're new here,
check out our <link>tutorial</link>! Otherwise, you might be
interested in our <link>SQL reference</link>, our <link>configuration
reference</link>, or our <link>banana plantation</link>. If none of
those sound like what you want, check out the <link>documentation
index</link>." Obviously in order to actually work, something like
this would need to be expanded into enough paragraphs to actually
cover all of the important sections of the documentation, and probably
not mention banana plantations. Or maybe it wouldn't be just
paragraphs, but a two-column table, with each row of the table having
a main title and link in the narrower lefthand column and a blurb with
more links in the wider righthand column.
I'm sure there are a lot of other ways to do this, too. Our main
documentation page is very old-school, and there are probably a bunch
of ways to do better.
But I'm not sure how easy it would be to get agreement on something
specific, and I don't know how well our toolchain can support anything
other than what we've already got. I've also learned from painful
experience that you can't fix bad content with good markup. I think it
is worth spending some effort on trying to beat the existing format
into submission, promoting things that seem to deserve it and demoting
those that seem to deserve that. At some point, we'll probably reach a
point of diminishing returns, either because we all agree we've done
as well as we can, or because we can't agree on what else to do, and
maybe at that point the only way to improve further is with better web
design and/or a different documentation toolchain. But I think it's
fairly clear that we're not at that point now.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-19 09:05 Daniel Gustafsson <[email protected]>
parent: Laurenz Albe <[email protected]>
2 siblings, 1 reply; 38+ messages in thread
From: Daniel Gustafsson @ 2024-03-19 09:05 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; +Cc: Robert Haas <[email protected]>; pgsql-hackers
> On 18 Mar 2024, at 22:40, Laurenz Albe <[email protected]> wrote:
> On Mon, 2024-03-18 at 10:11 -0400, Robert Haas wrote:
>> For reasons that I don't understand, all chapters except
>> for those in "VI. Reference" are numbered, but the chapters in that
>> section have Roman numerals instead.
>
> That last fact is very odd indeed and could be easily fixed.
It's actually not very odd, the reference section is using <reference> elements
and we had missed the arabic numerals setting on those. The attached fixes
that for me. That being said, we've had roman numerals for the reference
section since forever (all the way down to the 7.2 docs online has it) so maybe
it was intentional? Or no one managed to see it until Robert did, I've
certainly never noticed it until now.
--
Daniel Gustafsson
Attachments:
[application/octet-stream] reference-autolabel.diff (672B, ../../[email protected]/2-reference-autolabel.diff)
download | inline diff:
diff --git a/doc/src/sgml/stylesheet-common.xsl b/doc/src/sgml/stylesheet-common.xsl
index a2e3db67a3..319c92e8ab 100644
--- a/doc/src/sgml/stylesheet-common.xsl
+++ b/doc/src/sgml/stylesheet-common.xsl
@@ -35,6 +35,7 @@
<xsl:param name="section.label.includes.component.label" select="1"></xsl:param>
<xsl:param name="refentry.generate.name" select="0"></xsl:param>
<xsl:param name="refentry.generate.title" select="1"></xsl:param>
+<xsl:param name="reference.autolabel" select="1"></xsl:param>
<xsl:param name="refentry.xref.manvolnum" select="0"/>
<xsl:param name="formal.procedures" select="0"></xsl:param>
<xsl:param name="generate.consistent.ids" select="1"/>
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-19 13:50 Tom Lane <[email protected]>
parent: Daniel Gustafsson <[email protected]>
0 siblings, 1 reply; 38+ messages in thread
From: Tom Lane @ 2024-03-19 13:50 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
Daniel Gustafsson <[email protected]> writes:
> It's actually not very odd, the reference section is using <reference> elements
> and we had missed the arabic numerals setting on those. The attached fixes
> that for me. That being said, we've had roman numerals for the reference
> section since forever (all the way down to the 7.2 docs online has it) so maybe
> it was intentional?
I'm quite sure it *was* intentional. Maybe it was a bad idea, but
it's not that way simply because nobody thought about it.
regards, tom lane
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-19 21:39 Andrew Dunstan <[email protected]>
parent: Robert Haas <[email protected]>
3 siblings, 3 replies; 38+ messages in thread
From: Andrew Dunstan @ 2024-03-19 21:39 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
On Mon, Mar 18, 2024 at 10:12 AM Robert Haas <[email protected]> wrote:
> I was looking at the documentation index this morning[1], and I can't
> help feeling like there are some parts of it that are over-emphasized
> and some parts that are under-emphasized. I'm not sure what we can do
> about this exactly, but I thought it worth writing an email and seeing
> what other people think.
>
> The two sections of the documentation that seem really
> under-emphasized to me are the GUC documentation and the SQL
> reference. The GUC documentation is all buried under "20. Server
> Configuration" and the SQL command reference is under "I. SQL
> commands". For reasons that I don't understand, all chapters except
> for those in "VI. Reference" are numbered, but the chapters in that
> section have Roman numerals instead.
>
> I don't know what other people's experience is, but for me, wanting to
> know what a command does or what a setting does is extremely common.
> Therefore, I think these chapters are disproportionately important and
> should be emphasized more. In the case of the GUC reference, one idea
> I have is to split up "III. Server Administration". My proposal is
> that we divide it into three sections. The first would be called "III.
> Server Installation" and would cover chapters 16 (installation from
> binaries) through 19 (server setup and operation). The second would be
> called "IV. Server Configuration" -- so every section that's currently
> a subsection of "server configuration" would become a top-level
> chapter. The third division would be "V. Server Administration," and
> would cover the current chapters 21-33. This is probably far from
> perfect, but it seems like a relatively simple change and better than
> what we have now.
>
> I don't know what to do about "I. SQL commands". It's obviously
> impractical to promote that to a top-level section, because it's got a
> zillion sub-pages which I don't think we want in the top-level
> documentation index. But having it as one of several unnumbered
> chapters interposed between 51 and 52 doesn't seem great either.
>
> The stuff that I think is over-emphasized is as follows: (a) chapters
> 1-3, the tutorial; (b) chapters 4-6, which are essentially a
> continuation of the tutorial, and not at all similar to chapters 8-11
> which are chalk-full of detailed technical information; (c) chapters
> 43-46, one per procedural language; perhaps these could just be
> demoted to sub-sections of chapter 42 on procedural languages; (d)
> chapters 47 (server programming interface), 50 (replication progress
> tracking), and 51 (archive modules), all of which are important to
> document but none of which seem important enough to put them in the
> top-level documentation index; and (e) large parts of section "VII.
> Internals," which again contain tons of stuff of very marginal
> interest. The first ~4 chapters of the internals section seem like
> they might be mainstream enough to justify the level of prominence
> that we give them, but the rest has got to be of interest to a tiny
> minority of readers.
>
> I think it might be possible to consolidate the internals section by
> grouping a bunch of existing entries together by category. Basically,
> after the first few chapters, you've got stuff that is of interest to
> C programmers writing core or extension code; and you've got
> explainers on things like GEQO and index op-classes and support
> functions which might be of interest even to non-programmers. I think
> for example that we don't need separate top-level chapters on writing
> procedural language handlers, FDWs, tablesample methods, custom scan
> providers, table access methods, index access methods, and WAL
> resource managers. Some or all of those could be grouped under a
> single chapter, perhaps, e.g. Using PostgreSQL Extensibility
> Interfaces.
>
> Thoughts? I realize that this topic is HIGHLY prone to ENDLESS
> bikeshedding, and it's inevitable that not everybody is going to
> agree. But I hope we can agree that it's completely silly that it's
> vastly easier to find the documentation about the backup manifest
> format than it is to find the documentation on CREATE TABLE or
> shared_buffers, and if we can agree on that, then perhaps we can agree
> on some way to make things better.
>
>
>
+many for improving the index.
My own pet docs peeve is a purely editorial one: func.sgml is a 30k line
beast, and I think there's a good case for splitting out at least the
larger chunks of it.
cheers
andrew
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-20 14:32 Robert Haas <[email protected]>
parent: Laurenz Albe <[email protected]>
2 siblings, 0 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-20 14:32 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; +Cc: pgsql-hackers
On Mon, Mar 18, 2024 at 5:40 PM Laurenz Albe <[email protected]> wrote:
> I also disagree that chapters 4 to 6 are a continuation of the tutorial.
> Or at least, they shouldn't be.
> When I am looking for a documentation reference on something like
> security considerations of SECURITY DEFINER functions, my first
> impulse is to look in chapter 5 (Data Definition) or in chapter 38
> (Extending SQL), and I am surprised to find it discussed in the
> SQL reference of CREATE FUNCTION.
I looked at this a bit more closely. There's actually a lot of
detailed technical information in chapters 4 and 5, but chapter 6 is
extremely short and mostly recapitulates chapter 2.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-20 16:43 Robert Haas <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 3 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-20 16:43 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: pgsql-hackers
On Tue, Mar 19, 2024 at 5:39 PM Andrew Dunstan <[email protected]> wrote:
> +many for improving the index.
Here's a series of four patches. Taken together, they cut down the
number of numbered chapters from 76 to 68. I think we could easily
save that much again if I wrote a few more patches along similar
lines, but I'm posting these first to see what people think.
0001 removes the "Installation from Binaries" chapter. The whole thing
is four sentences. I moved the most important information into the
"Installation from Source Code" chapter and retitled it
"Installation".
0002 removes the "Monitoring Disk Usage" chapter by folding it into
the immediately-preceding "Monitoring Database Activity" chapter. I
kind of feel like the "Monitoring Disk Usage" chapter might be in need
of a bigger rewrite or just outright removal, but there's surely not
enough content here to justify making it a top-level chapter.
0003 merges all of the "Internals" chapters whose names are the names
of built-in index access methods (Btree, Gin, etc.) into a single
chapter called "Built-In Index Access Methods". All of these chapters
have a very similar structure and none of them are very long, so it
makes a lot of sense, at least in my mind, to consolidate them into
one.
0004 merges the "Generic WAL Records" and "Custom WAL Resource
Managers" chapter together, creating a new chapter called "Write Ahead
Logging for Extensions".
Overall, I think this achieves a minor but pleasant level of
de-cluttering of the index. It's going to take a lot more than one
morning's work to produce a major improvement, but at least this is
something.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v1-0001-docs-Remove-the-Installation-from-Binaries-chapte.patch (4.2K, ../../CA+TgmobvXwu+iJmBg3hncdjY0-agg3giP-aLi_KhDyrL9+OLew@mail.gmail.com/2-v1-0001-docs-Remove-the-Installation-from-Binaries-chapte.patch)
download | inline diff:
From ec645d3429006b0f7c693a6d65f1dc196264e0a1 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 20 Mar 2024 10:08:05 -0400
Subject: [PATCH v1 1/4] docs: Remove the "Installation from Binaries" chapter.
The entire chapter was four sentences. Move the most useful
information from those sentences to the "Installation from Source
Code" chapter, and rename that to just "Installation".
---
doc/src/sgml/filelist.sgml | 1 -
doc/src/sgml/install-binaries.sgml | 24 ------------------------
doc/src/sgml/installation.sgml | 23 +++++++++++++++++------
doc/src/sgml/postgres.sgml | 1 -
4 files changed, 17 insertions(+), 32 deletions(-)
delete mode 100644 doc/src/sgml/install-binaries.sgml
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index e0dca81cb2..b7d1222e3e 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -36,7 +36,6 @@
<!ENTITY client-auth SYSTEM "client-auth.sgml">
<!ENTITY diskusage SYSTEM "diskusage.sgml">
<!ENTITY high-availability SYSTEM "high-availability.sgml">
-<!ENTITY installbin SYSTEM "install-binaries.sgml">
<!ENTITY installation SYSTEM "installation.sgml">
<!ENTITY targets-meson SYSTEM "targets-meson.sgml">
<!ENTITY maintenance SYSTEM "maintenance.sgml">
diff --git a/doc/src/sgml/install-binaries.sgml b/doc/src/sgml/install-binaries.sgml
deleted file mode 100644
index 001c3c7be0..0000000000
--- a/doc/src/sgml/install-binaries.sgml
+++ /dev/null
@@ -1,24 +0,0 @@
-<!-- doc/src/sgml/install-binaries.sgml -->
-<chapter id="install-binaries">
- <title>Installation from Binaries</title>
-
- <indexterm>
- <primary>installation</primary>
- <secondary>binaries</secondary>
- </indexterm>
-
- <para>
- <productname>PostgreSQL</productname> is available in the form of binary
- packages for most common operating systems today. When available, this is
- the recommended way to install PostgreSQL for users of the system. Building
- from source (see <xref linkend="installation" />) is only recommended for
- people developing <productname>PostgreSQL</productname> or extensions.
- </para>
-
- <para>
- For an updated list of platforms providing binary packages, please visit
- the download section on the <productname>PostgreSQL</productname> website at
- <ulink url="https://www.postgresql.org/download/"></ulink> and follow the
- instructions for the specific platform.
- </para>
-</chapter>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index a453f804cd..7b2e6786f4 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1,18 +1,29 @@
<!-- doc/src/sgml/installation.sgml -->
<chapter id="installation">
- <title>Installation from Source Code</title>
+ <title>Installation</title>
<indexterm zone="installation">
<primary>installation</primary>
</indexterm>
<para>
- This chapter describes the installation of
- <productname>PostgreSQL</productname> using the source code
- distribution. If you are installing a pre-packaged distribution,
- such as an RPM or Debian package, ignore this chapter
- and see <xref linkend="install-binaries" /> instead.
+ Since <productname>PostgreSQL</productname> is available in the form of
+ binary packages for most common operating systems today, it is typically not
+ necessary to build from source.
+ </para>
+
+ <para>
+ For an updated list of platforms providing binary packages, please visit
+ the download section on the <productname>PostgreSQL</productname> website at
+ <ulink url="https://www.postgresql.org/download/"></ulink> and follow the
+ instructions for the specific platform.
+ </para>
+
+ <para>
+ If you wish to compile from source code, for example to develop
+ <productname>PostgreSQL</productname> or an extension, the following
+ sections explain how to do this.
</para>
<sect1 id="install-requirements">
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 2c107199d3..7c234ff1db 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -150,7 +150,6 @@ break is not needed in a wider output rendering.
</para>
</partintro>
- &installbin;
&installation;
&runtime;
&config;
--
2.39.3 (Apple Git-145)
[application/octet-stream] v1-0002-docs-Demote-Monitoring-Disk-Usage-from-chapter-to.patch (11.7K, ../../CA+TgmobvXwu+iJmBg3hncdjY0-agg3giP-aLi_KhDyrL9+OLew@mail.gmail.com/3-v1-0002-docs-Demote-Monitoring-Disk-Usage-from-chapter-to.patch)
download | inline diff:
From 756789a3b8f3b8ae609851e18f365bb5863f6f7a Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 20 Mar 2024 10:52:16 -0400
Subject: [PATCH v1 2/4] docs: Demote "Monitoring Disk Usage" from chapter to
section.
This chapter is very short, and the immediately preceding chapter is
called "Monitoring Database Activity". So, instead of having a
separate chapter for this, make it the last section of the preceding
chapter instead.
---
doc/src/sgml/diskusage.sgml | 144 -----------------------------------
doc/src/sgml/filelist.sgml | 1 -
doc/src/sgml/monitoring.sgml | 143 ++++++++++++++++++++++++++++++++++
doc/src/sgml/postgres.sgml | 1 -
4 files changed, 143 insertions(+), 146 deletions(-)
delete mode 100644 doc/src/sgml/diskusage.sgml
diff --git a/doc/src/sgml/diskusage.sgml b/doc/src/sgml/diskusage.sgml
deleted file mode 100644
index 75467582e4..0000000000
--- a/doc/src/sgml/diskusage.sgml
+++ /dev/null
@@ -1,144 +0,0 @@
-<!-- doc/src/sgml/diskusage.sgml -->
-
-<chapter id="diskusage">
- <title>Monitoring Disk Usage</title>
-
- <para>
- This chapter discusses how to monitor the disk usage of a
- <productname>PostgreSQL</productname> database system.
- </para>
-
- <sect1 id="disk-usage">
- <title>Determining Disk Usage</title>
-
- <indexterm zone="disk-usage">
- <primary>disk usage</primary>
- </indexterm>
-
- <para>
- Each table has a primary heap disk file where most of the data is
- stored. If the table has any columns with potentially-wide values,
- there also might be a <acronym>TOAST</acronym> file associated with the table,
- which is used to store values too wide to fit comfortably in the main
- table (see <xref linkend="storage-toast"/>). There will be one valid index
- on the <acronym>TOAST</acronym> table, if present. There also might be indexes
- associated with the base table. Each table and index is stored in a
- separate disk file — possibly more than one file, if the file would
- exceed one gigabyte. Naming conventions for these files are described
- in <xref linkend="storage-file-layout"/>.
- </para>
-
- <para>
- You can monitor disk space in three ways:
- using the SQL functions listed in <xref linkend="functions-admin-dbsize"/>,
- using the <xref linkend="oid2name"/> module, or
- using manual inspection of the system catalogs.
- The SQL functions are the easiest to use and are generally recommended.
- The remainder of this section shows how to do it by inspection of the
- system catalogs.
- </para>
-
- <para>
- Using <application>psql</application> on a recently vacuumed or analyzed database,
- you can issue queries to see the disk usage of any table:
-<programlisting>
-SELECT pg_relation_filepath(oid), relpages FROM pg_class WHERE relname = 'customer';
-
- pg_relation_filepath | relpages
-----------------------+----------
- base/16384/16806 | 60
-(1 row)
-</programlisting>
- Each page is typically 8 kilobytes. (Remember, <structfield>relpages</structfield>
- is only updated by <command>VACUUM</command>, <command>ANALYZE</command>, and
- a few DDL commands such as <command>CREATE INDEX</command>.) The file path name
- is of interest if you want to examine the table's disk file directly.
- </para>
-
- <para>
- To show the space used by <acronym>TOAST</acronym> tables, use a query
- like the following:
-<programlisting>
-SELECT relname, relpages
-FROM pg_class,
- (SELECT reltoastrelid
- FROM pg_class
- WHERE relname = 'customer') AS ss
-WHERE oid = ss.reltoastrelid OR
- oid = (SELECT indexrelid
- FROM pg_index
- WHERE indrelid = ss.reltoastrelid)
-ORDER BY relname;
-
- relname | relpages
-----------------------+----------
- pg_toast_16806 | 0
- pg_toast_16806_index | 1
-</programlisting>
- </para>
-
- <para>
- You can easily display index sizes, too:
-<programlisting>
-SELECT c2.relname, c2.relpages
-FROM pg_class c, pg_class c2, pg_index i
-WHERE c.relname = 'customer' AND
- c.oid = i.indrelid AND
- c2.oid = i.indexrelid
-ORDER BY c2.relname;
-
- relname | relpages
--------------------+----------
- customer_id_index | 26
-</programlisting>
- </para>
-
- <para>
- It is easy to find your largest tables and indexes using this
- information:
-<programlisting>
-SELECT relname, relpages
-FROM pg_class
-ORDER BY relpages DESC;
-
- relname | relpages
-----------------------+----------
- bigtable | 3290
- customer | 3144
-</programlisting>
- </para>
- </sect1>
-
- <sect1 id="disk-full">
- <title>Disk Full Failure</title>
-
- <para>
- The most important disk monitoring task of a database administrator
- is to make sure the disk doesn't become full. A filled data disk will
- not result in data corruption, but it might prevent useful activity
- from occurring. If the disk holding the WAL files grows full, database
- server panic and consequent shutdown might occur.
- </para>
-
- <para>
- If you cannot free up additional space on the disk by deleting
- other things, you can move some of the database files to other file
- systems by making use of tablespaces. See <xref
- linkend="manage-ag-tablespaces"/> for more information about that.
- </para>
-
- <tip>
- <para>
- Some file systems perform badly when they are almost full, so do
- not wait until the disk is completely full to take action.
- </para>
- </tip>
-
- <para>
- If your system supports per-user disk quotas, then the database
- will naturally be subject to whatever quota is placed on the user
- the server runs as. Exceeding the quota will have the same bad
- effects as running out of disk space entirely.
- </para>
- </sect1>
-</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index b7d1222e3e..f39b4efe38 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -34,7 +34,6 @@
<!ENTITY backup SYSTEM "backup.sgml">
<!ENTITY charset SYSTEM "charset.sgml">
<!ENTITY client-auth SYSTEM "client-auth.sgml">
-<!ENTITY diskusage SYSTEM "diskusage.sgml">
<!ENTITY high-availability SYSTEM "high-availability.sgml">
<!ENTITY installation SYSTEM "installation.sgml">
<!ENTITY targets-meson SYSTEM "targets-meson.sgml">
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 8736eac284..eda54b4985 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -7282,4 +7282,147 @@ if (TRACE_POSTGRESQL_TRANSACTION_START_ENABLED())
</sect1>
+ <sect1 id="diskusage">
+ <title>Monitoring Disk Usage</title>
+
+ <para>
+ This section discusses how to monitor the disk usage of a
+ <productname>PostgreSQL</productname> database system.
+ </para>
+
+ <sect2 id="disk-usage">
+ <title>Determining Disk Usage</title>
+
+ <indexterm zone="disk-usage">
+ <primary>disk usage</primary>
+ </indexterm>
+
+ <para>
+ Each table has a primary heap disk file where most of the data is
+ stored. If the table has any columns with potentially-wide values,
+ there also might be a <acronym>TOAST</acronym> file associated with the table,
+ which is used to store values too wide to fit comfortably in the main
+ table (see <xref linkend="storage-toast"/>). There will be one valid index
+ on the <acronym>TOAST</acronym> table, if present. There also might be indexes
+ associated with the base table. Each table and index is stored in a
+ separate disk file — possibly more than one file, if the file would
+ exceed one gigabyte. Naming conventions for these files are described
+ in <xref linkend="storage-file-layout"/>.
+ </para>
+
+ <para>
+ You can monitor disk space in three ways:
+ using the SQL functions listed in <xref linkend="functions-admin-dbsize"/>,
+ using the <xref linkend="oid2name"/> module, or
+ using manual inspection of the system catalogs.
+ The SQL functions are the easiest to use and are generally recommended.
+ The remainder of this section shows how to do it by inspection of the
+ system catalogs.
+ </para>
+
+ <para>
+ Using <application>psql</application> on a recently vacuumed or analyzed
+ database, you can issue queries to see the disk usage of any table:
+<programlisting>
+SELECT pg_relation_filepath(oid), relpages FROM pg_class WHERE relname = 'customer';
+
+ pg_relation_filepath | relpages
+----------------------+----------
+ base/16384/16806 | 60
+(1 row)
+</programlisting>
+ Each page is typically 8 kilobytes. (Remember, <structfield>relpages</structfield>
+ is only updated by <command>VACUUM</command>, <command>ANALYZE</command>, and
+ a few DDL commands such as <command>CREATE INDEX</command>.) The file path name
+ is of interest if you want to examine the table's disk file directly.
+ </para>
+
+ <para>
+ To show the space used by <acronym>TOAST</acronym> tables, use a query
+ like the following:
+<programlisting>
+SELECT relname, relpages
+FROM pg_class,
+ (SELECT reltoastrelid
+ FROM pg_class
+ WHERE relname = 'customer') AS ss
+WHERE oid = ss.reltoastrelid OR
+ oid = (SELECT indexrelid
+ FROM pg_index
+ WHERE indrelid = ss.reltoastrelid)
+ORDER BY relname;
+
+ relname | relpages
+----------------------+----------
+ pg_toast_16806 | 0
+ pg_toast_16806_index | 1
+</programlisting>
+ </para>
+
+ <para>
+ You can easily display index sizes, too:
+<programlisting>
+SELECT c2.relname, c2.relpages
+FROM pg_class c, pg_class c2, pg_index i
+WHERE c.relname = 'customer' AND
+ c.oid = i.indrelid AND
+ c2.oid = i.indexrelid
+ORDER BY c2.relname;
+
+ relname | relpages
+-------------------+----------
+ customer_id_index | 26
+</programlisting>
+ </para>
+
+ <para>
+ It is easy to find your largest tables and indexes using this
+ information:
+<programlisting>
+SELECT relname, relpages
+FROM pg_class
+ORDER BY relpages DESC;
+
+ relname | relpages
+----------------------+----------
+ bigtable | 3290
+ customer | 3144
+</programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="disk-full">
+ <title>Disk Full Failure</title>
+
+ <para>
+ The most important disk monitoring task of a database administrator
+ is to make sure the disk doesn't become full. A filled data disk will
+ not result in data corruption, but it might prevent useful activity
+ from occurring. If the disk holding the WAL files grows full, database
+ server panic and consequent shutdown might occur.
+ </para>
+
+ <para>
+ If you cannot free up additional space on the disk by deleting
+ other things, you can move some of the database files to other file
+ systems by making use of tablespaces. See <xref
+ linkend="manage-ag-tablespaces"/> for more information about that.
+ </para>
+
+ <tip>
+ <para>
+ Some file systems perform badly when they are almost full, so do
+ not wait until the disk is completely full to take action.
+ </para>
+ </tip>
+
+ <para>
+ If your system supports per-user disk quotas, then the database
+ will naturally be subject to whatever quota is placed on the user
+ the server runs as. Exceeding the quota will have the same bad
+ effects as running out of disk space entirely.
+ </para>
+ </sect2>
+ </sect1>
+
</chapter>
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 7c234ff1db..73b497bcf8 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -161,7 +161,6 @@ break is not needed in a wider output rendering.
&backup;
&high-availability;
&monitoring;
- &diskusage;
&wal;
&logical-replication;
&jit;
--
2.39.3 (Apple Git-145)
[application/octet-stream] v1-0004-docs-Consolidate-into-new-WAL-for-Extensions-chap.patch (23.8K, ../../CA+TgmobvXwu+iJmBg3hncdjY0-agg3giP-aLi_KhDyrL9+OLew@mail.gmail.com/4-v1-0004-docs-Consolidate-into-new-WAL-for-Extensions-chap.patch)
download | inline diff:
From 5304db1a398bee08e540a6b638de4693a01e5dff Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 20 Mar 2024 12:12:48 -0400
Subject: [PATCH v1 4/4] docs: Consolidate into new "WAL for Extensions"
chapter.
Previously, we had consecutive, very short chapters called "Generic
WAL" and "Custom WAL Resource Managers," explaining different approaches
to the same problem. Merge them into a single chapter.
---
doc/src/sgml/custom-rmgr.sgml | 105 ----------
doc/src/sgml/filelist.sgml | 3 +-
doc/src/sgml/generic-wal.sgml | 174 ----------------
doc/src/sgml/postgres.sgml | 3 +-
doc/src/sgml/wal-for-extensions.sgml | 283 +++++++++++++++++++++++++++
5 files changed, 285 insertions(+), 283 deletions(-)
delete mode 100644 doc/src/sgml/custom-rmgr.sgml
delete mode 100644 doc/src/sgml/generic-wal.sgml
create mode 100644 doc/src/sgml/wal-for-extensions.sgml
diff --git a/doc/src/sgml/custom-rmgr.sgml b/doc/src/sgml/custom-rmgr.sgml
deleted file mode 100644
index 0d98229295..0000000000
--- a/doc/src/sgml/custom-rmgr.sgml
+++ /dev/null
@@ -1,105 +0,0 @@
-<!-- doc/src/sgml/custom-rmgr.sgml -->
-
-<chapter id="custom-rmgr">
- <title>Custom WAL Resource Managers</title>
-
- <para>
- This chapter explains the interface between the core
- <productname>PostgreSQL</productname> system and custom WAL resource
- managers, which enable extensions to integrate directly with the <link
- linkend="wal"><acronym>WAL</acronym></link>.
- </para>
- <para>
- An extension, especially a <link linkend="tableam">Table Access
- Method</link> or <link linkend="indexam">Index Access Method</link>, may
- need to use WAL for recovery, replication, and/or <link
- linkend="logicaldecoding">Logical Decoding</link>. Custom resource managers
- are a more flexible alternative to <link linkend="generic-wal">Generic
- WAL</link> (which does not support logical decoding), but more complex for
- an extension to implement.
- </para>
- <para>
- To create a new custom WAL resource manager, first define an
- <structname>RmgrData</structname> structure with implementations for the
- resource manager methods. Refer to
- <filename>src/backend/access/transam/README</filename> and
- <filename>src/include/access/xlog_internal.h</filename> in the
- <productname>PostgreSQL</productname> source.
-<programlisting>
-/*
- * Method table for resource managers.
- *
- * This struct must be kept in sync with the PG_RMGR definition in
- * rmgr.c.
- *
- * rm_identify must return a name for the record based on xl_info (without
- * reference to the rmid). For example, XLOG_BTREE_VACUUM would be named
- * "VACUUM". rm_desc can then be called to obtain additional detail for the
- * record, if available (e.g. the last block).
- *
- * rm_mask takes as input a page modified by the resource manager and masks
- * out bits that shouldn't be flagged by wal_consistency_checking.
- *
- * RmgrTable[] is indexed by RmgrId values (see rmgrlist.h). If rm_name is
- * NULL, the corresponding RmgrTable entry is considered invalid.
- */
-typedef struct RmgrData
-{
- const char *rm_name;
- void (*rm_redo) (XLogReaderState *record);
- void (*rm_desc) (StringInfo buf, XLogReaderState *record);
- const char *(*rm_identify) (uint8 info);
- void (*rm_startup) (void);
- void (*rm_cleanup) (void);
- void (*rm_mask) (char *pagedata, BlockNumber blkno);
- void (*rm_decode) (struct LogicalDecodingContext *ctx,
- struct XLogRecordBuffer *buf);
-} RmgrData;
-</programlisting>
- </para>
-
- <para>
- The <filename>src/test/modules/test_custom_rmgrs</filename> module
- contains a working example, which demonstrates usage of custom WAL
- resource managers.
- </para>
-
- <para>
- Then, register your new resource
- manager.
-
-<programlisting>
-/*
- * Register a new custom WAL resource manager.
- *
- * Resource manager IDs must be globally unique across all extensions. Refer
- * to https://wiki.postgresql.org/wiki/CustomWALResourceManagers to reserve a
- * unique RmgrId for your extension, to avoid conflicts with other extension
- * developers. During development, use RM_EXPERIMENTAL_ID to avoid needlessly
- * reserving a new ID.
- */
-extern void RegisterCustomRmgr(RmgrId rmid, const RmgrData *rmgr);
-</programlisting>
- <function>RegisterCustomRmgr</function> must be called from the
- extension module's <link linkend="xfunc-c-dynload">_PG_init</link> function.
- While developing a new extension, use <literal>RM_EXPERIMENTAL_ID</literal>
- for <parameter>rmid</parameter>. When you are ready to release the extension
- to users, reserve a new resource manager ID at the <ulink
- url="https://wiki.postgresql.org/wiki/CustomWALResourceManagers">Custom WAL
- Resource Manager</ulink> page.
- </para>
-
- <para>
- Place the extension module implementing the custom resource manager in <xref
- linkend="guc-shared-preload-libraries"/> so that it will be loaded early
- during <productname>PostgreSQL</productname> startup.
- </para>
- <note>
- <para>
- The extension must remain in <varname>shared_preload_libraries</varname>
- as long as any custom WAL records may exist in the system. Otherwise
- <productname>PostgreSQL</productname> will not be able to apply or decode
- the custom WAL records, which may prevent the server from starting.
- </para>
- </note>
-</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index de683e3400..6e5ff6fa69 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -100,8 +100,7 @@
<!ENTITY storage SYSTEM "storage.sgml">
<!ENTITY transaction SYSTEM "xact.sgml">
<!ENTITY tablesample-method SYSTEM "tablesample-method.sgml">
-<!ENTITY generic-wal SYSTEM "generic-wal.sgml">
-<!ENTITY custom-rmgr SYSTEM "custom-rmgr.sgml">
+<!ENTITY wal-for-extensions SYSTEM "wal-for-extensions.sgml">
<!ENTITY backup-manifest SYSTEM "backup-manifest.sgml">
<!-- contrib information -->
diff --git a/doc/src/sgml/generic-wal.sgml b/doc/src/sgml/generic-wal.sgml
deleted file mode 100644
index a028856d2e..0000000000
--- a/doc/src/sgml/generic-wal.sgml
+++ /dev/null
@@ -1,174 +0,0 @@
-<!-- doc/src/sgml/generic-wal.sgml -->
-
-<chapter id="generic-wal">
- <title>Generic WAL Records</title>
-
- <para>
- Although all built-in WAL-logged modules have their own types of WAL
- records, there is also a generic WAL record type, which describes changes
- to pages in a generic way. This is useful for extensions that provide
- custom access methods.
- </para>
-
- <para>
- In comparison with <link linkend="custom-rmgr">Custom WAL Resource
- Managers</link>, Generic WAL is simpler for an extension to implement and
- does not require the extension library to be loaded in order to apply the
- records.
- </para>
-
- <note>
- <para>
- Generic WAL records are ignored during <link
- linkend="logicaldecoding">Logical Decoding</link>. If logical decoding is
- required for your extension, consider a Custom WAL Resource Manager.
- </para>
- </note>
-
- <para>
- The API for constructing generic WAL records is defined in
- <filename>access/generic_xlog.h</filename> and implemented
- in <filename>access/transam/generic_xlog.c</filename>.
- </para>
-
- <para>
- To perform a WAL-logged data update using the generic WAL record
- facility, follow these steps:
-
- <orderedlist>
- <listitem>
- <para>
- <function>state = GenericXLogStart(relation)</function> — start
- construction of a generic WAL record for the given relation.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <function>page = GenericXLogRegisterBuffer(state, buffer, flags)</function>
- — register a buffer to be modified within the current generic WAL
- record. This function returns a pointer to a temporary copy of the
- buffer's page, where modifications should be made. (Do not modify the
- buffer's contents directly.) The third argument is a bit mask of flags
- applicable to the operation. Currently the only such flag is
- <literal>GENERIC_XLOG_FULL_IMAGE</literal>, which indicates that a full-page
- image rather than a delta update should be included in the WAL record.
- Typically this flag would be set if the page is new or has been
- rewritten completely.
- <function>GenericXLogRegisterBuffer</function> can be repeated if the
- WAL-logged action needs to modify multiple pages.
- </para>
- </listitem>
-
- <listitem>
- <para>
- Apply modifications to the page images obtained in the previous step.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <function>GenericXLogFinish(state)</function> — apply the changes to
- the buffers and emit the generic WAL record.
- </para>
- </listitem>
- </orderedlist>
- </para>
-
- <para>
- WAL record construction can be canceled between any of the above steps by
- calling <function>GenericXLogAbort(state)</function>. This will discard all
- changes to the page image copies.
- </para>
-
- <para>
- Please note the following points when using the generic WAL record
- facility:
-
- <itemizedlist>
- <listitem>
- <para>
- No direct modifications of buffers are allowed! All modifications must
- be done in copies acquired from <function>GenericXLogRegisterBuffer()</function>.
- In other words, code that makes generic WAL records should never call
- <function>BufferGetPage()</function> for itself. However, it remains the
- caller's responsibility to pin/unpin and lock/unlock the buffers at
- appropriate times. Exclusive lock must be held on each target buffer
- from before <function>GenericXLogRegisterBuffer()</function> until after
- <function>GenericXLogFinish()</function>.
- </para>
- </listitem>
-
- <listitem>
- <para>
- Registrations of buffers (step 2) and modifications of page images
- (step 3) can be mixed freely, i.e., both steps may be repeated in any
- sequence. Keep in mind that buffers should be registered in the same
- order in which locks are to be obtained on them during replay.
- </para>
- </listitem>
-
- <listitem>
- <para>
- The maximum number of buffers that can be registered for a generic WAL
- record is <literal>MAX_GENERIC_XLOG_PAGES</literal>. An error will be thrown
- if this limit is exceeded.
- </para>
- </listitem>
-
- <listitem>
- <para>
- Generic WAL assumes that the pages to be modified have standard
- layout, and in particular that there is no useful data between
- <structfield>pd_lower</structfield> and <structfield>pd_upper</structfield>.
- </para>
- </listitem>
-
- <listitem>
- <para>
- Since you are modifying copies of buffer
- pages, <function>GenericXLogStart()</function> does not start a critical
- section. Thus, you can safely do memory allocation, error throwing,
- etc. between <function>GenericXLogStart()</function> and
- <function>GenericXLogFinish()</function>. The only actual critical section is
- present inside <function>GenericXLogFinish()</function>. There is no need to
- worry about calling <function>GenericXLogAbort()</function> during an error
- exit, either.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <function>GenericXLogFinish()</function> takes care of marking buffers dirty
- and setting their LSNs. You do not need to do this explicitly.
- </para>
- </listitem>
-
- <listitem>
- <para>
- For unlogged relations, everything works the same except that no
- actual WAL record is emitted. Thus, you typically do not need to do
- any explicit checks for unlogged relations.
- </para>
- </listitem>
-
- <listitem>
- <para>
- The generic WAL redo function will acquire exclusive locks to buffers
- in the same order as they were registered. After redoing all changes,
- the locks will be released in the same order.
- </para>
- </listitem>
-
- <listitem>
- <para>
- If <literal>GENERIC_XLOG_FULL_IMAGE</literal> is not specified for a
- registered buffer, the generic WAL record contains a delta between
- the old and the new page images. This delta is based on byte-by-byte
- comparison. This is not very compact for the case of moving data
- within a page, and might be improved in the future.
- </para>
- </listitem>
- </itemizedlist>
- </para>
-</chapter>
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 0235c0e352..5bc47a9e71 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -255,8 +255,7 @@ break is not needed in a wider output rendering.
&geqo;
&tableam;
&indexam;
- &generic-wal;
- &custom-rmgr;
+ &wal-for-extensions;
&indextypes;
&storage;
&transaction;
diff --git a/doc/src/sgml/wal-for-extensions.sgml b/doc/src/sgml/wal-for-extensions.sgml
new file mode 100644
index 0000000000..cc233358dd
--- /dev/null
+++ b/doc/src/sgml/wal-for-extensions.sgml
@@ -0,0 +1,283 @@
+<!-- doc/src/sgml/wal-for-extensions.sgml -->
+
+<chapter id="wal-for-extensions">
+ <title>Write Ahead Logging for Extensions</title>
+
+ <sect1 id="generic-wal">
+ <title>Generic WAL Records</title>
+
+ <para>
+ Although all built-in WAL-logged modules have their own types of WAL
+ records, there is also a generic WAL record type, which describes changes
+ to pages in a generic way. This is useful for extensions that provide
+ custom access methods.
+ </para>
+
+ <para>
+ In comparison with <link linkend="custom-rmgr">Custom WAL Resource
+ Managers</link>, Generic WAL is simpler for an extension to implement and
+ does not require the extension library to be loaded in order to apply the
+ records.
+ </para>
+
+ <note>
+ <para>
+ Generic WAL records are ignored during <link
+ linkend="logicaldecoding">Logical Decoding</link>. If logical decoding is
+ required for your extension, consider a Custom WAL Resource Manager.
+ </para>
+ </note>
+
+ <para>
+ The API for constructing generic WAL records is defined in
+ <filename>access/generic_xlog.h</filename> and implemented
+ in <filename>access/transam/generic_xlog.c</filename>.
+ </para>
+
+ <para>
+ To perform a WAL-logged data update using the generic WAL record
+ facility, follow these steps:
+
+ <orderedlist>
+ <listitem>
+ <para>
+ <function>state = GenericXLogStart(relation)</function> — start
+ construction of a generic WAL record for the given relation.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <function>page = GenericXLogRegisterBuffer(state, buffer, flags)</function>
+ — register a buffer to be modified within the current generic WAL
+ record. This function returns a pointer to a temporary copy of the
+ buffer's page, where modifications should be made. (Do not modify the
+ buffer's contents directly.) The third argument is a bit mask of flags
+ applicable to the operation. Currently the only such flag is
+ <literal>GENERIC_XLOG_FULL_IMAGE</literal>, which indicates that a full-page
+ image rather than a delta update should be included in the WAL record.
+ Typically this flag would be set if the page is new or has been
+ rewritten completely.
+ <function>GenericXLogRegisterBuffer</function> can be repeated if the
+ WAL-logged action needs to modify multiple pages.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Apply modifications to the page images obtained in the previous step.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <function>GenericXLogFinish(state)</function> — apply the changes to
+ the buffers and emit the generic WAL record.
+ </para>
+ </listitem>
+ </orderedlist>
+ </para>
+
+ <para>
+ WAL record construction can be canceled between any of the above steps by
+ calling <function>GenericXLogAbort(state)</function>. This will discard all
+ changes to the page image copies.
+ </para>
+
+ <para>
+ Please note the following points when using the generic WAL record
+ facility:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ No direct modifications of buffers are allowed! All modifications must
+ be done in copies acquired from <function>GenericXLogRegisterBuffer()</function>.
+ In other words, code that makes generic WAL records should never call
+ <function>BufferGetPage()</function> for itself. However, it remains the
+ caller's responsibility to pin/unpin and lock/unlock the buffers at
+ appropriate times. Exclusive lock must be held on each target buffer
+ from before <function>GenericXLogRegisterBuffer()</function> until after
+ <function>GenericXLogFinish()</function>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Registrations of buffers (step 2) and modifications of page images
+ (step 3) can be mixed freely, i.e., both steps may be repeated in any
+ sequence. Keep in mind that buffers should be registered in the same
+ order in which locks are to be obtained on them during replay.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The maximum number of buffers that can be registered for a generic WAL
+ record is <literal>MAX_GENERIC_XLOG_PAGES</literal>. An error will be thrown
+ if this limit is exceeded.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Generic WAL assumes that the pages to be modified have standard
+ layout, and in particular that there is no useful data between
+ <structfield>pd_lower</structfield> and <structfield>pd_upper</structfield>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Since you are modifying copies of buffer
+ pages, <function>GenericXLogStart()</function> does not start a critical
+ section. Thus, you can safely do memory allocation, error throwing,
+ etc. between <function>GenericXLogStart()</function> and
+ <function>GenericXLogFinish()</function>. The only actual critical section is
+ present inside <function>GenericXLogFinish()</function>. There is no need to
+ worry about calling <function>GenericXLogAbort()</function> during an error
+ exit, either.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <function>GenericXLogFinish()</function> takes care of marking buffers dirty
+ and setting their LSNs. You do not need to do this explicitly.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ For unlogged relations, everything works the same except that no
+ actual WAL record is emitted. Thus, you typically do not need to do
+ any explicit checks for unlogged relations.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The generic WAL redo function will acquire exclusive locks to buffers
+ in the same order as they were registered. After redoing all changes,
+ the locks will be released in the same order.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ If <literal>GENERIC_XLOG_FULL_IMAGE</literal> is not specified for a
+ registered buffer, the generic WAL record contains a delta between
+ the old and the new page images. This delta is based on byte-by-byte
+ comparison. This is not very compact for the case of moving data
+ within a page, and might be improved in the future.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect1>
+
+ <sect1 id="custom-rmgr">
+ <title>Custom WAL Resource Managers</title>
+
+ <para>
+ This section explains the interface between the core
+ <productname>PostgreSQL</productname> system and custom WAL resource
+ managers, which enable extensions to integrate directly with the <link
+ linkend="wal"><acronym>WAL</acronym></link>.
+ </para>
+ <para>
+ An extension, especially a <link linkend="tableam">Table Access
+ Method</link> or <link linkend="indexam">Index Access Method</link>, may
+ need to use WAL for recovery, replication, and/or <link
+ linkend="logicaldecoding">Logical Decoding</link>. Custom resource managers
+ are a more flexible alternative to <link linkend="generic-wal">Generic
+ WAL</link> (which does not support logical decoding), but more complex for
+ an extension to implement.
+ </para>
+ <para>
+ To create a new custom WAL resource manager, first define an
+ <structname>RmgrData</structname> structure with implementations for the
+ resource manager methods. Refer to
+ <filename>src/backend/access/transam/README</filename> and
+ <filename>src/include/access/xlog_internal.h</filename> in the
+ <productname>PostgreSQL</productname> source.
+<programlisting>
+/*
+ * Method table for resource managers.
+ *
+ * This struct must be kept in sync with the PG_RMGR definition in
+ * rmgr.c.
+ *
+ * rm_identify must return a name for the record based on xl_info (without
+ * reference to the rmid). For example, XLOG_BTREE_VACUUM would be named
+ * "VACUUM". rm_desc can then be called to obtain additional detail for the
+ * record, if available (e.g. the last block).
+ *
+ * rm_mask takes as input a page modified by the resource manager and masks
+ * out bits that shouldn't be flagged by wal_consistency_checking.
+ *
+ * RmgrTable[] is indexed by RmgrId values (see rmgrlist.h). If rm_name is
+ * NULL, the corresponding RmgrTable entry is considered invalid.
+ */
+typedef struct RmgrData
+{
+ const char *rm_name;
+ void (*rm_redo) (XLogReaderState *record);
+ void (*rm_desc) (StringInfo buf, XLogReaderState *record);
+ const char *(*rm_identify) (uint8 info);
+ void (*rm_startup) (void);
+ void (*rm_cleanup) (void);
+ void (*rm_mask) (char *pagedata, BlockNumber blkno);
+ void (*rm_decode) (struct LogicalDecodingContext *ctx,
+ struct XLogRecordBuffer *buf);
+} RmgrData;
+</programlisting>
+ </para>
+
+ <para>
+ The <filename>src/test/modules/test_custom_rmgrs</filename> module
+ contains a working example, which demonstrates usage of custom WAL
+ resource managers.
+ </para>
+
+ <para>
+ Then, register your new resource
+ manager.
+
+<programlisting>
+/*
+ * Register a new custom WAL resource manager.
+ *
+ * Resource manager IDs must be globally unique across all extensions. Refer
+ * to https://wiki.postgresql.org/wiki/CustomWALResourceManagers to reserve a
+ * unique RmgrId for your extension, to avoid conflicts with other extension
+ * developers. During development, use RM_EXPERIMENTAL_ID to avoid needlessly
+ * reserving a new ID.
+ */
+extern void RegisterCustomRmgr(RmgrId rmid, const RmgrData *rmgr);
+</programlisting>
+ <function>RegisterCustomRmgr</function> must be called from the
+ extension module's <link linkend="xfunc-c-dynload">_PG_init</link> function.
+ While developing a new extension, use <literal>RM_EXPERIMENTAL_ID</literal>
+ for <parameter>rmid</parameter>. When you are ready to release the extension
+ to users, reserve a new resource manager ID at the <ulink
+ url="https://wiki.postgresql.org/wiki/CustomWALResourceManagers">Custom WAL
+ Resource Manager</ulink> page.
+ </para>
+
+ <para>
+ Place the extension module implementing the custom resource manager in <xref
+ linkend="guc-shared-preload-libraries"/> so that it will be loaded early
+ during <productname>PostgreSQL</productname> startup.
+ </para>
+ <note>
+ <para>
+ The extension must remain in <varname>shared_preload_libraries</varname>
+ as long as any custom WAL records may exist in the system. Otherwise
+ <productname>PostgreSQL</productname> will not be able to apply or decode
+ the custom WAL records, which may prevent the server from starting.
+ </para>
+ </note>
+ </sect1>
+
+</chapter>
--
2.39.3 (Apple Git-145)
[application/octet-stream] v1-0003-docs-Merge-separate-chapters-on-built-in-index-AM.patch (485.0K, ../../CA+TgmobvXwu+iJmBg3hncdjY0-agg3giP-aLi_KhDyrL9+OLew@mail.gmail.com/5-v1-0003-docs-Merge-separate-chapters-on-built-in-index-AM.patch)
download | inline diff:
From 322cf03dbb7aaf61109f89b44e03e6e9a8d6b3f8 Mon Sep 17 00:00:00 2001
From: Robert Haas <[email protected]>
Date: Wed, 20 Mar 2024 11:51:53 -0400
Subject: [PATCH v1 3/4] docs: Merge separate chapters on built-in index AMs
into one.
The documetation index is getting very long, which makes it hard
to find things. Since these chapters are all very similar in structure
and content, merging them is a natural way of reducing the size of
the toplevel index.
---
doc/src/sgml/brin.sgml | 1344 --------
doc/src/sgml/btree.sgml | 914 ------
doc/src/sgml/filelist.sgml | 7 +-
doc/src/sgml/gin.sgml | 714 -----
doc/src/sgml/gist.sgml | 1387 ---------
doc/src/sgml/hash.sgml | 162 -
doc/src/sgml/indextypes.sgml | 5597 ++++++++++++++++++++++++++++++++++
doc/src/sgml/postgres.sgml | 7 +-
doc/src/sgml/spgist.sgml | 1076 -------
9 files changed, 5599 insertions(+), 5609 deletions(-)
delete mode 100644 doc/src/sgml/brin.sgml
delete mode 100644 doc/src/sgml/btree.sgml
delete mode 100644 doc/src/sgml/gin.sgml
delete mode 100644 doc/src/sgml/gist.sgml
delete mode 100644 doc/src/sgml/hash.sgml
create mode 100644 doc/src/sgml/indextypes.sgml
delete mode 100644 doc/src/sgml/spgist.sgml
diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml
deleted file mode 100644
index d898cc4720..0000000000
--- a/doc/src/sgml/brin.sgml
+++ /dev/null
@@ -1,1344 +0,0 @@
-<!-- doc/src/sgml/brin.sgml -->
-
-<chapter id="brin">
-<title>BRIN Indexes</title>
-
- <indexterm>
- <primary>index</primary>
- <secondary>BRIN</secondary>
- </indexterm>
-
-<sect1 id="brin-intro">
- <title>Introduction</title>
-
- <para>
- <acronym>BRIN</acronym> stands for Block Range Index.
- <acronym>BRIN</acronym> is designed for handling very large tables
- in which certain columns have some natural correlation with their
- physical location within the table.
- </para>
-
- <para>
- <acronym>BRIN</acronym> works in terms of <firstterm>block ranges</firstterm>
- (or <quote>page ranges</quote>).
- A block range is a group of pages that are physically
- adjacent in the table; for each block range, some summary info is stored
- by the index.
- For example, a table storing a store's sale orders might have
- a date column on which each order was placed, and most of the time
- the entries for earlier orders will appear earlier in the table as well;
- a table storing a ZIP code column might have all codes for a city
- grouped together naturally.
- </para>
-
- <para>
- <acronym>BRIN</acronym> indexes can satisfy queries via regular bitmap
- index scans, and will return all tuples in all pages within each range if
- the summary info stored by the index is <firstterm>consistent</firstterm> with the
- query conditions.
- The query executor is in charge of rechecking these tuples and discarding
- those that do not match the query conditions — in other words, these
- indexes are lossy.
- Because a <acronym>BRIN</acronym> index is very small, scanning the index
- adds little overhead compared to a sequential scan, but may avoid scanning
- large parts of the table that are known not to contain matching tuples.
- </para>
-
- <para>
- The specific data that a <acronym>BRIN</acronym> index will store,
- as well as the specific queries that the index will be able to satisfy,
- depend on the operator class selected for each column of the index.
- Data types having a linear sort order can have operator classes that
- store the minimum and maximum value within each block range, for instance;
- geometrical types might store the bounding box for all the objects
- in the block range.
- </para>
-
- <para>
- The size of the block range is determined at index creation time by
- the <literal>pages_per_range</literal> storage parameter. The number of index
- entries will be equal to the size of the relation in pages divided by
- the selected value for <literal>pages_per_range</literal>. Therefore, the smaller
- the number, the larger the index becomes (because of the need to
- store more index entries), but at the same time the summary data stored can
- be more precise and more data blocks can be skipped during an index scan.
- </para>
-
- <sect2 id="brin-operation">
- <title>Index Maintenance</title>
-
- <para>
- At the time of creation, all existing heap pages are scanned and a
- summary index tuple is created for each range, including the
- possibly-incomplete range at the end.
- As new pages are filled with data, page ranges that are already
- summarized will cause the summary information to be updated with data
- from the new tuples.
- When a new page is created that does not fall within the last
- summarized range, the range that the new page belongs to
- does not automatically acquire a summary tuple;
- those tuples remain unsummarized until a summarization run is
- invoked later, creating the initial summary for that range.
- </para>
-
- <para>
- There are several ways to trigger the initial summarization of a page range.
- If the table is vacuumed, either manually or by
- <link linkend="autovacuum">autovacuum</link>, all existing unsummarized
- page ranges are summarized.
- Also, if the index's
- <xref linkend="index-reloption-autosummarize"/> parameter is enabled,
- which it isn't by default,
- whenever autovacuum runs in that database, summarization will occur for all
- unsummarized page ranges that have been filled,
- regardless of whether the table itself is processed by autovacuum; see below.
- </para>
-
- <para>
- Lastly, the following functions can be used (while these functions run,
- <xref linkend="guc-search-path"/> is temporarily changed to
- <literal>pg_catalog, pg_temp</literal>):
- <simplelist>
- <member>
- <function>brin_summarize_new_values(regclass)</function>
- which summarizes all unsummarized ranges;
- </member>
- <member>
- <function>brin_summarize_range(regclass, bigint)</function>
- which summarizes only the range containing the given page,
- if it is unsummarized.
- </member>
- </simplelist>
- </para>
-
- <para>
- When autosummarization is enabled, a request is sent to
- <literal>autovacuum</literal> to execute a targeted summarization
- for a block range when an insertion is detected for the first item
- of the first page of the next block range,
- to be fulfilled the next time an autovacuum
- worker finishes running in the
- same database. If the request queue is full, the request is not recorded
- and a message is sent to the server log:
-<screen>
-LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was not recorded
-</screen>
- When this happens, the range will remain unsummarized until the next
- regular vacuum run on the table, or one of the functions mentioned above
- are invoked.
- </para>
-
- <para>
- Conversely, a range can be de-summarized using the
- <function>brin_desummarize_range(regclass, bigint)</function> function,
- which is useful when the index tuple is no longer a very good
- representation because the existing values have changed.
- See <xref linkend="functions-admin-index"/> for details.
- </para>
-
- </sect2>
-</sect1>
-
-<sect1 id="brin-builtin-opclasses">
- <title>Built-in Operator Classes</title>
-
- <para>
- The core <productname>PostgreSQL</productname> distribution
- includes the <acronym>BRIN</acronym> operator classes shown in
- <xref linkend="brin-builtin-opclasses-table"/>.
- </para>
-
- <para>
- The <firstterm>minmax</firstterm>
- operator classes store the minimum and the maximum values appearing
- in the indexed column within the range. The <firstterm>inclusion</firstterm>
- operator classes store a value which includes the values in the indexed
- column within the range. The <firstterm>bloom</firstterm> operator
- classes build a Bloom filter for all values in the range. The
- <firstterm>minmax-multi</firstterm> operator classes store multiple
- minimum and maximum values, representing values appearing in the indexed
- column within the range.
- </para>
-
- <table id="brin-builtin-opclasses-table">
- <title>Built-in <acronym>BRIN</acronym> Operator Classes</title>
- <tgroup cols="2">
- <thead>
- <row>
- <entry>Name</entry>
- <entry>Indexable Operators</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry valign="middle" morerows="4"><literal>bit_minmax_ops</literal></entry>
- <entry><literal>= (bit,bit)</literal></entry>
- </row>
- <row><entry><literal>< (bit,bit)</literal></entry></row>
- <row><entry><literal>> (bit,bit)</literal></entry></row>
- <row><entry><literal><= (bit,bit)</literal></entry></row>
- <row><entry><literal>>= (bit,bit)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="12"><literal>box_inclusion_ops</literal></entry>
- <entry><literal>@> (box,point)</literal></entry>
- </row>
- <row><entry><literal><< (box,box)</literal></entry></row>
- <row><entry><literal>&< (box,box)</literal></entry></row>
- <row><entry><literal>&> (box,box)</literal></entry></row>
- <row><entry><literal>>> (box,box)</literal></entry></row>
- <row><entry><literal><@ (box,box)</literal></entry></row>
- <row><entry><literal>@> (box,box)</literal></entry></row>
- <row><entry><literal>~= (box,box)</literal></entry></row>
- <row><entry><literal>&& (box,box)</literal></entry></row>
- <row><entry><literal><<| (box,box)</literal></entry></row>
- <row><entry><literal>&<| (box,box)</literal></entry></row>
- <row><entry><literal>|&> (box,box)</literal></entry></row>
- <row><entry><literal>|>> (box,box)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>bpchar_bloom_ops</literal></entry>
- <entry><literal>= (character,character)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>bpchar_minmax_ops</literal></entry>
- <entry><literal>= (character,character)</literal></entry>
- </row>
- <row><entry><literal>< (character,character)</literal></entry></row>
- <row><entry><literal><= (character,character)</literal></entry></row>
- <row><entry><literal>> (character,character)</literal></entry></row>
- <row><entry><literal>>= (character,character)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>bytea_bloom_ops</literal></entry>
- <entry><literal>= (bytea,bytea)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>bytea_minmax_ops</literal></entry>
- <entry><literal>= (bytea,bytea)</literal></entry>
- </row>
- <row><entry><literal>< (bytea,bytea)</literal></entry></row>
- <row><entry><literal><= (bytea,bytea)</literal></entry></row>
- <row><entry><literal>> (bytea,bytea)</literal></entry></row>
- <row><entry><literal>>= (bytea,bytea)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>char_bloom_ops</literal></entry>
- <entry><literal>= ("char","char")</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>char_minmax_ops</literal></entry>
- <entry><literal>= ("char","char")</literal></entry>
- </row>
- <row><entry><literal>< ("char","char")</literal></entry></row>
- <row><entry><literal><= ("char","char")</literal></entry></row>
- <row><entry><literal>> ("char","char")</literal></entry></row>
- <row><entry><literal>>= ("char","char")</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>date_bloom_ops</literal></entry>
- <entry><literal>= (date,date)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>date_minmax_ops</literal></entry>
- <entry><literal>= (date,date)</literal></entry>
- </row>
- <row><entry><literal>< (date,date)</literal></entry></row>
- <row><entry><literal><= (date,date)</literal></entry></row>
- <row><entry><literal>> (date,date)</literal></entry></row>
- <row><entry><literal>>= (date,date)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>date_minmax_multi_ops</literal></entry>
- <entry><literal>= (date,date)</literal></entry>
- </row>
- <row><entry><literal>< (date,date)</literal></entry></row>
- <row><entry><literal><= (date,date)</literal></entry></row>
- <row><entry><literal>> (date,date)</literal></entry></row>
- <row><entry><literal>>= (date,date)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>float4_bloom_ops</literal></entry>
- <entry><literal>= (float4,float4)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>float4_minmax_ops</literal></entry>
- <entry><literal>= (float4,float4)</literal></entry>
- </row>
- <row><entry><literal>< (float4,float4)</literal></entry></row>
- <row><entry><literal>> (float4,float4)</literal></entry></row>
- <row><entry><literal><= (float4,float4)</literal></entry></row>
- <row><entry><literal>>= (float4,float4)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>float4_minmax_multi_ops</literal></entry>
- <entry><literal>= (float4,float4)</literal></entry>
- </row>
- <row><entry><literal>< (float4,float4)</literal></entry></row>
- <row><entry><literal>> (float4,float4)</literal></entry></row>
- <row><entry><literal><= (float4,float4)</literal></entry></row>
- <row><entry><literal>>= (float4,float4)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>float8_bloom_ops</literal></entry>
- <entry><literal>= (float8,float8)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>float8_minmax_ops</literal></entry>
- <entry><literal>= (float8,float8)</literal></entry>
- </row>
- <row><entry><literal>< (float8,float8)</literal></entry></row>
- <row><entry><literal><= (float8,float8)</literal></entry></row>
- <row><entry><literal>> (float8,float8)</literal></entry></row>
- <row><entry><literal>>= (float8,float8)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>float8_minmax_multi_ops</literal></entry>
- <entry><literal>= (float8,float8)</literal></entry>
- </row>
- <row><entry><literal>< (float8,float8)</literal></entry></row>
- <row><entry><literal><= (float8,float8)</literal></entry></row>
- <row><entry><literal>> (float8,float8)</literal></entry></row>
- <row><entry><literal>>= (float8,float8)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="5"><literal>inet_inclusion_ops</literal></entry>
- <entry><literal><< (inet,inet)</literal></entry>
- </row>
- <row><entry><literal><<= (inet,inet)</literal></entry></row>
- <row><entry><literal>>> (inet,inet)</literal></entry></row>
- <row><entry><literal>>>= (inet,inet)</literal></entry></row>
- <row><entry><literal>= (inet,inet)</literal></entry></row>
- <row><entry><literal>&& (inet,inet)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>inet_bloom_ops</literal></entry>
- <entry><literal>= (inet,inet)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>inet_minmax_ops</literal></entry>
- <entry><literal>= (inet,inet)</literal></entry>
- </row>
- <row><entry><literal>< (inet,inet)</literal></entry></row>
- <row><entry><literal><= (inet,inet)</literal></entry></row>
- <row><entry><literal>> (inet,inet)</literal></entry></row>
- <row><entry><literal>>= (inet,inet)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>inet_minmax_multi_ops</literal></entry>
- <entry><literal>= (inet,inet)</literal></entry>
- </row>
- <row><entry><literal>< (inet,inet)</literal></entry></row>
- <row><entry><literal><= (inet,inet)</literal></entry></row>
- <row><entry><literal>> (inet,inet)</literal></entry></row>
- <row><entry><literal>>= (inet,inet)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>int2_bloom_ops</literal></entry>
- <entry><literal>= (int2,int2)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>int2_minmax_ops</literal></entry>
- <entry><literal>= (int2,int2)</literal></entry>
- </row>
- <row><entry><literal>< (int2,int2)</literal></entry></row>
- <row><entry><literal>> (int2,int2)</literal></entry></row>
- <row><entry><literal><= (int2,int2)</literal></entry></row>
- <row><entry><literal>>= (int2,int2)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>int2_minmax_multi_ops</literal></entry>
- <entry><literal>= (int2,int2)</literal></entry>
- </row>
- <row><entry><literal>< (int2,int2)</literal></entry></row>
- <row><entry><literal>> (int2,int2)</literal></entry></row>
- <row><entry><literal><= (int2,int2)</literal></entry></row>
- <row><entry><literal>>= (int2,int2)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>int4_bloom_ops</literal></entry>
- <entry><literal>= (int4,int4)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>int4_minmax_ops</literal></entry>
- <entry><literal>= (int4,int4)</literal></entry>
- </row>
- <row><entry><literal>< (int4,int4)</literal></entry></row>
- <row><entry><literal>> (int4,int4)</literal></entry></row>
- <row><entry><literal><= (int4,int4)</literal></entry></row>
- <row><entry><literal>>= (int4,int4)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>int4_minmax_multi_ops</literal></entry>
- <entry><literal>= (int4,int4)</literal></entry>
- </row>
- <row><entry><literal>< (int4,int4)</literal></entry></row>
- <row><entry><literal>> (int4,int4)</literal></entry></row>
- <row><entry><literal><= (int4,int4)</literal></entry></row>
- <row><entry><literal>>= (int4,int4)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>int8_bloom_ops</literal></entry>
- <entry><literal>= (bigint,bigint)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>int8_minmax_ops</literal></entry>
- <entry><literal>= (bigint,bigint)</literal></entry>
- </row>
- <row><entry><literal>< (bigint,bigint)</literal></entry></row>
- <row><entry><literal>> (bigint,bigint)</literal></entry></row>
- <row><entry><literal><= (bigint,bigint)</literal></entry></row>
- <row><entry><literal>>= (bigint,bigint)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>int8_minmax_multi_ops</literal></entry>
- <entry><literal>= (bigint,bigint)</literal></entry>
- </row>
- <row><entry><literal>< (bigint,bigint)</literal></entry></row>
- <row><entry><literal>> (bigint,bigint)</literal></entry></row>
- <row><entry><literal><= (bigint,bigint)</literal></entry></row>
- <row><entry><literal>>= (bigint,bigint)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>interval_bloom_ops</literal></entry>
- <entry><literal>= (interval,interval)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>interval_minmax_ops</literal></entry>
- <entry><literal>= (interval,interval)</literal></entry>
- </row>
- <row><entry><literal>< (interval,interval)</literal></entry></row>
- <row><entry><literal><= (interval,interval)</literal></entry></row>
- <row><entry><literal>> (interval,interval)</literal></entry></row>
- <row><entry><literal>>= (interval,interval)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>interval_minmax_multi_ops</literal></entry>
- <entry><literal>= (interval,interval)</literal></entry>
- </row>
- <row><entry><literal>< (interval,interval)</literal></entry></row>
- <row><entry><literal><= (interval,interval)</literal></entry></row>
- <row><entry><literal>> (interval,interval)</literal></entry></row>
- <row><entry><literal>>= (interval,interval)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>macaddr_bloom_ops</literal></entry>
- <entry><literal>= (macaddr,macaddr)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>macaddr_minmax_ops</literal></entry>
- <entry><literal>= (macaddr,macaddr)</literal></entry>
- </row>
- <row><entry><literal>< (macaddr,macaddr)</literal></entry></row>
- <row><entry><literal><= (macaddr,macaddr)</literal></entry></row>
- <row><entry><literal>> (macaddr,macaddr)</literal></entry></row>
- <row><entry><literal>>= (macaddr,macaddr)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>macaddr_minmax_multi_ops</literal></entry>
- <entry><literal>= (macaddr,macaddr)</literal></entry>
- </row>
- <row><entry><literal>< (macaddr,macaddr)</literal></entry></row>
- <row><entry><literal><= (macaddr,macaddr)</literal></entry></row>
- <row><entry><literal>> (macaddr,macaddr)</literal></entry></row>
- <row><entry><literal>>= (macaddr,macaddr)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>macaddr8_bloom_ops</literal></entry>
- <entry><literal>= (macaddr8,macaddr8)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>macaddr8_minmax_ops</literal></entry>
- <entry><literal>= (macaddr8,macaddr8)</literal></entry>
- </row>
- <row><entry><literal>< (macaddr8,macaddr8)</literal></entry></row>
- <row><entry><literal><= (macaddr8,macaddr8)</literal></entry></row>
- <row><entry><literal>> (macaddr8,macaddr8)</literal></entry></row>
- <row><entry><literal>>= (macaddr8,macaddr8)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>macaddr8_minmax_multi_ops</literal></entry>
- <entry><literal>= (macaddr8,macaddr8)</literal></entry>
- </row>
- <row><entry><literal>< (macaddr8,macaddr8)</literal></entry></row>
- <row><entry><literal><= (macaddr8,macaddr8)</literal></entry></row>
- <row><entry><literal>> (macaddr8,macaddr8)</literal></entry></row>
- <row><entry><literal>>= (macaddr8,macaddr8)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>name_bloom_ops</literal></entry>
- <entry><literal>= (name,name)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>name_minmax_ops</literal></entry>
- <entry><literal>= (name,name)</literal></entry>
- </row>
- <row><entry><literal>< (name,name)</literal></entry></row>
- <row><entry><literal><= (name,name)</literal></entry></row>
- <row><entry><literal>> (name,name)</literal></entry></row>
- <row><entry><literal>>= (name,name)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>numeric_bloom_ops</literal></entry>
- <entry><literal>= (numeric,numeric)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>numeric_minmax_ops</literal></entry>
- <entry><literal>= (numeric,numeric)</literal></entry>
- </row>
- <row><entry><literal>< (numeric,numeric)</literal></entry></row>
- <row><entry><literal><= (numeric,numeric)</literal></entry></row>
- <row><entry><literal>> (numeric,numeric)</literal></entry></row>
- <row><entry><literal>>= (numeric,numeric)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>numeric_minmax_multi_ops</literal></entry>
- <entry><literal>= (numeric,numeric)</literal></entry>
- </row>
- <row><entry><literal>< (numeric,numeric)</literal></entry></row>
- <row><entry><literal><= (numeric,numeric)</literal></entry></row>
- <row><entry><literal>> (numeric,numeric)</literal></entry></row>
- <row><entry><literal>>= (numeric,numeric)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>oid_bloom_ops</literal></entry>
- <entry><literal>= (oid,oid)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>oid_minmax_ops</literal></entry>
- <entry><literal>= (oid,oid)</literal></entry>
- </row>
- <row><entry><literal>< (oid,oid)</literal></entry></row>
- <row><entry><literal>> (oid,oid)</literal></entry></row>
- <row><entry><literal><= (oid,oid)</literal></entry></row>
- <row><entry><literal>>= (oid,oid)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>oid_minmax_multi_ops</literal></entry>
- <entry><literal>= (oid,oid)</literal></entry>
- </row>
- <row><entry><literal>< (oid,oid)</literal></entry></row>
- <row><entry><literal>> (oid,oid)</literal></entry></row>
- <row><entry><literal><= (oid,oid)</literal></entry></row>
- <row><entry><literal>>= (oid,oid)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>pg_lsn_bloom_ops</literal></entry>
- <entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>pg_lsn_minmax_ops</literal></entry>
- <entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
- </row>
- <row><entry><literal>< (pg_lsn,pg_lsn)</literal></entry></row>
- <row><entry><literal>> (pg_lsn,pg_lsn)</literal></entry></row>
- <row><entry><literal><= (pg_lsn,pg_lsn)</literal></entry></row>
- <row><entry><literal>>= (pg_lsn,pg_lsn)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>pg_lsn_minmax_multi_ops</literal></entry>
- <entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
- </row>
- <row><entry><literal>< (pg_lsn,pg_lsn)</literal></entry></row>
- <row><entry><literal>> (pg_lsn,pg_lsn)</literal></entry></row>
- <row><entry><literal><= (pg_lsn,pg_lsn)</literal></entry></row>
- <row><entry><literal>>= (pg_lsn,pg_lsn)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="13"><literal>range_inclusion_ops</literal></entry>
- <entry><literal>= (anyrange,anyrange)</literal></entry>
- </row>
- <row><entry><literal>< (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal><= (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>>= (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>> (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>&& (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>@> (anyrange,anyelement)</literal></entry></row>
- <row><entry><literal>@> (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal><@ (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal><< (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>>> (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>&< (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>&> (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>-|- (anyrange,anyrange)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>text_bloom_ops</literal></entry>
- <entry><literal>= (text,text)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>text_minmax_ops</literal></entry>
- <entry><literal>= (text,text)</literal></entry>
- </row>
- <row><entry><literal>< (text,text)</literal></entry></row>
- <row><entry><literal><= (text,text)</literal></entry></row>
- <row><entry><literal>> (text,text)</literal></entry></row>
- <row><entry><literal>>= (text,text)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>tid_bloom_ops</literal></entry>
- <entry><literal>= (tid,tid)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>tid_minmax_ops</literal></entry>
- <entry><literal>= (tid,tid)</literal></entry>
- </row>
- <row><entry><literal>< (tid,tid)</literal></entry></row>
- <row><entry><literal>> (tid,tid)</literal></entry></row>
- <row><entry><literal><= (tid,tid)</literal></entry></row>
- <row><entry><literal>>= (tid,tid)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>tid_minmax_multi_ops</literal></entry>
- <entry><literal>= (tid,tid)</literal></entry>
- </row>
- <row><entry><literal>< (tid,tid)</literal></entry></row>
- <row><entry><literal>> (tid,tid)</literal></entry></row>
- <row><entry><literal><= (tid,tid)</literal></entry></row>
- <row><entry><literal>>= (tid,tid)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>timestamp_bloom_ops</literal></entry>
- <entry><literal>= (timestamp,timestamp)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>timestamp_minmax_ops</literal></entry>
- <entry><literal>= (timestamp,timestamp)</literal></entry>
- </row>
- <row><entry><literal>< (timestamp,timestamp)</literal></entry></row>
- <row><entry><literal><= (timestamp,timestamp)</literal></entry></row>
- <row><entry><literal>> (timestamp,timestamp)</literal></entry></row>
- <row><entry><literal>>= (timestamp,timestamp)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>timestamp_minmax_multi_ops</literal></entry>
- <entry><literal>= (timestamp,timestamp)</literal></entry>
- </row>
- <row><entry><literal>< (timestamp,timestamp)</literal></entry></row>
- <row><entry><literal><= (timestamp,timestamp)</literal></entry></row>
- <row><entry><literal>> (timestamp,timestamp)</literal></entry></row>
- <row><entry><literal>>= (timestamp,timestamp)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>timestamptz_bloom_ops</literal></entry>
- <entry><literal>= (timestamptz,timestamptz)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>timestamptz_minmax_ops</literal></entry>
- <entry><literal>= (timestamptz,timestamptz)</literal></entry>
- </row>
- <row><entry><literal>< (timestamptz,timestamptz)</literal></entry></row>
- <row><entry><literal><= (timestamptz,timestamptz)</literal></entry></row>
- <row><entry><literal>> (timestamptz,timestamptz)</literal></entry></row>
- <row><entry><literal>>= (timestamptz,timestamptz)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>timestamptz_minmax_multi_ops</literal></entry>
- <entry><literal>= (timestamptz,timestamptz)</literal></entry>
- </row>
- <row><entry><literal>< (timestamptz,timestamptz)</literal></entry></row>
- <row><entry><literal><= (timestamptz,timestamptz)</literal></entry></row>
- <row><entry><literal>> (timestamptz,timestamptz)</literal></entry></row>
- <row><entry><literal>>= (timestamptz,timestamptz)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>time_bloom_ops</literal></entry>
- <entry><literal>= (time,time)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>time_minmax_ops</literal></entry>
- <entry><literal>= (time,time)</literal></entry>
- </row>
- <row><entry><literal>< (time,time)</literal></entry></row>
- <row><entry><literal><= (time,time)</literal></entry></row>
- <row><entry><literal>> (time,time)</literal></entry></row>
- <row><entry><literal>>= (time,time)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>time_minmax_multi_ops</literal></entry>
- <entry><literal>= (time,time)</literal></entry>
- </row>
- <row><entry><literal>< (time,time)</literal></entry></row>
- <row><entry><literal><= (time,time)</literal></entry></row>
- <row><entry><literal>> (time,time)</literal></entry></row>
- <row><entry><literal>>= (time,time)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>timetz_bloom_ops</literal></entry>
- <entry><literal>= (timetz,timetz)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>timetz_minmax_ops</literal></entry>
- <entry><literal>= (timetz,timetz)</literal></entry>
- </row>
- <row><entry><literal>< (timetz,timetz)</literal></entry></row>
- <row><entry><literal><= (timetz,timetz)</literal></entry></row>
- <row><entry><literal>> (timetz,timetz)</literal></entry></row>
- <row><entry><literal>>= (timetz,timetz)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>timetz_minmax_multi_ops</literal></entry>
- <entry><literal>= (timetz,timetz)</literal></entry>
- </row>
- <row><entry><literal>< (timetz,timetz)</literal></entry></row>
- <row><entry><literal><= (timetz,timetz)</literal></entry></row>
- <row><entry><literal>> (timetz,timetz)</literal></entry></row>
- <row><entry><literal>>= (timetz,timetz)</literal></entry></row>
-
- <row>
- <entry valign="middle"><literal>uuid_bloom_ops</literal></entry>
- <entry><literal>= (uuid,uuid)</literal></entry>
- </row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>uuid_minmax_ops</literal></entry>
- <entry><literal>= (uuid,uuid)</literal></entry>
- </row>
- <row><entry><literal>< (uuid,uuid)</literal></entry></row>
- <row><entry><literal>> (uuid,uuid)</literal></entry></row>
- <row><entry><literal><= (uuid,uuid)</literal></entry></row>
- <row><entry><literal>>= (uuid,uuid)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>uuid_minmax_multi_ops</literal></entry>
- <entry><literal>= (uuid,uuid)</literal></entry>
- </row>
- <row><entry><literal>< (uuid,uuid)</literal></entry></row>
- <row><entry><literal>> (uuid,uuid)</literal></entry></row>
- <row><entry><literal><= (uuid,uuid)</literal></entry></row>
- <row><entry><literal>>= (uuid,uuid)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="4"><literal>varbit_minmax_ops</literal></entry>
- <entry><literal>= (varbit,varbit)</literal></entry>
- </row>
- <row><entry><literal>< (varbit,varbit)</literal></entry></row>
- <row><entry><literal>> (varbit,varbit)</literal></entry></row>
- <row><entry><literal><= (varbit,varbit)</literal></entry></row>
- <row><entry><literal>>= (varbit,varbit)</literal></entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <sect2 id="brin-builtin-opclasses--parameters">
- <title>Operator Class Parameters</title>
-
- <para>
- Some of the built-in operator classes allow specifying parameters affecting
- behavior of the operator class. Each operator class has its own set of
- allowed parameters. Only the <literal>bloom</literal> and <literal>minmax-multi</literal>
- operator classes allow specifying parameters:
- </para>
-
- <para>
- bloom operator classes accept these parameters:
- </para>
-
- <variablelist>
- <varlistentry>
- <term><literal>n_distinct_per_range</literal></term>
- <listitem>
- <para>
- Defines the estimated number of distinct non-null values in the block
- range, used by <acronym>BRIN</acronym> bloom indexes for sizing of the
- Bloom filter. It behaves similarly to <literal>n_distinct</literal> option
- for <xref linkend="sql-altertable"/>. When set to a positive value,
- each block range is assumed to contain this number of distinct non-null
- values. When set to a negative value, which must be greater than or
- equal to -1, the number of distinct non-null values is assumed to grow linearly with
- the maximum possible number of tuples in the block range (about 290
- rows per block). The default value is <literal>-0.1</literal>, and
- the minimum number of distinct non-null values is <literal>16</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><literal>false_positive_rate</literal></term>
- <listitem>
- <para>
- Defines the desired false positive rate used by <acronym>BRIN</acronym>
- bloom indexes for sizing of the Bloom filter. The values must be
- between 0.0001 and 0.25. The default value is 0.01, which is 1% false
- positive rate.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
-
- <para>
- minmax-multi operator classes accept these parameters:
- </para>
-
- <variablelist>
- <varlistentry>
- <term><literal>values_per_range</literal></term>
- <listitem>
- <para>
- Defines the maximum number of values stored by <acronym>BRIN</acronym>
- minmax indexes to summarize a block range. Each value may represent
- either a point, or a boundary of an interval. Values must be between
- 8 and 256, and the default value is 32.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </sect2>
-
-</sect1>
-
-<sect1 id="brin-extensibility">
- <title>Extensibility</title>
-
- <para>
- The <acronym>BRIN</acronym> interface has a high level of abstraction,
- requiring the access method implementer only to implement the semantics
- of the data type being accessed. The <acronym>BRIN</acronym> layer
- itself takes care of concurrency, logging and searching the index structure.
- </para>
-
- <para>
- All it takes to get a <acronym>BRIN</acronym> access method working is to
- implement a few user-defined methods, which define the behavior of
- summary values stored in the index and the way they interact with
- scan keys.
- In short, <acronym>BRIN</acronym> combines
- extensibility with generality, code reuse, and a clean interface.
- </para>
-
- <para>
- There are four methods that an operator class for <acronym>BRIN</acronym>
- must provide:
-
- <variablelist>
- <varlistentry>
- <term><function>BrinOpcInfo *opcInfo(Oid type_oid)</function></term>
- <listitem>
- <para>
- Returns internal information about the indexed columns' summary data.
- The return value must point to a palloc'd <structname>BrinOpcInfo</structname>,
- which has this definition:
-<programlisting>
-typedef struct BrinOpcInfo
-{
- /* Number of columns stored in an index column of this opclass */
- uint16 oi_nstored;
-
- /* Opaque pointer for the opclass' private use */
- void *oi_opaque;
-
- /* Type cache entries of the stored columns */
- TypeCacheEntry *oi_typcache[FLEXIBLE_ARRAY_MEMBER];
-} BrinOpcInfo;
-</programlisting>
- <structname>BrinOpcInfo</structname>.<structfield>oi_opaque</structfield> can be used by the
- operator class routines to pass information between support functions
- during an index scan.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>bool consistent(BrinDesc *bdesc, BrinValues *column,
- ScanKey *keys, int nkeys)</function></term>
- <listitem>
- <para>
- Returns whether all the ScanKey entries are consistent with the given
- indexed values for a range.
- The attribute number to use is passed as part of the scan key.
- Multiple scan keys for the same attribute may be passed at once; the
- number of entries is determined by the <literal>nkeys</literal> parameter.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>bool consistent(BrinDesc *bdesc, BrinValues *column,
- ScanKey key)</function></term>
- <listitem>
- <para>
- Returns whether the ScanKey is consistent with the given indexed
- values for a range.
- The attribute number to use is passed as part of the scan key.
- This is an older backward-compatible variant of the consistent function.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>bool addValue(BrinDesc *bdesc, BrinValues *column,
- Datum newval, bool isnull)</function></term>
- <listitem>
- <para>
- Given an index tuple and an indexed value, modifies the indicated
- attribute of the tuple so that it additionally represents the new value.
- If any modification was done to the tuple, <literal>true</literal> is
- returned.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>bool unionTuples(BrinDesc *bdesc, BrinValues *a,
- BrinValues *b)</function></term>
- <listitem>
- <para>
- Consolidates two index tuples. Given two index tuples, modifies the
- indicated attribute of the first of them so that it represents both tuples.
- The second tuple is not modified.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- An operator class for <acronym>BRIN</acronym> can optionally specify the
- following method:
-
- <variablelist>
- <varlistentry>
- <term><function>void options(local_relopts *relopts)</function></term>
- <listitem>
- <para>
- Defines a set of user-visible parameters that control operator class
- behavior.
- </para>
-
- <para>
- The <function>options</function> function is passed a pointer to a
- <structname>local_relopts</structname> struct, which needs to be
- filled with a set of operator class specific options. The options
- can be accessed from other support functions using the
- <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
- <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
- </para>
-
- <para>
- Since both key extraction of indexed values and representation of the
- key in <acronym>BRIN</acronym> are flexible, they may depend on
- user-specified parameters.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- The core distribution includes support for four types of operator classes:
- minmax, minmax-multi, inclusion and bloom. Operator class definitions
- using them are shipped for in-core data types as appropriate. Additional
- operator classes can be defined by the user for other data types using
- equivalent definitions, without having to write any source code;
- appropriate catalog entries being declared is enough. Note that
- assumptions about the semantics of operator strategies are embedded in the
- support functions' source code.
- </para>
-
- <para>
- Operator classes that implement completely different semantics are also
- possible, provided implementations of the four main support functions
- described above are written. Note that backwards compatibility across major
- releases is not guaranteed: for example, additional support functions might
- be required in later releases.
- </para>
-
- <para>
- To write an operator class for a data type that implements a totally
- ordered set, it is possible to use the minmax support functions
- alongside the corresponding operators, as shown in
- <xref linkend="brin-extensibility-minmax-table"/>.
- All operator class members (functions and operators) are mandatory.
- </para>
-
- <table id="brin-extensibility-minmax-table">
- <title>Function and Support Numbers for Minmax Operator Classes</title>
- <tgroup cols="2">
- <colspec colname="col1" colwidth="1*"/>
- <colspec colname="col2" colwidth="2*"/>
- <thead>
- <row>
- <entry>Operator class member</entry>
- <entry>Object</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>Support Function 1</entry>
- <entry>internal function <function>brin_minmax_opcinfo()</function></entry>
- </row>
- <row>
- <entry>Support Function 2</entry>
- <entry>internal function <function>brin_minmax_add_value()</function></entry>
- </row>
- <row>
- <entry>Support Function 3</entry>
- <entry>internal function <function>brin_minmax_consistent()</function></entry>
- </row>
- <row>
- <entry>Support Function 4</entry>
- <entry>internal function <function>brin_minmax_union()</function></entry>
- </row>
- <row>
- <entry>Operator Strategy 1</entry>
- <entry>operator less-than</entry>
- </row>
- <row>
- <entry>Operator Strategy 2</entry>
- <entry>operator less-than-or-equal-to</entry>
- </row>
- <row>
- <entry>Operator Strategy 3</entry>
- <entry>operator equal-to</entry>
- </row>
- <row>
- <entry>Operator Strategy 4</entry>
- <entry>operator greater-than-or-equal-to</entry>
- </row>
- <row>
- <entry>Operator Strategy 5</entry>
- <entry>operator greater-than</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- To write an operator class for a complex data type which has values
- included within another type, it's possible to use the inclusion support
- functions alongside the corresponding operators, as shown
- in <xref linkend="brin-extensibility-inclusion-table"/>. It requires
- only a single additional function, which can be written in any language.
- More functions can be defined for additional functionality. All operators
- are optional. Some operators require other operators, as shown as
- dependencies on the table.
- </para>
-
- <table id="brin-extensibility-inclusion-table">
- <title>Function and Support Numbers for Inclusion Operator Classes</title>
- <tgroup cols="3">
- <colspec colname="col1" colwidth="1*"/>
- <colspec colname="col2" colwidth="2*"/>
- <colspec colname="col3" colwidth="1*"/>
- <thead>
- <row>
- <entry>Operator class member</entry>
- <entry>Object</entry>
- <entry>Dependency</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>Support Function 1</entry>
- <entry>internal function <function>brin_inclusion_opcinfo()</function></entry>
- <entry></entry>
- </row>
- <row>
- <entry>Support Function 2</entry>
- <entry>internal function <function>brin_inclusion_add_value()</function></entry>
- <entry></entry>
- </row>
- <row>
- <entry>Support Function 3</entry>
- <entry>internal function <function>brin_inclusion_consistent()</function></entry>
- <entry></entry>
- </row>
- <row>
- <entry>Support Function 4</entry>
- <entry>internal function <function>brin_inclusion_union()</function></entry>
- <entry></entry>
- </row>
- <row>
- <entry>Support Function 11</entry>
- <entry>function to merge two elements</entry>
- <entry></entry>
- </row>
- <row>
- <entry>Support Function 12</entry>
- <entry>optional function to check whether two elements are mergeable</entry>
- <entry></entry>
- </row>
- <row>
- <entry>Support Function 13</entry>
- <entry>optional function to check if an element is contained within another</entry>
- <entry></entry>
- </row>
- <row>
- <entry>Support Function 14</entry>
- <entry>optional function to check whether an element is empty</entry>
- <entry></entry>
- </row>
- <row>
- <entry>Operator Strategy 1</entry>
- <entry>operator left-of</entry>
- <entry>Operator Strategy 4</entry>
- </row>
- <row>
- <entry>Operator Strategy 2</entry>
- <entry>operator does-not-extend-to-the-right-of</entry>
- <entry>Operator Strategy 5</entry>
- </row>
- <row>
- <entry>Operator Strategy 3</entry>
- <entry>operator overlaps</entry>
- <entry></entry>
- </row>
- <row>
- <entry>Operator Strategy 4</entry>
- <entry>operator does-not-extend-to-the-left-of</entry>
- <entry>Operator Strategy 1</entry>
- </row>
- <row>
- <entry>Operator Strategy 5</entry>
- <entry>operator right-of</entry>
- <entry>Operator Strategy 2</entry>
- </row>
- <row>
- <entry>Operator Strategy 6, 18</entry>
- <entry>operator same-as-or-equal-to</entry>
- <entry>Operator Strategy 7</entry>
- </row>
- <row>
- <entry>Operator Strategy 7, 16, 24, 25</entry>
- <entry>operator contains-or-equal-to</entry>
- <entry></entry>
- </row>
- <row>
- <entry>Operator Strategy 8, 26, 27</entry>
- <entry>operator is-contained-by-or-equal-to</entry>
- <entry>Operator Strategy 3</entry>
- </row>
- <row>
- <entry>Operator Strategy 9</entry>
- <entry>operator does-not-extend-above</entry>
- <entry>Operator Strategy 11</entry>
- </row>
- <row>
- <entry>Operator Strategy 10</entry>
- <entry>operator is-below</entry>
- <entry>Operator Strategy 12</entry>
- </row>
- <row>
- <entry>Operator Strategy 11</entry>
- <entry>operator is-above</entry>
- <entry>Operator Strategy 9</entry>
- </row>
- <row>
- <entry>Operator Strategy 12</entry>
- <entry>operator does-not-extend-below</entry>
- <entry>Operator Strategy 10</entry>
- </row>
- <row>
- <entry>Operator Strategy 20</entry>
- <entry>operator less-than</entry>
- <entry>Operator Strategy 5</entry>
- </row>
- <row>
- <entry>Operator Strategy 21</entry>
- <entry>operator less-than-or-equal-to</entry>
- <entry>Operator Strategy 5</entry>
- </row>
- <row>
- <entry>Operator Strategy 22</entry>
- <entry>operator greater-than</entry>
- <entry>Operator Strategy 1</entry>
- </row>
- <row>
- <entry>Operator Strategy 23</entry>
- <entry>operator greater-than-or-equal-to</entry>
- <entry>Operator Strategy 1</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- Support function numbers 1 through 10 are reserved for the BRIN internal
- functions, so the SQL level functions start with number 11. Support
- function number 11 is the main function required to build the index.
- It should accept two arguments with the same data type as the operator class,
- and return the union of them. The inclusion operator class can store union
- values with different data types if it is defined with the
- <literal>STORAGE</literal> parameter. The return value of the union
- function should match the <literal>STORAGE</literal> data type.
- </para>
-
- <para>
- Support function numbers 12 and 14 are provided to support
- irregularities of built-in data types. Function number 12
- is used to support network addresses from different families which
- are not mergeable. Function number 14 is used to support
- empty ranges. Function number 13 is an optional but
- recommended one, which allows the new value to be checked before
- it is passed to the union function. As the BRIN framework can shortcut
- some operations when the union is not changed, using this
- function can improve index performance.
- </para>
-
- <para>
- To write an operator class for a data type that implements only an equality
- operator and supports hashing, it is possible to use the bloom support procedures
- alongside the corresponding operators, as shown in
- <xref linkend="brin-extensibility-bloom-table"/>.
- All operator class members (procedures and operators) are mandatory.
- </para>
-
- <table id="brin-extensibility-bloom-table">
- <title>Procedure and Support Numbers for Bloom Operator Classes</title>
- <tgroup cols="2">
- <thead>
- <row>
- <entry>Operator class member</entry>
- <entry>Object</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>Support Procedure 1</entry>
- <entry>internal function <function>brin_bloom_opcinfo()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 2</entry>
- <entry>internal function <function>brin_bloom_add_value()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 3</entry>
- <entry>internal function <function>brin_bloom_consistent()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 4</entry>
- <entry>internal function <function>brin_bloom_union()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 5</entry>
- <entry>internal function <function>brin_bloom_options()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 11</entry>
- <entry>function to compute hash of an element</entry>
- </row>
- <row>
- <entry>Operator Strategy 1</entry>
- <entry>operator equal-to</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- Support procedure numbers 1-10 are reserved for the BRIN internal
- functions, so the SQL level functions start with number 11. Support
- function number 11 is the main function required to build the index.
- It should accept one argument with the same data type as the operator class,
- and return a hash of the value.
- </para>
-
- <para>
- The minmax-multi operator class is also intended for data types implementing
- a totally ordered set, and may be seen as a simple extension of the minmax
- operator class. While minmax operator class summarizes values from each block
- range into a single contiguous interval, minmax-multi allows summarization
- into multiple smaller intervals to improve handling of outlier values.
- It is possible to use the minmax-multi support procedures alongside the
- corresponding operators, as shown in
- <xref linkend="brin-extensibility-minmax-multi-table"/>.
- All operator class members (procedures and operators) are mandatory.
- </para>
-
- <table id="brin-extensibility-minmax-multi-table">
- <title>Procedure and Support Numbers for minmax-multi Operator Classes</title>
- <tgroup cols="2">
- <thead>
- <row>
- <entry>Operator class member</entry>
- <entry>Object</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>Support Procedure 1</entry>
- <entry>internal function <function>brin_minmax_multi_opcinfo()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 2</entry>
- <entry>internal function <function>brin_minmax_multi_add_value()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 3</entry>
- <entry>internal function <function>brin_minmax_multi_consistent()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 4</entry>
- <entry>internal function <function>brin_minmax_multi_union()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 5</entry>
- <entry>internal function <function>brin_minmax_multi_options()</function></entry>
- </row>
- <row>
- <entry>Support Procedure 11</entry>
- <entry>function to compute distance between two values (length of a range)</entry>
- </row>
- <row>
- <entry>Operator Strategy 1</entry>
- <entry>operator less-than</entry>
- </row>
- <row>
- <entry>Operator Strategy 2</entry>
- <entry>operator less-than-or-equal-to</entry>
- </row>
- <row>
- <entry>Operator Strategy 3</entry>
- <entry>operator equal-to</entry>
- </row>
- <row>
- <entry>Operator Strategy 4</entry>
- <entry>operator greater-than-or-equal-to</entry>
- </row>
- <row>
- <entry>Operator Strategy 5</entry>
- <entry>operator greater-than</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- Both minmax and inclusion operator classes support cross-data-type
- operators, though with these the dependencies become more complicated.
- The minmax operator class requires a full set of operators to be
- defined with both arguments having the same data type. It allows
- additional data types to be supported by defining extra sets
- of operators. Inclusion operator class operator strategies are dependent
- on another operator strategy as shown in
- <xref linkend="brin-extensibility-inclusion-table"/>, or the same
- operator strategy as themselves. They require the dependency
- operator to be defined with the <literal>STORAGE</literal> data type as the
- left-hand-side argument and the other supported data type to be the
- right-hand-side argument of the supported operator. See
- <literal>float4_minmax_ops</literal> as an example of minmax, and
- <literal>box_inclusion_ops</literal> as an example of inclusion.
- </para>
-</sect1>
-</chapter>
diff --git a/doc/src/sgml/btree.sgml b/doc/src/sgml/btree.sgml
deleted file mode 100644
index be8210286b..0000000000
--- a/doc/src/sgml/btree.sgml
+++ /dev/null
@@ -1,914 +0,0 @@
-<!-- doc/src/sgml/btree.sgml -->
-
-<chapter id="btree">
-<title>B-Tree Indexes</title>
-
- <indexterm>
- <primary>index</primary>
- <secondary>B-Tree</secondary>
- </indexterm>
-
-<sect1 id="btree-intro">
- <title>Introduction</title>
-
- <para>
- <productname>PostgreSQL</productname> includes an implementation of the
- standard <acronym>btree</acronym> (multi-way balanced tree) index data
- structure. Any data type that can be sorted into a well-defined linear
- order can be indexed by a btree index. The only limitation is that an
- index entry cannot exceed approximately one-third of a page (after TOAST
- compression, if applicable).
- </para>
-
- <para>
- Because each btree operator class imposes a sort order on its data type,
- btree operator classes (or, really, operator families) have come to be
- used as <productname>PostgreSQL</productname>'s general representation
- and understanding of sorting semantics. Therefore, they've acquired
- some features that go beyond what would be needed just to support btree
- indexes, and parts of the system that are quite distant from the
- btree <acronym>AM</acronym> make use of them.
- </para>
-
-</sect1>
-
-<sect1 id="btree-behavior">
- <title>Behavior of B-Tree Operator Classes</title>
-
- <para>
- As shown in <xref linkend="xindex-btree-strat-table"/>, a btree operator
- class must provide five comparison operators,
- <literal><</literal>,
- <literal><=</literal>,
- <literal>=</literal>,
- <literal>>=</literal> and
- <literal>></literal>.
- One might expect that <literal><></literal> should also be part of
- the operator class, but it is not, because it would almost never be
- useful to use a <literal><></literal> WHERE clause in an index
- search. (For some purposes, the planner treats <literal><></literal>
- as associated with a btree operator class; but it finds that operator via
- the <literal>=</literal> operator's negator link, rather than
- from <structname>pg_amop</structname>.)
- </para>
-
- <para>
- When several data types share near-identical sorting semantics, their
- operator classes can be grouped into an operator family. Doing so is
- advantageous because it allows the planner to make deductions about
- cross-type comparisons. Each operator class within the family should
- contain the single-type operators (and associated support functions)
- for its input data type, while cross-type comparison operators and
- support functions are <quote>loose</quote> in the family. It is
- recommendable that a complete set of cross-type operators be included
- in the family, thus ensuring that the planner can represent any
- comparison conditions that it deduces from transitivity.
- </para>
-
- <para>
- There are some basic assumptions that a btree operator family must
- satisfy:
- </para>
-
- <itemizedlist>
- <listitem>
- <para>
- An <literal>=</literal> operator must be an equivalence relation; that
- is, for all non-null values <replaceable>A</replaceable>,
- <replaceable>B</replaceable>, <replaceable>C</replaceable> of the
- data type:
-
- <itemizedlist>
- <listitem>
- <para>
- <replaceable>A</replaceable> <literal>=</literal>
- <replaceable>A</replaceable> is true
- (<firstterm>reflexive law</firstterm>)
- </para>
- </listitem>
- <listitem>
- <para>
- if <replaceable>A</replaceable> <literal>=</literal>
- <replaceable>B</replaceable>,
- then <replaceable>B</replaceable> <literal>=</literal>
- <replaceable>A</replaceable>
- (<firstterm>symmetric law</firstterm>)
- </para>
- </listitem>
- <listitem>
- <para>
- if <replaceable>A</replaceable> <literal>=</literal>
- <replaceable>B</replaceable> and <replaceable>B</replaceable>
- <literal>=</literal> <replaceable>C</replaceable>,
- then <replaceable>A</replaceable> <literal>=</literal>
- <replaceable>C</replaceable>
- (<firstterm>transitive law</firstterm>)
- </para>
- </listitem>
- </itemizedlist>
- </para>
- </listitem>
-
- <listitem>
- <para>
- A <literal><</literal> operator must be a strong ordering relation;
- that is, for all non-null values <replaceable>A</replaceable>,
- <replaceable>B</replaceable>, <replaceable>C</replaceable>:
-
- <itemizedlist>
- <listitem>
- <para>
- <replaceable>A</replaceable> <literal><</literal>
- <replaceable>A</replaceable> is false
- (<firstterm>irreflexive law</firstterm>)
- </para>
- </listitem>
- <listitem>
- <para>
- if <replaceable>A</replaceable> <literal><</literal>
- <replaceable>B</replaceable>
- and <replaceable>B</replaceable> <literal><</literal>
- <replaceable>C</replaceable>,
- then <replaceable>A</replaceable> <literal><</literal>
- <replaceable>C</replaceable>
- (<firstterm>transitive law</firstterm>)
- </para>
- </listitem>
- </itemizedlist>
- </para>
- </listitem>
-
- <listitem>
- <para>
- Furthermore, the ordering is total; that is, for all non-null
- values <replaceable>A</replaceable>, <replaceable>B</replaceable>:
-
- <itemizedlist>
- <listitem>
- <para>
- exactly one of <replaceable>A</replaceable> <literal><</literal>
- <replaceable>B</replaceable>, <replaceable>A</replaceable>
- <literal>=</literal> <replaceable>B</replaceable>, and
- <replaceable>B</replaceable> <literal><</literal>
- <replaceable>A</replaceable> is true
- (<firstterm>trichotomy law</firstterm>)
- </para>
- </listitem>
- </itemizedlist>
-
- (The trichotomy law justifies the definition of the comparison support
- function, of course.)
- </para>
- </listitem>
- </itemizedlist>
-
- <para>
- The other three operators are defined in terms of <literal>=</literal>
- and <literal><</literal> in the obvious way, and must act consistently
- with them.
- </para>
-
- <para>
- For an operator family supporting multiple data types, the above laws must
- hold when <replaceable>A</replaceable>, <replaceable>B</replaceable>,
- <replaceable>C</replaceable> are taken from any data types in the family.
- The transitive laws are the trickiest to ensure, as in cross-type
- situations they represent statements that the behaviors of two or three
- different operators are consistent.
- As an example, it would not work to put <type>float8</type>
- and <type>numeric</type> into the same operator family, at least not with
- the current semantics that <type>numeric</type> values are converted
- to <type>float8</type> for comparison to a <type>float8</type>. Because
- of the limited accuracy of <type>float8</type>, this means there are
- distinct <type>numeric</type> values that will compare equal to the
- same <type>float8</type> value, and thus the transitive law would fail.
- </para>
-
- <para>
- Another requirement for a multiple-data-type family is that any implicit
- or binary-coercion casts that are defined between data types included in
- the operator family must not change the associated sort ordering.
- </para>
-
- <para>
- It should be fairly clear why a btree index requires these laws to hold
- within a single data type: without them there is no ordering to arrange
- the keys with. Also, index searches using a comparison key of a
- different data type require comparisons to behave sanely across two
- data types. The extensions to three or more data types within a family
- are not strictly required by the btree index mechanism itself, but the
- planner relies on them for optimization purposes.
- </para>
-
-</sect1>
-
-<sect1 id="btree-support-funcs">
- <title>B-Tree Support Functions</title>
-
- <para>
- As shown in <xref linkend="xindex-btree-support-table"/>, btree defines
- one required and four optional support functions. The five
- user-defined methods are:
- </para>
- <variablelist>
- <varlistentry>
- <term><function>order</function></term>
- <listitem>
- <para>
- For each combination of data types that a btree operator family
- provides comparison operators for, it must provide a comparison
- support function, registered in
- <structname>pg_amproc</structname> with support function number 1
- and
- <structfield>amproclefttype</structfield>/<structfield>amprocrighttype</structfield>
- equal to the left and right data types for the comparison (i.e.,
- the same data types that the matching operators are registered
- with in <structname>pg_amop</structname>). The comparison
- function must take two non-null values
- <replaceable>A</replaceable> and <replaceable>B</replaceable> and
- return an <type>int32</type> value that is
- <literal><</literal> <literal>0</literal>,
- <literal>0</literal>, or <literal>></literal>
- <literal>0</literal> when <replaceable>A</replaceable>
- <literal><</literal> <replaceable>B</replaceable>,
- <replaceable>A</replaceable> <literal>=</literal>
- <replaceable>B</replaceable>, or <replaceable>A</replaceable>
- <literal>></literal> <replaceable>B</replaceable>,
- respectively. A null result is disallowed: all values of the
- data type must be comparable. See
- <filename>src/backend/access/nbtree/nbtcompare.c</filename> for
- examples.
- </para>
-
- <para>
- If the compared values are of a collatable data type, the
- appropriate collation OID will be passed to the comparison
- support function, using the standard
- <function>PG_GET_COLLATION()</function> mechanism.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term><function>sortsupport</function></term>
- <listitem>
- <para>
- Optionally, a btree operator family may provide <firstterm>sort
- support</firstterm> function(s), registered under support
- function number 2. These functions allow implementing
- comparisons for sorting purposes in a more efficient way than
- naively calling the comparison support function. The APIs
- involved in this are defined in
- <filename>src/include/utils/sortsupport.h</filename>.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term><function>in_range</function></term>
- <listitem>
- <indexterm>
- <primary>in_range support functions</primary>
- </indexterm>
-
- <indexterm>
- <primary>support functions</primary>
- <secondary>in_range</secondary>
- </indexterm>
- <para>
- Optionally, a btree operator family may provide
- <firstterm>in_range</firstterm> support function(s), registered
- under support function number 3. These are not used during btree
- index operations; rather, they extend the semantics of the
- operator family so that it can support window clauses containing
- the <literal>RANGE</literal> <replaceable>offset</replaceable>
- <literal>PRECEDING</literal> and <literal>RANGE</literal>
- <replaceable>offset</replaceable> <literal>FOLLOWING</literal>
- frame bound types (see <xref
- linkend="syntax-window-functions"/>). Fundamentally, the extra
- information provided is how to add or subtract an
- <replaceable>offset</replaceable> value in a way that is
- compatible with the family's data ordering.
- </para>
-
- <para>
- An <function>in_range</function> function must have the signature
-<synopsis>
-in_range(<replaceable>val</replaceable> type1, <replaceable>base</replaceable> type1, <replaceable>offset</replaceable> type2, <replaceable>sub</replaceable> bool, <replaceable>less</replaceable> bool)
-returns bool
-</synopsis>
- <replaceable>val</replaceable> and
- <replaceable>base</replaceable> must be of the same type, which
- is one of the types supported by the operator family (i.e., a
- type for which it provides an ordering). However,
- <replaceable>offset</replaceable> could be of a different type,
- which might be one otherwise unsupported by the family. An
- example is that the built-in <literal>time_ops</literal> family
- provides an <function>in_range</function> function that has
- <replaceable>offset</replaceable> of type <type>interval</type>.
- A family can provide <function>in_range</function> functions for
- any of its supported types and one or more
- <replaceable>offset</replaceable> types. Each
- <function>in_range</function> function should be entered in
- <structname>pg_amproc</structname> with
- <structfield>amproclefttype</structfield> equal to
- <type>type1</type> and <structfield>amprocrighttype</structfield>
- equal to <type>type2</type>.
- </para>
-
- <para>
- The essential semantics of an <function>in_range</function>
- function depend on the two Boolean flag parameters. It should
- add or subtract <replaceable>base</replaceable> and
- <replaceable>offset</replaceable>, then compare
- <replaceable>val</replaceable> to the result, as follows:
- <itemizedlist>
- <listitem>
- <para>
- if <literal>!</literal><replaceable>sub</replaceable> and
- <literal>!</literal><replaceable>less</replaceable>, return
- <replaceable>val</replaceable> <literal>>=</literal>
- (<replaceable>base</replaceable> <literal>+</literal>
- <replaceable>offset</replaceable>)
- </para>
- </listitem>
- <listitem>
- <para>
- if <literal>!</literal><replaceable>sub</replaceable> and
- <replaceable>less</replaceable>, return
- <replaceable>val</replaceable> <literal><=</literal>
- (<replaceable>base</replaceable> <literal>+</literal>
- <replaceable>offset</replaceable>)
- </para>
- </listitem>
- <listitem>
- <para>
- if <replaceable>sub</replaceable> and
- <literal>!</literal><replaceable>less</replaceable>, return
- <replaceable>val</replaceable> <literal>>=</literal>
- (<replaceable>base</replaceable> <literal>-</literal>
- <replaceable>offset</replaceable>)
- </para>
- </listitem>
- <listitem>
- <para>
- if <replaceable>sub</replaceable> and
- <replaceable>less</replaceable>, return
- <replaceable>val</replaceable> <literal><=</literal>
- (<replaceable>base</replaceable> <literal>-</literal>
- <replaceable>offset</replaceable>)
- </para>
- </listitem>
- </itemizedlist>
- Before doing so, the function should check the sign of
- <replaceable>offset</replaceable>: if it is less than zero, raise
- error
- <literal>ERRCODE_INVALID_PRECEDING_OR_FOLLOWING_SIZE</literal>
- (22013) with error text like <quote>invalid preceding or
- following size in window function</quote>. (This is required by
- the SQL standard, although nonstandard operator families might
- perhaps choose to ignore this restriction, since there seems to
- be little semantic necessity for it.) This requirement is
- delegated to the <function>in_range</function> function so that
- the core code needn't understand what <quote>less than
- zero</quote> means for a particular data type.
- </para>
-
- <para>
- An additional expectation is that <function>in_range</function>
- functions should, if practical, avoid throwing an error if
- <replaceable>base</replaceable> <literal>+</literal>
- <replaceable>offset</replaceable> or
- <replaceable>base</replaceable> <literal>-</literal>
- <replaceable>offset</replaceable> would overflow. The correct
- comparison result can be determined even if that value would be
- out of the data type's range. Note that if the data type
- includes concepts such as <quote>infinity</quote> or
- <quote>NaN</quote>, extra care may be needed to ensure that
- <function>in_range</function>'s results agree with the normal
- sort order of the operator family.
- </para>
-
- <para>
- The results of the <function>in_range</function> function must be
- consistent with the sort ordering imposed by the operator family.
- To be precise, given any fixed values of
- <replaceable>offset</replaceable> and
- <replaceable>sub</replaceable>, then:
- <itemizedlist>
- <listitem>
- <para>
- If <function>in_range</function> with
- <replaceable>less</replaceable> = true is true for some
- <replaceable>val1</replaceable> and
- <replaceable>base</replaceable>, it must be true for every
- <replaceable>val2</replaceable> <literal><=</literal>
- <replaceable>val1</replaceable> with the same
- <replaceable>base</replaceable>.
- </para>
- </listitem>
- <listitem>
- <para>
- If <function>in_range</function> with
- <replaceable>less</replaceable> = true is false for some
- <replaceable>val1</replaceable> and
- <replaceable>base</replaceable>, it must be false for every
- <replaceable>val2</replaceable> <literal>>=</literal>
- <replaceable>val1</replaceable> with the same
- <replaceable>base</replaceable>.
- </para>
- </listitem>
- <listitem>
- <para>
- If <function>in_range</function> with
- <replaceable>less</replaceable> = true is true for some
- <replaceable>val</replaceable> and
- <replaceable>base1</replaceable>, it must be true for every
- <replaceable>base2</replaceable> <literal>>=</literal>
- <replaceable>base1</replaceable> with the same
- <replaceable>val</replaceable>.
- </para>
- </listitem>
- <listitem>
- <para>
- If <function>in_range</function> with
- <replaceable>less</replaceable> = true is false for some
- <replaceable>val</replaceable> and
- <replaceable>base1</replaceable>, it must be false for every
- <replaceable>base2</replaceable> <literal><=</literal>
- <replaceable>base1</replaceable> with the same
- <replaceable>val</replaceable>.
- </para>
- </listitem>
- </itemizedlist>
- Analogous statements with inverted conditions hold when
- <replaceable>less</replaceable> = false.
- </para>
-
- <para>
- If the type being ordered (<type>type1</type>) is collatable, the
- appropriate collation OID will be passed to the
- <function>in_range</function> function, using the standard
- PG_GET_COLLATION() mechanism.
- </para>
-
- <para>
- <function>in_range</function> functions need not handle NULL
- inputs, and typically will be marked strict.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term><function>equalimage</function></term>
- <listitem>
- <para>
- Optionally, a btree operator family may provide
- <function>equalimage</function> (<quote>equality implies image
- equality</quote>) support functions, registered under support
- function number 4. These functions allow the core code to
- determine when it is safe to apply the btree deduplication
- optimization. Currently, <function>equalimage</function>
- functions are only called when building or rebuilding an index.
- </para>
- <para>
- An <function>equalimage</function> function must have the
- signature
-<synopsis>
-equalimage(<replaceable>opcintype</replaceable> <type>oid</type>) returns bool
-</synopsis>
- The return value is static information about an operator class
- and collation. Returning <literal>true</literal> indicates that
- the <function>order</function> function for the operator class is
- guaranteed to only return <literal>0</literal> (<quote>arguments
- are equal</quote>) when its <replaceable>A</replaceable> and
- <replaceable>B</replaceable> arguments are also interchangeable
- without any loss of semantic information. Not registering an
- <function>equalimage</function> function or returning
- <literal>false</literal> indicates that this condition cannot be
- assumed to hold.
- </para>
- <para>
- The <replaceable>opcintype</replaceable> argument is the
- <literal><structname>pg_type</structname>.oid</literal> of the
- data type that the operator class indexes. This is a convenience
- that allows reuse of the same underlying
- <function>equalimage</function> function across operator classes.
- If <replaceable>opcintype</replaceable> is a collatable data
- type, the appropriate collation OID will be passed to the
- <function>equalimage</function> function, using the standard
- <function>PG_GET_COLLATION()</function> mechanism.
- </para>
- <para>
- As far as the operator class is concerned, returning
- <literal>true</literal> indicates that deduplication is safe (or
- safe for the collation whose OID was passed to its
- <function>equalimage</function> function). However, the core
- code will only deem deduplication safe for an index when
- <emphasis>every</emphasis> indexed column uses an operator class
- that registers an <function>equalimage</function> function, and
- each function actually returns <literal>true</literal> when
- called.
- </para>
- <para>
- Image equality is <emphasis>almost</emphasis> the same condition
- as simple bitwise equality. There is one subtle difference: When
- indexing a varlena data type, the on-disk representation of two
- image equal datums may not be bitwise equal due to inconsistent
- application of <acronym>TOAST</acronym> compression on input.
- Formally, when an operator class's
- <function>equalimage</function> function returns
- <literal>true</literal>, it is safe to assume that the
- <literal>datum_image_eq()</literal> C function will always agree
- with the operator class's <function>order</function> function
- (provided that the same collation OID is passed to both the
- <function>equalimage</function> and <function>order</function>
- functions).
- </para>
- <para>
- The core code is fundamentally unable to deduce anything about
- the <quote>equality implies image equality</quote> status of an
- operator class within a multiple-data-type family based on
- details from other operator classes in the same family. Also, it
- is not sensible for an operator family to register a cross-type
- <function>equalimage</function> function, and attempting to do so
- will result in an error. This is because <quote>equality implies
- image equality</quote> status does not just depend on
- sorting/equality semantics, which are more or less defined at the
- operator family level. In general, the semantics that one
- particular data type implements must be considered separately.
- </para>
- <para>
- The convention followed by the operator classes included with the
- core <productname>PostgreSQL</productname> distribution is to
- register a stock, generic <function>equalimage</function>
- function. Most operator classes register
- <function>btequalimage()</function>, which indicates that
- deduplication is safe unconditionally. Operator classes for
- collatable data types such as <type>text</type> register
- <function>btvarstrequalimage()</function>, which indicates that
- deduplication is safe with deterministic collations. Best
- practice for third-party extensions is to register their own
- custom function to retain control.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term><function>options</function></term>
- <listitem>
- <para>
- Optionally, a B-tree operator family may provide
- <function>options</function> (<quote>operator class specific
- options</quote>) support functions, registered under support
- function number 5. These functions define a set of user-visible
- parameters that control operator class behavior.
- </para>
- <para>
- An <function>options</function> support function must have the
- signature
-<synopsis>
-options(<replaceable>relopts</replaceable> <type>local_relopts *</type>) returns void
-</synopsis>
- The function is passed a pointer to a <structname>local_relopts</structname>
- struct, which needs to be filled with a set of operator class
- specific options. The options can be accessed from other support
- functions using the <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
- <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
- </para>
- <para>
- Currently, no B-Tree operator class has an <function>options</function>
- support function. B-tree doesn't allow flexible representation of keys
- like GiST, SP-GiST, GIN and BRIN do. So, <function>options</function>
- probably doesn't have much application in the current B-tree index
- access method. Nevertheless, this support function was added to B-tree
- for uniformity, and will probably find uses during further
- evolution of B-tree in <productname>PostgreSQL</productname>.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
-</sect1>
-
-<sect1 id="btree-implementation">
- <title>Implementation</title>
-
- <para>
- This section covers B-Tree index implementation details that may be
- of use to advanced users. See
- <filename>src/backend/access/nbtree/README</filename> in the source
- distribution for a much more detailed, internals-focused description
- of the B-Tree implementation.
- </para>
- <sect2 id="btree-structure">
- <title>B-Tree Structure</title>
- <para>
- <productname>PostgreSQL</productname> B-Tree indexes are
- multi-level tree structures, where each level of the tree can be
- used as a doubly-linked list of pages. A single metapage is stored
- in a fixed position at the start of the first segment file of the
- index. All other pages are either leaf pages or internal pages.
- Leaf pages are the pages on the lowest level of the tree. All
- other levels consist of internal pages. Each leaf page contains
- tuples that point to table rows. Each internal page contains
- tuples that point to the next level down in the tree. Typically,
- over 99% of all pages are leaf pages. Both internal pages and leaf
- pages use the standard page format described in <xref
- linkend="storage-page-layout"/>.
- </para>
- <para>
- New leaf pages are added to a B-Tree index when an existing leaf
- page cannot fit an incoming tuple. A <firstterm>page
- split</firstterm> operation makes room for items that originally
- belonged on the overflowing page by moving a portion of the items
- to a new page. Page splits must also insert a new
- <firstterm>downlink</firstterm> to the new page in the parent page,
- which may cause the parent to split in turn. Page splits
- <quote>cascade upwards</quote> in a recursive fashion. When the
- root page finally cannot fit a new downlink, a <firstterm>root page
- split</firstterm> operation takes place. This adds a new level to
- the tree structure by creating a new root page that is one level
- above the original root page.
- </para>
- </sect2>
-
- <sect2 id="btree-deletion">
- <title>Bottom-up Index Deletion</title>
- <para>
- B-Tree indexes are not directly aware that under MVCC, there might
- be multiple extant versions of the same logical table row; to an
- index, each tuple is an independent object that needs its own index
- entry. <quote>Version churn</quote> tuples may sometimes
- accumulate and adversely affect query latency and throughput. This
- typically occurs with <command>UPDATE</command>-heavy workloads
- where most individual updates cannot apply the
- <link linkend="storage-hot"><acronym>HOT</acronym> optimization.</link>
- Changing the value of only
- one column covered by one index during an <command>UPDATE</command>
- <emphasis>always</emphasis> necessitates a new set of index tuples
- — one for <emphasis>each and every</emphasis> index on the
- table. Note in particular that this includes indexes that were not
- <quote>logically modified</quote> by the <command>UPDATE</command>.
- All indexes will need a successor physical index tuple that points
- to the latest version in the table. Each new tuple within each
- index will generally need to coexist with the original
- <quote>updated</quote> tuple for a short period of time (typically
- until shortly after the <command>UPDATE</command> transaction
- commits).
- </para>
- <para>
- B-Tree indexes incrementally delete version churn index tuples by
- performing <firstterm>bottom-up index deletion</firstterm> passes.
- Each deletion pass is triggered in reaction to an anticipated
- <quote>version churn page split</quote>. This only happens with
- indexes that are not logically modified by
- <command>UPDATE</command> statements, where concentrated build up
- of obsolete versions in particular pages would occur otherwise. A
- page split will usually be avoided, though it's possible that
- certain implementation-level heuristics will fail to identify and
- delete even one garbage index tuple (in which case a page split or
- deduplication pass resolves the issue of an incoming new tuple not
- fitting on a leaf page). The worst-case number of versions that
- any index scan must traverse (for any single logical row) is an
- important contributor to overall system responsiveness and
- throughput. A bottom-up index deletion pass targets suspected
- garbage tuples in a single leaf page based on
- <emphasis>qualitative</emphasis> distinctions involving logical
- rows and versions. This contrasts with the <quote>top-down</quote>
- index cleanup performed by autovacuum workers, which is triggered
- when certain <emphasis>quantitative</emphasis> table-level
- thresholds are exceeded (see <xref linkend="autovacuum"/>).
- </para>
- <note>
- <para>
- Not all deletion operations that are performed within B-Tree
- indexes are bottom-up deletion operations. There is a distinct
- category of index tuple deletion: <firstterm>simple index tuple
- deletion</firstterm>. This is a deferred maintenance operation
- that deletes index tuples that are known to be safe to delete
- (those whose item identifier's <literal>LP_DEAD</literal> bit is
- already set). Like bottom-up index deletion, simple index
- deletion takes place at the point that a page split is anticipated
- as a way of avoiding the split.
- </para>
- <para>
- Simple deletion is opportunistic in the sense that it can only
- take place when recent index scans set the
- <literal>LP_DEAD</literal> bits of affected items in passing.
- Prior to <productname>PostgreSQL</productname> 14, the only
- category of B-Tree deletion was simple deletion. The main
- differences between it and bottom-up deletion are that only the
- former is opportunistically driven by the activity of passing
- index scans, while only the latter specifically targets version
- churn from <command>UPDATE</command>s that do not logically modify
- indexed columns.
- </para>
- </note>
- <para>
- Bottom-up index deletion performs the vast majority of all garbage
- index tuple cleanup for particular indexes with certain workloads.
- This is expected with any B-Tree index that is subject to
- significant version churn from <command>UPDATE</command>s that
- rarely or never logically modify the columns that the index covers.
- The average and worst-case number of versions per logical row can
- be kept low purely through targeted incremental deletion passes.
- It's quite possible that the on-disk size of certain indexes will
- never increase by even one single page/block despite
- <emphasis>constant</emphasis> version churn from
- <command>UPDATE</command>s. Even then, an exhaustive <quote>clean
- sweep</quote> by a <command>VACUUM</command> operation (typically
- run in an autovacuum worker process) will eventually be required as
- a part of <emphasis>collective</emphasis> cleanup of the table and
- each of its indexes.
- </para>
- <para>
- Unlike <command>VACUUM</command>, bottom-up index deletion does not
- provide any strong guarantees about how old the oldest garbage
- index tuple may be. No index can be permitted to retain
- <quote>floating garbage</quote> index tuples that became dead prior
- to a conservative cutoff point shared by the table and all of its
- indexes collectively. This fundamental table-level invariant makes
- it safe to recycle table <acronym>TID</acronym>s. This is how it
- is possible for distinct logical rows to reuse the same table
- <acronym>TID</acronym> over time (though this can never happen with
- two logical rows whose lifetimes span the same
- <command>VACUUM</command> cycle).
- </para>
- </sect2>
-
- <sect2 id="btree-deduplication">
- <title>Deduplication</title>
- <para>
- A duplicate is a leaf page tuple (a tuple that points to a table
- row) where <emphasis>all</emphasis> indexed key columns have values
- that match corresponding column values from at least one other leaf
- page tuple in the same index. Duplicate tuples are quite common in
- practice. B-Tree indexes can use a special, space-efficient
- representation for duplicates when an optional technique is
- enabled: <firstterm>deduplication</firstterm>.
- </para>
- <para>
- Deduplication works by periodically merging groups of duplicate
- tuples together, forming a single <firstterm>posting list</firstterm> tuple for each
- group. The column key value(s) only appear once in this
- representation. This is followed by a sorted array of
- <acronym>TID</acronym>s that point to rows in the table. This
- significantly reduces the storage size of indexes where each value
- (or each distinct combination of column values) appears several
- times on average. The latency of queries can be reduced
- significantly. Overall query throughput may increase
- significantly. The overhead of routine index vacuuming may also be
- reduced significantly.
- </para>
- <note>
- <para>
- B-Tree deduplication is just as effective with
- <quote>duplicates</quote> that contain a NULL value, even though
- NULL values are never equal to each other according to the
- <literal>=</literal> member of any B-Tree operator class. As far
- as any part of the implementation that understands the on-disk
- B-Tree structure is concerned, NULL is just another value from the
- domain of indexed values.
- </para>
- </note>
- <para>
- The deduplication process occurs lazily, when a new item is
- inserted that cannot fit on an existing leaf page, though only when
- index tuple deletion could not free sufficient space for the new
- item (typically deletion is briefly considered and then skipped
- over). Unlike GIN posting list tuples, B-Tree posting list tuples
- do not need to expand every time a new duplicate is inserted; they
- are merely an alternative physical representation of the original
- logical contents of the leaf page. This design prioritizes
- consistent performance with mixed read-write workloads. Most
- client applications will at least see a moderate performance
- benefit from using deduplication. Deduplication is enabled by
- default.
- </para>
- <para>
- <command>CREATE INDEX</command> and <command>REINDEX</command>
- apply deduplication to create posting list tuples, though the
- strategy they use is slightly different. Each group of duplicate
- ordinary tuples encountered in the sorted input taken from the
- table is merged into a posting list tuple
- <emphasis>before</emphasis> being added to the current pending leaf
- page. Individual posting list tuples are packed with as many
- <acronym>TID</acronym>s as possible. Leaf pages are written out in
- the usual way, without any separate deduplication pass. This
- strategy is well-suited to <command>CREATE INDEX</command> and
- <command>REINDEX</command> because they are once-off batch
- operations.
- </para>
- <para>
- Write-heavy workloads that don't benefit from deduplication due to
- having few or no duplicate values in indexes will incur a small,
- fixed performance penalty (unless deduplication is explicitly
- disabled). The <literal>deduplicate_items</literal> storage
- parameter can be used to disable deduplication within individual
- indexes. There is never any performance penalty with read-only
- workloads, since reading posting list tuples is at least as
- efficient as reading the standard tuple representation. Disabling
- deduplication isn't usually helpful.
- </para>
- <para>
- It is sometimes possible for unique indexes (as well as unique
- constraints) to use deduplication. This allows leaf pages to
- temporarily <quote>absorb</quote> extra version churn duplicates.
- Deduplication in unique indexes augments bottom-up index deletion,
- especially in cases where a long-running transaction holds a
- snapshot that blocks garbage collection. The goal is to buy time
- for the bottom-up index deletion strategy to become effective
- again. Delaying page splits until a single long-running
- transaction naturally goes away can allow a bottom-up deletion pass
- to succeed where an earlier deletion pass failed.
- </para>
- <tip>
- <para>
- A special heuristic is applied to determine whether a
- deduplication pass in a unique index should take place. It can
- often skip straight to splitting a leaf page, avoiding a
- performance penalty from wasting cycles on unhelpful deduplication
- passes. If you're concerned about the overhead of deduplication,
- consider setting <literal>deduplicate_items = off</literal>
- selectively. Leaving deduplication enabled in unique indexes has
- little downside.
- </para>
- </tip>
- <para>
- Deduplication cannot be used in all cases due to
- implementation-level restrictions. Deduplication safety is
- determined when <command>CREATE INDEX</command> or
- <command>REINDEX</command> is run.
- </para>
- <para>
- Note that deduplication is deemed unsafe and cannot be used in the
- following cases involving semantically significant differences
- among equal datums:
- </para>
- <para>
- <itemizedlist>
- <listitem>
- <para>
- <type>text</type>, <type>varchar</type>, and <type>char</type>
- cannot use deduplication when a
- <emphasis>nondeterministic</emphasis> collation is used. Case
- and accent differences must be preserved among equal datums.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <type>numeric</type> cannot use deduplication. Numeric display
- scale must be preserved among equal datums.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <type>jsonb</type> cannot use deduplication, since the
- <type>jsonb</type> B-Tree operator class uses
- <type>numeric</type> internally.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <type>float4</type> and <type>float8</type> cannot use
- deduplication. These types have distinct representations for
- <literal>-0</literal> and <literal>0</literal>, which are
- nevertheless considered equal. This difference must be
- preserved.
- </para>
- </listitem>
- </itemizedlist>
- </para>
- <para>
- There is one further implementation-level restriction that may be
- lifted in a future version of
- <productname>PostgreSQL</productname>:
- </para>
- <para>
- <itemizedlist>
- <listitem>
- <para>
- Container types (such as composite types, arrays, or range
- types) cannot use deduplication.
- </para>
- </listitem>
- </itemizedlist>
- </para>
- <para>
- There is one further implementation-level restriction that applies
- regardless of the operator class or collation used:
- </para>
- <para>
- <itemizedlist>
- <listitem>
- <para>
- <literal>INCLUDE</literal> indexes can never use deduplication.
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- </sect2>
-</sect1>
-
-</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index f39b4efe38..de683e3400 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -84,12 +84,7 @@
<!ENTITY catalogs SYSTEM "catalogs.sgml">
<!ENTITY system-views SYSTEM "system-views.sgml">
<!ENTITY geqo SYSTEM "geqo.sgml">
-<!ENTITY btree SYSTEM "btree.sgml">
-<!ENTITY gist SYSTEM "gist.sgml">
-<!ENTITY spgist SYSTEM "spgist.sgml">
-<!ENTITY gin SYSTEM "gin.sgml">
-<!ENTITY brin SYSTEM "brin.sgml">
-<!ENTITY hash SYSTEM "hash.sgml">
+<!ENTITY indextypes SYSTEM "indextypes.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY tableam SYSTEM "tableam.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml
deleted file mode 100644
index 5bd1efae92..0000000000
--- a/doc/src/sgml/gin.sgml
+++ /dev/null
@@ -1,714 +0,0 @@
-<!-- doc/src/sgml/gin.sgml -->
-
-<chapter id="gin">
-<title>GIN Indexes</title>
-
- <indexterm>
- <primary>index</primary>
- <secondary>GIN</secondary>
- </indexterm>
-
-<sect1 id="gin-intro">
- <title>Introduction</title>
-
- <para>
- <acronym>GIN</acronym> stands for Generalized Inverted Index.
- <acronym>GIN</acronym> is designed for handling cases where the items
- to be indexed are composite values, and the queries to be handled by
- the index need to search for element values that appear within
- the composite items. For example, the items could be documents,
- and the queries could be searches for documents containing specific words.
- </para>
-
- <para>
- We use the word <firstterm>item</firstterm> to refer to a composite value that
- is to be indexed, and the word <firstterm>key</firstterm> to refer to an element
- value. <acronym>GIN</acronym> always stores and searches for keys,
- not item values per se.
- </para>
-
- <para>
- A <acronym>GIN</acronym> index stores a set of (key, posting list) pairs,
- where a <firstterm>posting list</firstterm> is a set of row IDs in which the key
- occurs. The same row ID can appear in multiple posting lists, since
- an item can contain more than one key. Each key value is stored only
- once, so a <acronym>GIN</acronym> index is very compact for cases
- where the same key appears many times.
- </para>
-
- <para>
- <acronym>GIN</acronym> is generalized in the sense that the
- <acronym>GIN</acronym> access method code does not need to know the
- specific operations that it accelerates.
- Instead, it uses custom strategies defined for particular data types.
- The strategy defines how keys are extracted from indexed items and
- query conditions, and how to determine whether a row that contains
- some of the key values in a query actually satisfies the query.
- </para>
-
- <para>
- One advantage of <acronym>GIN</acronym> is that it allows the development
- of custom data types with the appropriate access methods, by
- an expert in the domain of the data type, rather than a database expert.
- This is much the same advantage as using <acronym>GiST</acronym>.
- </para>
-
- <para>
- The <acronym>GIN</acronym>
- implementation in <productname>PostgreSQL</productname> is primarily
- maintained by Teodor Sigaev and Oleg Bartunov. There is more
- information about <acronym>GIN</acronym> on their
- <ulink url="http://www.sai.msu.su/~megera/wiki/Gin">website</ulink>.
- </para>
-</sect1>
-
-<sect1 id="gin-builtin-opclasses">
- <title>Built-in Operator Classes</title>
-
- <para>
- The core <productname>PostgreSQL</productname> distribution
- includes the <acronym>GIN</acronym> operator classes shown in
- <xref linkend="gin-builtin-opclasses-table"/>.
- (Some of the optional modules described in <xref linkend="contrib"/>
- provide additional <acronym>GIN</acronym> operator classes.)
- </para>
-
- <table id="gin-builtin-opclasses-table">
- <title>Built-in <acronym>GIN</acronym> Operator Classes</title>
- <tgroup cols="2">
- <thead>
- <row>
- <entry>Name</entry>
- <entry>Indexable Operators</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry morerows="3" valign="middle"><literal>array_ops</literal></entry>
- <entry><literal>&& (anyarray,anyarray)</literal></entry>
- </row>
- <row>
- <entry><literal>@> (anyarray,anyarray)</literal></entry>
- </row>
- <row>
- <entry><literal><@ (anyarray,anyarray)</literal></entry>
- </row>
- <row>
- <entry><literal>= (anyarray,anyarray)</literal></entry>
- </row>
- <row>
- <entry morerows="5" valign="middle"><literal>jsonb_ops</literal></entry>
- <entry><literal>@> (jsonb,jsonb)</literal></entry>
- </row>
- <row>
- <entry><literal>@? (jsonb,jsonpath)</literal></entry>
- </row>
- <row>
- <entry><literal>@@ (jsonb,jsonpath)</literal></entry>
- </row>
- <row>
- <entry><literal>? (jsonb,text)</literal></entry>
- </row>
- <row>
- <entry><literal>?| (jsonb,text[])</literal></entry>
- </row>
- <row>
- <entry><literal>?& (jsonb,text[])</literal></entry>
- </row>
- <row>
- <entry morerows="2" valign="middle"><literal>jsonb_path_ops</literal></entry>
- <entry><literal>@> (jsonb,jsonb)</literal></entry>
- </row>
- <row>
- <entry><literal>@? (jsonb,jsonpath)</literal></entry>
- </row>
- <row>
- <entry><literal>@@ (jsonb,jsonpath)</literal></entry>
- </row>
- <row>
- <entry valign="middle"><literal>tsvector_ops</literal></entry>
- <entry><literal>@@ (tsvector,tsquery)</literal></entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- Of the two operator classes for type <type>jsonb</type>, <literal>jsonb_ops</literal>
- is the default. <literal>jsonb_path_ops</literal> supports fewer operators but
- offers better performance for those operators.
- See <xref linkend="json-indexing"/> for details.
- </para>
-
-</sect1>
-
-<sect1 id="gin-extensibility">
- <title>Extensibility</title>
-
- <para>
- The <acronym>GIN</acronym> interface has a high level of abstraction,
- requiring the access method implementer only to implement the semantics of
- the data type being accessed. The <acronym>GIN</acronym> layer itself
- takes care of concurrency, logging and searching the tree structure.
- </para>
-
- <para>
- All it takes to get a <acronym>GIN</acronym> access method working is to
- implement a few user-defined methods, which define the behavior of
- keys in the tree and the relationships between keys, indexed items,
- and indexable queries. In short, <acronym>GIN</acronym> combines
- extensibility with generality, code reuse, and a clean interface.
- </para>
-
- <para>
- There are two methods that an operator class for
- <acronym>GIN</acronym> must provide:
-
- <variablelist>
- <varlistentry>
- <term><function>Datum *extractValue(Datum itemValue, int32 *nkeys,
- bool **nullFlags)</function></term>
- <listitem>
- <para>
- Returns a palloc'd array of keys given an item to be indexed. The
- number of returned keys must be stored into <literal>*nkeys</literal>.
- If any of the keys can be null, also palloc an array of
- <literal>*nkeys</literal> <type>bool</type> fields, store its address at
- <literal>*nullFlags</literal>, and set these null flags as needed.
- <literal>*nullFlags</literal> can be left <symbol>NULL</symbol> (its initial value)
- if all keys are non-null.
- The return value can be <symbol>NULL</symbol> if the item contains no keys.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>Datum *extractQuery(Datum query, int32 *nkeys,
- StrategyNumber n, bool **pmatch, Pointer **extra_data,
- bool **nullFlags, int32 *searchMode)</function></term>
- <listitem>
- <para>
- Returns a palloc'd array of keys given a value to be queried; that is,
- <literal>query</literal> is the value on the right-hand side of an
- indexable operator whose left-hand side is the indexed column.
- <literal>n</literal> is the strategy number of the operator within the
- operator class (see <xref linkend="xindex-strategies"/>).
- Often, <function>extractQuery</function> will need
- to consult <literal>n</literal> to determine the data type of
- <literal>query</literal> and the method it should use to extract key values.
- The number of returned keys must be stored into <literal>*nkeys</literal>.
- If any of the keys can be null, also palloc an array of
- <literal>*nkeys</literal> <type>bool</type> fields, store its address at
- <literal>*nullFlags</literal>, and set these null flags as needed.
- <literal>*nullFlags</literal> can be left <symbol>NULL</symbol> (its initial value)
- if all keys are non-null.
- The return value can be <symbol>NULL</symbol> if the <literal>query</literal> contains no keys.
- </para>
-
- <para>
- <literal>searchMode</literal> is an output argument that allows
- <function>extractQuery</function> to specify details about how the search
- will be done.
- If <literal>*searchMode</literal> is set to
- <literal>GIN_SEARCH_MODE_DEFAULT</literal> (which is the value it is
- initialized to before call), only items that match at least one of
- the returned keys are considered candidate matches.
- If <literal>*searchMode</literal> is set to
- <literal>GIN_SEARCH_MODE_INCLUDE_EMPTY</literal>, then in addition to items
- containing at least one matching key, items that contain no keys at
- all are considered candidate matches. (This mode is useful for
- implementing is-subset-of operators, for example.)
- If <literal>*searchMode</literal> is set to <literal>GIN_SEARCH_MODE_ALL</literal>,
- then all non-null items in the index are considered candidate
- matches, whether they match any of the returned keys or not. (This
- mode is much slower than the other two choices, since it requires
- scanning essentially the entire index, but it may be necessary to
- implement corner cases correctly. An operator that needs this mode
- in most cases is probably not a good candidate for a GIN operator
- class.)
- The symbols to use for setting this mode are defined in
- <filename>access/gin.h</filename>.
- </para>
-
- <para>
- <literal>pmatch</literal> is an output argument for use when partial match
- is supported. To use it, <function>extractQuery</function> must allocate
- an array of <literal>*nkeys</literal> <type>bool</type>s and store its address at
- <literal>*pmatch</literal>. Each element of the array should be set to true
- if the corresponding key requires partial match, false if not.
- If <literal>*pmatch</literal> is set to <symbol>NULL</symbol> then GIN assumes partial match
- is not required. The variable is initialized to <symbol>NULL</symbol> before call,
- so this argument can simply be ignored by operator classes that do
- not support partial match.
- </para>
-
- <para>
- <literal>extra_data</literal> is an output argument that allows
- <function>extractQuery</function> to pass additional data to the
- <function>consistent</function> and <function>comparePartial</function> methods.
- To use it, <function>extractQuery</function> must allocate
- an array of <literal>*nkeys</literal> pointers and store its address at
- <literal>*extra_data</literal>, then store whatever it wants to into the
- individual pointers. The variable is initialized to <symbol>NULL</symbol> before
- call, so this argument can simply be ignored by operator classes that
- do not require extra data. If <literal>*extra_data</literal> is set, the
- whole array is passed to the <function>consistent</function> method, and
- the appropriate element to the <function>comparePartial</function> method.
- </para>
-
- </listitem>
- </varlistentry>
- </variablelist>
-
- An operator class must also provide a function to check if an indexed item
- matches the query. It comes in two flavors, a Boolean <function>consistent</function>
- function, and a ternary <function>triConsistent</function> function.
- <function>triConsistent</function> covers the functionality of both, so providing
- <function>triConsistent</function> alone is sufficient. However, if the Boolean
- variant is significantly cheaper to calculate, it can be advantageous to
- provide both. If only the Boolean variant is provided, some optimizations
- that depend on refuting index items before fetching all the keys are
- disabled.
-
- <variablelist>
- <varlistentry>
- <term><function>bool consistent(bool check[], StrategyNumber n, Datum query,
- int32 nkeys, Pointer extra_data[], bool *recheck,
- Datum queryKeys[], bool nullFlags[])</function></term>
- <listitem>
- <para>
- Returns true if an indexed item satisfies the query operator with
- strategy number <literal>n</literal> (or might satisfy it, if the recheck
- indication is returned). This function does not have direct access
- to the indexed item's value, since <acronym>GIN</acronym> does not
- store items explicitly. Rather, what is available is knowledge
- about which key values extracted from the query appear in a given
- indexed item. The <literal>check</literal> array has length
- <literal>nkeys</literal>, which is the same as the number of keys previously
- returned by <function>extractQuery</function> for this <literal>query</literal> datum.
- Each element of the
- <literal>check</literal> array is true if the indexed item contains the
- corresponding query key, i.e., if (check[i] == true) the i-th key of the
- <function>extractQuery</function> result array is present in the indexed item.
- The original <literal>query</literal> datum is
- passed in case the <function>consistent</function> method needs to consult it,
- and so are the <literal>queryKeys[]</literal> and <literal>nullFlags[]</literal>
- arrays previously returned by <function>extractQuery</function>.
- <literal>extra_data</literal> is the extra-data array returned by
- <function>extractQuery</function>, or <symbol>NULL</symbol> if none.
- </para>
-
- <para>
- When <function>extractQuery</function> returns a null key in
- <literal>queryKeys[]</literal>, the corresponding <literal>check[]</literal> element
- is true if the indexed item contains a null key; that is, the
- semantics of <literal>check[]</literal> are like <literal>IS NOT DISTINCT
- FROM</literal>. The <function>consistent</function> function can examine the
- corresponding <literal>nullFlags[]</literal> element if it needs to tell
- the difference between a regular value match and a null match.
- </para>
-
- <para>
- On success, <literal>*recheck</literal> should be set to true if the heap
- tuple needs to be rechecked against the query operator, or false if
- the index test is exact. That is, a false return value guarantees
- that the heap tuple does not match the query; a true return value with
- <literal>*recheck</literal> set to false guarantees that the heap tuple does
- match the query; and a true return value with
- <literal>*recheck</literal> set to true means that the heap tuple might match
- the query, so it needs to be fetched and rechecked by evaluating the
- query operator directly against the originally indexed item.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>GinTernaryValue triConsistent(GinTernaryValue check[], StrategyNumber n, Datum query,
- int32 nkeys, Pointer extra_data[],
- Datum queryKeys[], bool nullFlags[])</function></term>
- <listitem>
- <para>
- <function>triConsistent</function> is similar to <function>consistent</function>,
- but instead of Booleans in the <literal>check</literal> vector, there are
- three possible values for each
- key: <literal>GIN_TRUE</literal>, <literal>GIN_FALSE</literal> and
- <literal>GIN_MAYBE</literal>. <literal>GIN_FALSE</literal> and <literal>GIN_TRUE</literal>
- have the same meaning as regular Boolean values, while
- <literal>GIN_MAYBE</literal> means that the presence of that key is not known.
- When <literal>GIN_MAYBE</literal> values are present, the function should only
- return <literal>GIN_TRUE</literal> if the item certainly matches whether or
- not the index item contains the corresponding query keys. Likewise, the
- function must return <literal>GIN_FALSE</literal> only if the item certainly
- does not match, whether or not it contains the <literal>GIN_MAYBE</literal>
- keys. If the result depends on the <literal>GIN_MAYBE</literal> entries, i.e.,
- the match cannot be confirmed or refuted based on the known query keys,
- the function must return <literal>GIN_MAYBE</literal>.
- </para>
- <para>
- When there are no <literal>GIN_MAYBE</literal> values in the <literal>check</literal>
- vector, a <literal>GIN_MAYBE</literal> return value is the equivalent of
- setting the <literal>recheck</literal> flag in the
- Boolean <function>consistent</function> function.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- <para>
- In addition, GIN must have a way to sort the key values stored in the index.
- The operator class can define the sort ordering by specifying a comparison
- method:
-
- <variablelist>
- <varlistentry>
- <term><function>int compare(Datum a, Datum b)</function></term>
- <listitem>
- <para>
- Compares two keys (not indexed items!) and returns an integer less than
- zero, zero, or greater than zero, indicating whether the first key is
- less than, equal to, or greater than the second. Null keys are never
- passed to this function.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- Alternatively, if the operator class does not provide a <function>compare</function>
- method, GIN will look up the default btree operator class for the index
- key data type, and use its comparison function. It is recommended to
- specify the comparison function in a GIN operator class that is meant for
- just one data type, as looking up the btree operator class costs a few
- cycles. However, polymorphic GIN operator classes (such
- as <literal>array_ops</literal>) typically cannot specify a single comparison
- function.
- </para>
-
- <para>
- An operator class for <acronym>GIN</acronym> can optionally supply the
- following methods:
-
- <variablelist>
- <varlistentry>
- <term><function>int comparePartial(Datum partial_key, Datum key, StrategyNumber n,
- Pointer extra_data)</function></term>
- <listitem>
- <para>
- Compare a partial-match query key to an index key. Returns an integer
- whose sign indicates the result: less than zero means the index key
- does not match the query, but the index scan should continue; zero
- means that the index key does match the query; greater than zero
- indicates that the index scan should stop because no more matches
- are possible. The strategy number <literal>n</literal> of the operator
- that generated the partial match query is provided, in case its
- semantics are needed to determine when to end the scan. Also,
- <literal>extra_data</literal> is the corresponding element of the extra-data
- array made by <function>extractQuery</function>, or <symbol>NULL</symbol> if none.
- Null keys are never passed to this function.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term><function>void options(local_relopts *relopts)</function></term>
- <listitem>
- <para>
- Defines a set of user-visible parameters that control operator class
- behavior.
- </para>
-
- <para>
- The <function>options</function> function is passed a pointer to a
- <structname>local_relopts</structname> struct, which needs to be
- filled with a set of operator class specific options. The options
- can be accessed from other support functions using the
- <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
- <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
- </para>
-
- <para>
- Since both key extraction of indexed values and representation of the
- key in <acronym>GIN</acronym> are flexible, they may depend on
- user-specified parameters.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- <para>
- To support <quote>partial match</quote> queries, an operator class must
- provide the <function>comparePartial</function> method, and its
- <function>extractQuery</function> method must set the <literal>pmatch</literal>
- parameter when a partial-match query is encountered. See
- <xref linkend="gin-partial-match"/> for details.
- </para>
-
- <para>
- The actual data types of the various <literal>Datum</literal> values mentioned
- above vary depending on the operator class. The item values passed to
- <function>extractValue</function> are always of the operator class's input type, and
- all key values must be of the class's <literal>STORAGE</literal> type. The type of
- the <literal>query</literal> argument passed to <function>extractQuery</function>,
- <function>consistent</function> and <function>triConsistent</function> is whatever is the
- right-hand input type of the class member operator identified by the
- strategy number. This need not be the same as the indexed type, so long as
- key values of the correct type can be extracted from it. However, it is
- recommended that the SQL declarations of these three support functions use
- the opclass's indexed data type for the <literal>query</literal> argument, even
- though the actual type might be something else depending on the operator.
- </para>
-
-</sect1>
-
-<sect1 id="gin-implementation">
- <title>Implementation</title>
-
- <para>
- Internally, a <acronym>GIN</acronym> index contains a B-tree index
- constructed over keys, where each key is an element of one or more indexed
- items (a member of an array, for example) and where each tuple in a leaf
- page contains either a pointer to a B-tree of heap pointers (a
- <quote>posting tree</quote>), or a simple list of heap pointers (a <quote>posting
- list</quote>) when the list is small enough to fit into a single index tuple along
- with the key value. <xref linkend="gin-internals-figure"/> illustrates
- these components of a GIN index.
- </para>
-
- <para>
- As of <productname>PostgreSQL</productname> 9.1, null key values can be
- included in the index. Also, placeholder nulls are included in the index
- for indexed items that are null or contain no keys according to
- <function>extractValue</function>. This allows searches that should find empty
- items to do so.
- </para>
-
- <para>
- Multicolumn <acronym>GIN</acronym> indexes are implemented by building
- a single B-tree over composite values (column number, key value). The
- key values for different columns can be of different types.
- </para>
-
- <figure id="gin-internals-figure">
- <title>GIN Internals</title>
- <mediaobject>
- <imageobject>
- <imagedata fileref="images/gin.svg" format="SVG" width="100%"/>
- </imageobject>
- </mediaobject>
- </figure>
-
- <sect2 id="gin-fast-update">
- <title>GIN Fast Update Technique</title>
-
- <para>
- Updating a <acronym>GIN</acronym> index tends to be slow because of the
- intrinsic nature of inverted indexes: inserting or updating one heap row
- can cause many inserts into the index (one for each key extracted
- from the indexed item).
- <acronym>GIN</acronym> is capable of postponing much of this work by inserting
- new tuples into a temporary, unsorted list of pending entries.
- When the table is vacuumed or autoanalyzed, or when
- <function>gin_clean_pending_list</function> function is called, or if the
- pending list becomes larger than
- <xref linkend="guc-gin-pending-list-limit"/>, the entries are moved to the
- main <acronym>GIN</acronym> data structure using the same bulk insert
- techniques used during initial index creation. This greatly improves
- <acronym>GIN</acronym> index update speed, even counting the additional
- vacuum overhead. Moreover the overhead work can be done by a background
- process instead of in foreground query processing.
- </para>
-
- <para>
- The main disadvantage of this approach is that searches must scan the list
- of pending entries in addition to searching the regular index, and so
- a large list of pending entries will slow searches significantly.
- Another disadvantage is that, while most updates are fast, an update
- that causes the pending list to become <quote>too large</quote> will incur an
- immediate cleanup cycle and thus be much slower than other updates.
- Proper use of autovacuum can minimize both of these problems.
- </para>
-
- <para>
- If consistent response time is more important than update speed,
- use of pending entries can be disabled by turning off the
- <literal>fastupdate</literal> storage parameter for a
- <acronym>GIN</acronym> index. See <xref linkend="sql-createindex"/>
- for details.
- </para>
- </sect2>
-
- <sect2 id="gin-partial-match">
- <title>Partial Match Algorithm</title>
-
- <para>
- GIN can support <quote>partial match</quote> queries, in which the query
- does not determine an exact match for one or more keys, but the possible
- matches fall within a reasonably narrow range of key values (within the
- key sorting order determined by the <function>compare</function> support method).
- The <function>extractQuery</function> method, instead of returning a key value
- to be matched exactly, returns a key value that is the lower bound of
- the range to be searched, and sets the <literal>pmatch</literal> flag true.
- The key range is then scanned using the <function>comparePartial</function>
- method. <function>comparePartial</function> must return zero for a matching
- index key, less than zero for a non-match that is still within the range
- to be searched, or greater than zero if the index key is past the range
- that could match.
- </para>
- </sect2>
-
-</sect1>
-
-<sect1 id="gin-tips">
-<title>GIN Tips and Tricks</title>
-
- <variablelist>
- <varlistentry>
- <term>Create vs. insert</term>
- <listitem>
- <para>
- Insertion into a <acronym>GIN</acronym> index can be slow
- due to the likelihood of many keys being inserted for each item.
- So, for bulk insertions into a table it is advisable to drop the GIN
- index and recreate it after finishing bulk insertion.
- </para>
-
- <para>
- When <literal>fastupdate</literal> is enabled for <acronym>GIN</acronym>
- (see <xref linkend="gin-fast-update"/> for details), the penalty is
- less than when it is not. But for very large updates it may still be
- best to drop and recreate the index.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><xref linkend="guc-maintenance-work-mem"/></term>
- <listitem>
- <para>
- Build time for a <acronym>GIN</acronym> index is very sensitive to
- the <varname>maintenance_work_mem</varname> setting; it doesn't pay to
- skimp on work memory during index creation.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><xref linkend="guc-gin-pending-list-limit"/></term>
- <listitem>
- <para>
- During a series of insertions into an existing <acronym>GIN</acronym>
- index that has <literal>fastupdate</literal> enabled, the system will clean up
- the pending-entry list whenever the list grows larger than
- <varname>gin_pending_list_limit</varname>. To avoid fluctuations in observed
- response time, it's desirable to have pending-list cleanup occur in the
- background (i.e., via autovacuum). Foreground cleanup operations
- can be avoided by increasing <varname>gin_pending_list_limit</varname>
- or making autovacuum more aggressive.
- However, enlarging the threshold of the cleanup operation means that
- if a foreground cleanup does occur, it will take even longer.
- </para>
- <para>
- <varname>gin_pending_list_limit</varname> can be overridden for individual
- GIN indexes by changing storage parameters, which allows each
- GIN index to have its own cleanup threshold.
- For example, it's possible to increase the threshold only for the GIN
- index which can be updated heavily, and decrease it otherwise.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><xref linkend="guc-gin-fuzzy-search-limit"/></term>
- <listitem>
- <para>
- The primary goal of developing <acronym>GIN</acronym> indexes was
- to create support for highly scalable full-text search in
- <productname>PostgreSQL</productname>, and there are often situations when
- a full-text search returns a very large set of results. Moreover, this
- often happens when the query contains very frequent words, so that the
- large result set is not even useful. Since reading many
- tuples from the disk and sorting them could take a lot of time, this is
- unacceptable for production. (Note that the index search itself is very
- fast.)
- </para>
- <para>
- To facilitate controlled execution of such queries,
- <acronym>GIN</acronym> has a configurable soft upper limit on the
- number of rows returned: the
- <varname>gin_fuzzy_search_limit</varname> configuration parameter.
- It is set to 0 (meaning no limit) by default.
- If a non-zero limit is set, then the returned set is a subset of
- the whole result set, chosen at random.
- </para>
- <para>
- <quote>Soft</quote> means that the actual number of returned results
- could differ somewhat from the specified limit, depending on the query
- and the quality of the system's random number generator.
- </para>
- <para>
- From experience, values in the thousands (e.g., 5000 — 20000)
- work well.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
-</sect1>
-
-<sect1 id="gin-limit">
- <title>Limitations</title>
-
- <para>
- <acronym>GIN</acronym> assumes that indexable operators are strict. This
- means that <function>extractValue</function> will not be called at all on a null
- item value (instead, a placeholder index entry is created automatically),
- and <function>extractQuery</function> will not be called on a null query
- value either (instead, the query is presumed to be unsatisfiable). Note
- however that null key values contained within a non-null composite item
- or query value are supported.
- </para>
-</sect1>
-
-<sect1 id="gin-examples">
- <title>Examples</title>
-
- <para>
- The core <productname>PostgreSQL</productname> distribution
- includes the <acronym>GIN</acronym> operator classes previously shown in
- <xref linkend="gin-builtin-opclasses-table"/>.
- The following <filename>contrib</filename> modules also contain
- <acronym>GIN</acronym> operator classes:
-
- <variablelist>
- <varlistentry>
- <term><filename>btree_gin</filename></term>
- <listitem>
- <para>B-tree equivalent functionality for several data types</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>hstore</filename></term>
- <listitem>
- <para>Module for storing (key, value) pairs</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>intarray</filename></term>
- <listitem>
- <para>Enhanced support for <type>int[]</type></para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>pg_trgm</filename></term>
- <listitem>
- <para>Text similarity using trigram matching</para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-</sect1>
-
-</chapter>
diff --git a/doc/src/sgml/gist.sgml b/doc/src/sgml/gist.sgml
deleted file mode 100644
index 8a19f156d8..0000000000
--- a/doc/src/sgml/gist.sgml
+++ /dev/null
@@ -1,1387 +0,0 @@
-<!-- doc/src/sgml/gist.sgml -->
-
-<chapter id="gist">
-<title>GiST Indexes</title>
-
- <indexterm>
- <primary>index</primary>
- <secondary>GiST</secondary>
- </indexterm>
-
-<sect1 id="gist-intro">
- <title>Introduction</title>
-
- <para>
- <acronym>GiST</acronym> stands for Generalized Search Tree. It is a
- balanced, tree-structured access method, that acts as a base template in
- which to implement arbitrary indexing schemes. B-trees, R-trees and many
- other indexing schemes can be implemented in <acronym>GiST</acronym>.
- </para>
-
- <para>
- One advantage of <acronym>GiST</acronym> is that it allows the development
- of custom data types with the appropriate access methods, by
- an expert in the domain of the data type, rather than a database expert.
- </para>
-
- <para>
- Some of the information here is derived from the University of California
- at Berkeley's GiST Indexing Project
- <ulink url="http://gist.cs.berkeley.edu/">web site</ulink> and
- Marcel Kornacker's thesis,
- <ulink url="http://www.sai.msu.su/~megera/postgres/gist/papers/concurrency/access-methods-for-next-generation.pdf.gz">
- Access Methods for Next-Generation Database Systems</ulink>.
- The <acronym>GiST</acronym>
- implementation in <productname>PostgreSQL</productname> is primarily
- maintained by Teodor Sigaev and Oleg Bartunov, and there is more
- information on their
- <ulink url="http://www.sai.msu.su/~megera/postgres/gist/">web site</ulink>.
- </para>
-
-</sect1>
-
-<sect1 id="gist-builtin-opclasses">
- <title>Built-in Operator Classes</title>
-
- <para>
- The core <productname>PostgreSQL</productname> distribution
- includes the <acronym>GiST</acronym> operator classes shown in
- <xref linkend="gist-builtin-opclasses-table"/>.
- (Some of the optional modules described in <xref linkend="contrib"/>
- provide additional <acronym>GiST</acronym> operator classes.)
- </para>
-
- <table id="gist-builtin-opclasses-table">
- <title>Built-in <acronym>GiST</acronym> Operator Classes</title>
- <tgroup cols="3">
- <colspec colname="col1" colwidth="2*"/>
- <colspec colname="col2" colwidth="3*"/>
- <colspec colname="col3" colwidth="2*"/>
- <thead>
- <row>
- <entry>Name</entry>
- <entry>Indexable Operators</entry>
- <entry>Ordering Operators</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry valign="middle" morerows="13"><literal>box_ops</literal></entry>
- <entry><literal><< (box, box)</literal></entry>
- <entry valign="middle" morerows="13"><literal><-> (box, point)</literal></entry>
- </row>
- <row><entry><literal>&< (box, box)</literal></entry></row>
- <row><entry><literal>&& (box, box)</literal></entry></row>
- <row><entry><literal>&> (box, box)</literal></entry></row>
- <row><entry><literal>>> (box, box)</literal></entry></row>
- <row><entry><literal>~= (box, box)</literal></entry></row>
- <row><entry><literal>@> (box, box)</literal></entry></row>
- <row><entry><literal><@ (box, box)</literal></entry></row>
- <row><entry><literal>&<| (box, box)</literal></entry></row>
- <row><entry><literal><<| (box, box)</literal></entry></row>
- <row><entry><literal>|>> (box, box)</literal></entry></row>
- <row><entry><literal>|&> (box, box)</literal></entry></row>
- <row><entry><literal>~ (box, box)</literal></entry></row>
- <row><entry><literal>@ (box, box)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="13"><literal>circle_ops</literal></entry>
- <entry><literal><< (circle, circle)</literal></entry>
- <entry valign="middle" morerows="13"><literal><-> (circle, point)</literal></entry>
- </row>
- <row><entry><literal>&< (circle, circle)</literal></entry></row>
- <row><entry><literal>&> (circle, circle)</literal></entry></row>
- <row><entry><literal>>> (circle, circle)</literal></entry></row>
- <row><entry><literal><@ (circle, circle)</literal></entry></row>
- <row><entry><literal>@> (circle, circle)</literal></entry></row>
- <row><entry><literal>~= (circle, circle)</literal></entry></row>
- <row><entry><literal>&& (circle, circle)</literal></entry></row>
- <row><entry><literal>|>> (circle, circle)</literal></entry></row>
- <row><entry><literal><<| (circle, circle)</literal></entry></row>
- <row><entry><literal>&<| (circle, circle)</literal></entry></row>
- <row><entry><literal>|&> (circle, circle)</literal></entry></row>
- <row><entry><literal>@ (circle, circle)</literal></entry></row>
- <row><entry><literal>~ (circle, circle)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="10"><literal>inet_ops</literal></entry>
- <entry><literal><< (inet, inet)</literal></entry>
- <entry valign="middle" morerows="10"></entry>
- </row>
- <row><entry><literal><<= (inet, inet)</literal></entry></row>
- <row><entry><literal>>> (inet, inet)</literal></entry></row>
- <row><entry><literal>>>= (inet, inet)</literal></entry></row>
- <row><entry><literal>= (inet, inet)</literal></entry></row>
- <row><entry><literal><> (inet, inet)</literal></entry></row>
- <row><entry><literal>< (inet, inet)</literal></entry></row>
- <row><entry><literal><= (inet, inet)</literal></entry></row>
- <row><entry><literal>> (inet, inet)</literal></entry></row>
- <row><entry><literal>>= (inet, inet)</literal></entry></row>
- <row><entry><literal>&& (inet, inet)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="17"><literal>multirange_ops</literal></entry>
- <entry><literal>= (anymultirange, anymultirange)</literal></entry>
- <entry valign="middle" morerows="17"></entry>
- </row>
- <row><entry><literal>&& (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal>&& (anymultirange, anyrange)</literal></entry></row>
- <row><entry><literal>@> (anymultirange, anyelement)</literal></entry></row>
- <row><entry><literal>@> (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal>@> (anymultirange, anyrange)</literal></entry></row>
- <row><entry><literal><@ (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal><@ (anymultirange, anyrange)</literal></entry></row>
- <row><entry><literal><< (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal><< (anymultirange, anyrange)</literal></entry></row>
- <row><entry><literal>>> (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal>>> (anymultirange, anyrange)</literal></entry></row>
- <row><entry><literal>&< (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal>&< (anymultirange, anyrange)</literal></entry></row>
- <row><entry><literal>&> (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal>&> (anymultirange, anyrange)</literal></entry></row>
- <row><entry><literal>-|- (anymultirange, anymultirange)</literal></entry></row>
- <row><entry><literal>-|- (anymultirange, anyrange)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="7"><literal>point_ops</literal></entry>
- <entry><literal>|>> (point, point)</literal></entry>
- <entry valign="middle" morerows="7"><literal><-> (point, point)</literal></entry>
- </row>
- <row><entry><literal><< (point, point)</literal></entry></row>
- <row><entry><literal>>> (point, point)</literal></entry></row>
- <row><entry><literal><<| (point, point)</literal></entry></row>
- <row><entry><literal>~= (point, point)</literal></entry></row>
- <row><entry><literal><@ (point, box)</literal></entry></row>
- <row><entry><literal><@ (point, polygon)</literal></entry></row>
- <row><entry><literal><@ (point, circle)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="13"><literal>poly_ops</literal></entry>
- <entry><literal><< (polygon, polygon)</literal></entry>
- <entry valign="middle" morerows="13"><literal><-> (polygon, point)</literal></entry>
- </row>
- <row><entry><literal>&< (polygon, polygon)</literal></entry></row>
- <row><entry><literal>&> (polygon, polygon)</literal></entry></row>
- <row><entry><literal>>> (polygon, polygon)</literal></entry></row>
- <row><entry><literal><@ (polygon, polygon)</literal></entry></row>
- <row><entry><literal>@> (polygon, polygon)</literal></entry></row>
- <row><entry><literal>~= (polygon, polygon)</literal></entry></row>
- <row><entry><literal>&& (polygon, polygon)</literal></entry></row>
- <row><entry><literal><<| (polygon, polygon)</literal></entry></row>
- <row><entry><literal>&<| (polygon, polygon)</literal></entry></row>
- <row><entry><literal>|&> (polygon, polygon)</literal></entry></row>
- <row><entry><literal>|>> (polygon, polygon)</literal></entry></row>
- <row><entry><literal>@ (polygon, polygon)</literal></entry></row>
- <row><entry><literal>~ (polygon, polygon)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="17"><literal>range_ops</literal></entry>
- <entry><literal>= (anyrange, anyrange)</literal></entry>
- <entry valign="middle" morerows="17"></entry>
- </row>
- <row><entry><literal>&& (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal>&& (anyrange, anymultirange)</literal></entry></row>
- <row><entry><literal>@> (anyrange, anyelement)</literal></entry></row>
- <row><entry><literal>@> (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal>@> (anyrange, anymultirange)</literal></entry></row>
- <row><entry><literal><@ (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal><@ (anyrange, anymultirange)</literal></entry></row>
- <row><entry><literal><< (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal><< (anyrange, anymultirange)</literal></entry></row>
- <row><entry><literal>>> (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal>>> (anyrange, anymultirange)</literal></entry></row>
- <row><entry><literal>&< (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal>&< (anyrange, anymultirange)</literal></entry></row>
- <row><entry><literal>&> (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal>&> (anyrange, anymultirange)</literal></entry></row>
- <row><entry><literal>-|- (anyrange, anyrange)</literal></entry></row>
- <row><entry><literal>-|- (anyrange, anymultirange)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="1"><literal>tsquery_ops</literal></entry>
- <entry><literal><@ (tsquery, tsquery)</literal></entry>
- <entry valign="middle" morerows="1"></entry>
- </row>
- <row><entry><literal>@> (tsquery, tsquery)</literal></entry></row>
- <row>
- <entry valign="middle"><literal>tsvector_ops</literal></entry>
- <entry><literal>@@ (tsvector, tsquery)</literal></entry>
- <entry></entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- For historical reasons, the <literal>inet_ops</literal> operator class is
- not the default class for types <type>inet</type> and <type>cidr</type>.
- To use it, mention the class name in <command>CREATE INDEX</command>,
- for example
-<programlisting>
-CREATE INDEX ON my_table USING GIST (my_inet_column inet_ops);
-</programlisting>
- </para>
-
-</sect1>
-
-<sect1 id="gist-extensibility">
- <title>Extensibility</title>
-
- <para>
- Traditionally, implementing a new index access method meant a lot of
- difficult work. It was necessary to understand the inner workings of the
- database, such as the lock manager and Write-Ahead Log. The
- <acronym>GiST</acronym> interface has a high level of abstraction,
- requiring the access method implementer only to implement the semantics of
- the data type being accessed. The <acronym>GiST</acronym> layer itself
- takes care of concurrency, logging and searching the tree structure.
- </para>
-
- <para>
- This extensibility should not be confused with the extensibility of the
- other standard search trees in terms of the data they can handle. For
- example, <productname>PostgreSQL</productname> supports extensible B-trees
- and hash indexes. That means that you can use
- <productname>PostgreSQL</productname> to build a B-tree or hash over any
- data type you want. But B-trees only support range predicates
- (<literal><</literal>, <literal>=</literal>, <literal>></literal>),
- and hash indexes only support equality queries.
- </para>
-
- <para>
- So if you index, say, an image collection with a
- <productname>PostgreSQL</productname> B-tree, you can only issue queries
- such as <quote>is imagex equal to imagey</quote>, <quote>is imagex less
- than imagey</quote> and <quote>is imagex greater than imagey</quote>.
- Depending on how you define <quote>equals</quote>, <quote>less than</quote>
- and <quote>greater than</quote> in this context, this could be useful.
- However, by using a <acronym>GiST</acronym> based index, you could create
- ways to ask domain-specific questions, perhaps <quote>find all images of
- horses</quote> or <quote>find all over-exposed images</quote>.
- </para>
-
- <para>
- All it takes to get a <acronym>GiST</acronym> access method up and running
- is to implement several user-defined methods, which define the behavior of
- keys in the tree. Of course these methods have to be pretty fancy to
- support fancy queries, but for all the standard queries (B-trees,
- R-trees, etc.) they're relatively straightforward. In short,
- <acronym>GiST</acronym> combines extensibility along with generality, code
- reuse, and a clean interface.
- </para>
-
- <para>
- There are five methods that an index operator class for
- <acronym>GiST</acronym> must provide, and seven that are optional.
- Correctness of the index is ensured
- by proper implementation of the <function>same</function>, <function>consistent</function>
- and <function>union</function> methods, while efficiency (size and speed) of the
- index will depend on the <function>penalty</function> and <function>picksplit</function>
- methods.
- Two optional methods are <function>compress</function> and
- <function>decompress</function>, which allow an index to have internal tree data of
- a different type than the data it indexes. The leaves are to be of the
- indexed data type, while the other tree nodes can be of any C struct (but
- you still have to follow <productname>PostgreSQL</productname> data type rules here,
- see about <literal>varlena</literal> for variable sized data). If the tree's
- internal data type exists at the SQL level, the <literal>STORAGE</literal> option
- of the <command>CREATE OPERATOR CLASS</command> command can be used.
- The optional eighth method is <function>distance</function>, which is needed
- if the operator class wishes to support ordered scans (nearest-neighbor
- searches). The optional ninth method <function>fetch</function> is needed if the
- operator class wishes to support index-only scans, except when the
- <function>compress</function> method is omitted. The optional tenth method
- <function>options</function> is needed if the operator class has
- user-specified parameters.
- The optional eleventh method <function>sortsupport</function> is used to
- speed up building a <acronym>GiST</acronym> index.
- The optional twelfth method <function>stratnum</function> is used to
- translate well-known <literal>RT*StrategyNumber</literal>s (from
- <filename>src/include/access/stratnum.h</filename>) into strategy numbers
- used by the operator class. This lets the core code look up operators for
- temporal constraint indexes.
- </para>
-
- <variablelist>
- <varlistentry>
- <term><function>consistent</function></term>
- <listitem>
- <para>
- Given an index entry <literal>p</literal> and a query value <literal>q</literal>,
- this function determines whether the index entry is
- <quote>consistent</quote> with the query; that is, could the predicate
- <quote><replaceable>indexed_column</replaceable>
- <replaceable>indexable_operator</replaceable> <literal>q</literal></quote> be true for
- any row represented by the index entry? For a leaf index entry this is
- equivalent to testing the indexable condition, while for an internal
- tree node this determines whether it is necessary to scan the subtree
- of the index represented by the tree node. When the result is
- <literal>true</literal>, a <literal>recheck</literal> flag must also be returned.
- This indicates whether the predicate is certainly true or only possibly
- true. If <literal>recheck</literal> = <literal>false</literal> then the index has
- tested the predicate condition exactly, whereas if <literal>recheck</literal>
- = <literal>true</literal> the row is only a candidate match. In that case the
- system will automatically evaluate the
- <replaceable>indexable_operator</replaceable> against the actual row value to see
- if it is really a match. This convention allows
- <acronym>GiST</acronym> to support both lossless and lossy index
- structures.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_consistent(internal, data_type, smallint, oid, internal)
-RETURNS bool
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_consistent);
-
-Datum
-my_consistent(PG_FUNCTION_ARGS)
-{
- GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
- data_type *query = PG_GETARG_DATA_TYPE_P(1);
- StrategyNumber strategy = (StrategyNumber) PG_GETARG_UINT16(2);
- /* Oid subtype = PG_GETARG_OID(3); */
- bool *recheck = (bool *) PG_GETARG_POINTER(4);
- data_type *key = DatumGetDataType(entry->key);
- bool retval;
-
- /*
- * determine return value as a function of strategy, key and query.
- *
- * Use GIST_LEAF(entry) to know where you're called in the index tree,
- * which comes handy when supporting the = operator for example (you could
- * check for non empty union() in non-leaf nodes and equality in leaf
- * nodes).
- */
-
- *recheck = true; /* or false if check is exact */
-
- PG_RETURN_BOOL(retval);
-}
-</programlisting>
-
- Here, <varname>key</varname> is an element in the index and <varname>query</varname>
- the value being looked up in the index. The <literal>StrategyNumber</literal>
- parameter indicates which operator of your operator class is being
- applied — it matches one of the operator numbers in the
- <command>CREATE OPERATOR CLASS</command> command.
- </para>
-
- <para>
- Depending on which operators you have included in the class, the data
- type of <varname>query</varname> could vary with the operator, since it will
- be whatever type is on the right-hand side of the operator, which might
- be different from the indexed data type appearing on the left-hand side.
- (The above code skeleton assumes that only one type is possible; if
- not, fetching the <varname>query</varname> argument value would have to depend
- on the operator.) It is recommended that the SQL declaration of
- the <function>consistent</function> function use the opclass's indexed data
- type for the <varname>query</varname> argument, even though the actual type
- might be something else depending on the operator.
- </para>
-
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>union</function></term>
- <listitem>
- <para>
- This method consolidates information in the tree. Given a set of
- entries, this function generates a new index entry that represents
- all the given entries.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_union(internal, internal)
-RETURNS storage_type
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_union);
-
-Datum
-my_union(PG_FUNCTION_ARGS)
-{
- GistEntryVector *entryvec = (GistEntryVector *) PG_GETARG_POINTER(0);
- GISTENTRY *ent = entryvec->vector;
- data_type *out,
- *tmp,
- *old;
- int numranges,
- i = 0;
-
- numranges = entryvec->n;
- tmp = DatumGetDataType(ent[0].key);
- out = tmp;
-
- if (numranges == 1)
- {
- out = data_type_deep_copy(tmp);
-
- PG_RETURN_DATA_TYPE_P(out);
- }
-
- for (i = 1; i < numranges; i++)
- {
- old = out;
- tmp = DatumGetDataType(ent[i].key);
- out = my_union_implementation(out, tmp);
- }
-
- PG_RETURN_DATA_TYPE_P(out);
-}
-</programlisting>
- </para>
-
- <para>
- As you can see, in this skeleton we're dealing with a data type
- where <literal>union(X, Y, Z) = union(union(X, Y), Z)</literal>. It's easy
- enough to support data types where this is not the case, by
- implementing the proper union algorithm in this
- <acronym>GiST</acronym> support method.
- </para>
-
- <para>
- The result of the <function>union</function> function must be a value of the
- index's storage type, whatever that is (it might or might not be
- different from the indexed column's type). The <function>union</function>
- function should return a pointer to newly <function>palloc()</function>ed
- memory. You can't just return the input value as-is, even if there is
- no type change.
- </para>
-
- <para>
- As shown above, the <function>union</function> function's
- first <type>internal</type> argument is actually
- a <structname>GistEntryVector</structname> pointer. The second argument is a
- pointer to an integer variable, which can be ignored. (It used to be
- required that the <function>union</function> function store the size of its
- result value into that variable, but this is no longer necessary.)
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>compress</function></term>
- <listitem>
- <para>
- Converts a data item into a format suitable for physical storage in
- an index page.
- If the <function>compress</function> method is omitted, data items are stored
- in the index without modification.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_compress(internal)
-RETURNS internal
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_compress);
-
-Datum
-my_compress(PG_FUNCTION_ARGS)
-{
- GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
- GISTENTRY *retval;
-
- if (entry->leafkey)
- {
- /* replace entry->key with a compressed version */
- compressed_data_type *compressed_data = palloc(sizeof(compressed_data_type));
-
- /* fill *compressed_data from entry->key ... */
-
- retval = palloc(sizeof(GISTENTRY));
- gistentryinit(*retval, PointerGetDatum(compressed_data),
- entry->rel, entry->page, entry->offset, FALSE);
- }
- else
- {
- /* typically we needn't do anything with non-leaf entries */
- retval = entry;
- }
-
- PG_RETURN_POINTER(retval);
-}
-</programlisting>
- </para>
-
- <para>
- You have to adapt <replaceable>compressed_data_type</replaceable> to the specific
- type you're converting to in order to compress your leaf nodes, of
- course.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>decompress</function></term>
- <listitem>
- <para>
- Converts the stored representation of a data item into a format that
- can be manipulated by the other GiST methods in the operator class.
- If the <function>decompress</function> method is omitted, it is assumed that
- the other GiST methods can work directly on the stored data format.
- (<function>decompress</function> is not necessarily the reverse of
- the <function>compress</function> method; in particular,
- if <function>compress</function> is lossy then it's impossible
- for <function>decompress</function> to exactly reconstruct the original
- data. <function>decompress</function> is not necessarily equivalent
- to <function>fetch</function>, either, since the other GiST methods might not
- require full reconstruction of the data.)
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_decompress(internal)
-RETURNS internal
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_decompress);
-
-Datum
-my_decompress(PG_FUNCTION_ARGS)
-{
- PG_RETURN_POINTER(PG_GETARG_POINTER(0));
-}
-</programlisting>
-
- The above skeleton is suitable for the case where no decompression
- is needed. (But, of course, omitting the method altogether is even
- easier, and is recommended in such cases.)
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>penalty</function></term>
- <listitem>
- <para>
- Returns a value indicating the <quote>cost</quote> of inserting the new
- entry into a particular branch of the tree. Items will be inserted
- down the path of least <function>penalty</function> in the tree.
- Values returned by <function>penalty</function> should be non-negative.
- If a negative value is returned, it will be treated as zero.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_penalty(internal, internal, internal)
-RETURNS internal
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT; -- in some cases penalty functions need not be strict
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_penalty);
-
-Datum
-my_penalty(PG_FUNCTION_ARGS)
-{
- GISTENTRY *origentry = (GISTENTRY *) PG_GETARG_POINTER(0);
- GISTENTRY *newentry = (GISTENTRY *) PG_GETARG_POINTER(1);
- float *penalty = (float *) PG_GETARG_POINTER(2);
- data_type *orig = DatumGetDataType(origentry->key);
- data_type *new = DatumGetDataType(newentry->key);
-
- *penalty = my_penalty_implementation(orig, new);
- PG_RETURN_POINTER(penalty);
-}
-</programlisting>
-
- For historical reasons, the <function>penalty</function> function doesn't
- just return a <type>float</type> result; instead it has to store the value
- at the location indicated by the third argument. The return
- value per se is ignored, though it's conventional to pass back the
- address of that argument.
- </para>
-
- <para>
- The <function>penalty</function> function is crucial to good performance of
- the index. It'll get used at insertion time to determine which branch
- to follow when choosing where to add the new entry in the tree. At
- query time, the more balanced the index, the quicker the lookup.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>picksplit</function></term>
- <listitem>
- <para>
- When an index page split is necessary, this function decides which
- entries on the page are to stay on the old page, and which are to move
- to the new page.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_picksplit(internal, internal)
-RETURNS internal
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_picksplit);
-
-Datum
-my_picksplit(PG_FUNCTION_ARGS)
-{
- GistEntryVector *entryvec = (GistEntryVector *) PG_GETARG_POINTER(0);
- GIST_SPLITVEC *v = (GIST_SPLITVEC *) PG_GETARG_POINTER(1);
- OffsetNumber maxoff = entryvec->n - 1;
- GISTENTRY *ent = entryvec->vector;
- int i,
- nbytes;
- OffsetNumber *left,
- *right;
- data_type *tmp_union;
- data_type *unionL;
- data_type *unionR;
- GISTENTRY **raw_entryvec;
-
- maxoff = entryvec->n - 1;
- nbytes = (maxoff + 1) * sizeof(OffsetNumber);
-
- v->spl_left = (OffsetNumber *) palloc(nbytes);
- left = v->spl_left;
- v->spl_nleft = 0;
-
- v->spl_right = (OffsetNumber *) palloc(nbytes);
- right = v->spl_right;
- v->spl_nright = 0;
-
- unionL = NULL;
- unionR = NULL;
-
- /* Initialize the raw entry vector. */
- raw_entryvec = (GISTENTRY **) malloc(entryvec->n * sizeof(void *));
- for (i = FirstOffsetNumber; i <= maxoff; i = OffsetNumberNext(i))
- raw_entryvec[i] = &(entryvec->vector[i]);
-
- for (i = FirstOffsetNumber; i <= maxoff; i = OffsetNumberNext(i))
- {
- int real_index = raw_entryvec[i] - entryvec->vector;
-
- tmp_union = DatumGetDataType(entryvec->vector[real_index].key);
- Assert(tmp_union != NULL);
-
- /*
- * Choose where to put the index entries and update unionL and unionR
- * accordingly. Append the entries to either v->spl_left or
- * v->spl_right, and care about the counters.
- */
-
- if (my_choice_is_left(unionL, curl, unionR, curr))
- {
- if (unionL == NULL)
- unionL = tmp_union;
- else
- unionL = my_union_implementation(unionL, tmp_union);
-
- *left = real_index;
- ++left;
- ++(v->spl_nleft);
- }
- else
- {
- /*
- * Same on the right
- */
- }
- }
-
- v->spl_ldatum = DataTypeGetDatum(unionL);
- v->spl_rdatum = DataTypeGetDatum(unionR);
- PG_RETURN_POINTER(v);
-}
-</programlisting>
-
- Notice that the <function>picksplit</function> function's result is delivered
- by modifying the passed-in <structname>v</structname> structure. The return
- value per se is ignored, though it's conventional to pass back the
- address of <structname>v</structname>.
- </para>
-
- <para>
- Like <function>penalty</function>, the <function>picksplit</function> function
- is crucial to good performance of the index. Designing suitable
- <function>penalty</function> and <function>picksplit</function> implementations
- is where the challenge of implementing well-performing
- <acronym>GiST</acronym> indexes lies.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>same</function></term>
- <listitem>
- <para>
- Returns true if two index entries are identical, false otherwise.
- (An <quote>index entry</quote> is a value of the index's storage type,
- not necessarily the original indexed column's type.)
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_same(storage_type, storage_type, internal)
-RETURNS internal
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_same);
-
-Datum
-my_same(PG_FUNCTION_ARGS)
-{
- prefix_range *v1 = PG_GETARG_PREFIX_RANGE_P(0);
- prefix_range *v2 = PG_GETARG_PREFIX_RANGE_P(1);
- bool *result = (bool *) PG_GETARG_POINTER(2);
-
- *result = my_eq(v1, v2);
- PG_RETURN_POINTER(result);
-}
-</programlisting>
-
- For historical reasons, the <function>same</function> function doesn't
- just return a Boolean result; instead it has to store the flag
- at the location indicated by the third argument. The return
- value per se is ignored, though it's conventional to pass back the
- address of that argument.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>distance</function></term>
- <listitem>
- <para>
- Given an index entry <literal>p</literal> and a query value <literal>q</literal>,
- this function determines the index entry's
- <quote>distance</quote> from the query value. This function must be
- supplied if the operator class contains any ordering operators.
- A query using the ordering operator will be implemented by returning
- index entries with the smallest <quote>distance</quote> values first,
- so the results must be consistent with the operator's semantics.
- For a leaf index entry the result just represents the distance to
- the index entry; for an internal tree node, the result must be the
- smallest distance that any child entry could have.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_distance(internal, data_type, smallint, oid, internal)
-RETURNS float8
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- And the matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_distance);
-
-Datum
-my_distance(PG_FUNCTION_ARGS)
-{
- GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
- data_type *query = PG_GETARG_DATA_TYPE_P(1);
- StrategyNumber strategy = (StrategyNumber) PG_GETARG_UINT16(2);
- /* Oid subtype = PG_GETARG_OID(3); */
- /* bool *recheck = (bool *) PG_GETARG_POINTER(4); */
- data_type *key = DatumGetDataType(entry->key);
- double retval;
-
- /*
- * determine return value as a function of strategy, key and query.
- */
-
- PG_RETURN_FLOAT8(retval);
-}
-</programlisting>
-
- The arguments to the <function>distance</function> function are identical to
- the arguments of the <function>consistent</function> function.
- </para>
-
- <para>
- Some approximation is allowed when determining the distance, so long
- as the result is never greater than the entry's actual distance. Thus,
- for example, distance to a bounding box is usually sufficient in
- geometric applications. For an internal tree node, the distance
- returned must not be greater than the distance to any of the child
- nodes. If the returned distance is not exact, the function must set
- <literal>*recheck</literal> to true. (This is not necessary for internal tree
- nodes; for them, the calculation is always assumed to be inexact.) In
- this case the executor will calculate the accurate distance after
- fetching the tuple from the heap, and reorder the tuples if necessary.
- </para>
-
- <para>
- If the distance function returns <literal>*recheck = true</literal> for any
- leaf node, the original ordering operator's return type must
- be <type>float8</type> or <type>float4</type>, and the distance function's
- result values must be comparable to those of the original ordering
- operator, since the executor will sort using both distance function
- results and recalculated ordering-operator results. Otherwise, the
- distance function's result values can be any finite <type>float8</type>
- values, so long as the relative order of the result values matches the
- order returned by the ordering operator. (Infinity and minus infinity
- are used internally to handle cases such as nulls, so it is not
- recommended that <function>distance</function> functions return these values.)
- </para>
-
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>fetch</function></term>
- <listitem>
- <para>
- Converts the compressed index representation of a data item into the
- original data type, for index-only scans. The returned data must be an
- exact, non-lossy copy of the originally indexed value.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_fetch(internal)
-RETURNS internal
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- The argument is a pointer to a <structname>GISTENTRY</structname> struct. On
- entry, its <structfield>key</structfield> field contains a non-NULL leaf datum in
- compressed form. The return value is another <structname>GISTENTRY</structname>
- struct, whose <structfield>key</structfield> field contains the same datum in its
- original, uncompressed form. If the opclass's compress function does
- nothing for leaf entries, the <function>fetch</function> method can return the
- argument as-is. Or, if the opclass does not have a compress function,
- the <function>fetch</function> method can be omitted as well, since it would
- necessarily be a no-op.
- </para>
-
- <para>
- The matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_fetch);
-
-Datum
-my_fetch(PG_FUNCTION_ARGS)
-{
- GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
- input_data_type *in = DatumGetPointer(entry->key);
- fetched_data_type *fetched_data;
- GISTENTRY *retval;
-
- retval = palloc(sizeof(GISTENTRY));
- fetched_data = palloc(sizeof(fetched_data_type));
-
- /*
- * Convert 'fetched_data' into the a Datum of the original datatype.
- */
-
- /* fill *retval from fetched_data. */
- gistentryinit(*retval, PointerGetDatum(converted_datum),
- entry->rel, entry->page, entry->offset, FALSE);
-
- PG_RETURN_POINTER(retval);
-}
-</programlisting>
- </para>
-
- <para>
- If the compress method is lossy for leaf entries, the operator class
- cannot support index-only scans, and must not define
- a <function>fetch</function> function.
- </para>
-
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>options</function></term>
- <listitem>
- <para>
- Allows definition of user-visible parameters that control operator
- class behavior.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_options(internal)
-RETURNS void
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
- </para>
-
- <para>
- The function is passed a pointer to a <structname>local_relopts</structname>
- struct, which needs to be filled with a set of operator class
- specific options. The options can be accessed from other support
- functions using the <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
- <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
- </para>
-
- <para>
- An example implementation of my_options() and parameters use
- from other support functions are given below:
-
-<programlisting>
-typedef enum MyEnumType
-{
- MY_ENUM_ON,
- MY_ENUM_OFF,
- MY_ENUM_AUTO
-} MyEnumType;
-
-typedef struct
-{
- int32 vl_len_; /* varlena header (do not touch directly!) */
- int int_param; /* integer parameter */
- double real_param; /* real parameter */
- MyEnumType enum_param; /* enum parameter */
- int str_param; /* string parameter */
-} MyOptionsStruct;
-
-/* String representation of enum values */
-static relopt_enum_elt_def myEnumValues[] =
-{
- {"on", MY_ENUM_ON},
- {"off", MY_ENUM_OFF},
- {"auto", MY_ENUM_AUTO},
- {(const char *) NULL} /* list terminator */
-};
-
-static char *str_param_default = "default";
-
-/*
- * Sample validator: checks that string is not longer than 8 bytes.
- */
-static void
-validate_my_string_relopt(const char *value)
-{
- if (strlen(value) > 8)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("str_param must be at most 8 bytes")));
-}
-
-/*
- * Sample filler: switches characters to lower case.
- */
-static Size
-fill_my_string_relopt(const char *value, void *ptr)
-{
- char *tmp = str_tolower(value, strlen(value), DEFAULT_COLLATION_OID);
- int len = strlen(tmp);
-
- if (ptr)
- strcpy((char *) ptr, tmp);
-
- pfree(tmp);
- return len + 1;
-}
-
-PG_FUNCTION_INFO_V1(my_options);
-
-Datum
-my_options(PG_FUNCTION_ARGS)
-{
- local_relopts *relopts = (local_relopts *) PG_GETARG_POINTER(0);
-
- init_local_reloptions(relopts, sizeof(MyOptionsStruct));
- add_local_int_reloption(relopts, "int_param", "integer parameter",
- 100, 0, 1000000,
- offsetof(MyOptionsStruct, int_param));
- add_local_real_reloption(relopts, "real_param", "real parameter",
- 1.0, 0.0, 1000000.0,
- offsetof(MyOptionsStruct, real_param));
- add_local_enum_reloption(relopts, "enum_param", "enum parameter",
- myEnumValues, MY_ENUM_ON,
- "Valid values are: \"on\", \"off\" and \"auto\".",
- offsetof(MyOptionsStruct, enum_param));
- add_local_string_reloption(relopts, "str_param", "string parameter",
- str_param_default,
- &validate_my_string_relopt,
- &fill_my_string_relopt,
- offsetof(MyOptionsStruct, str_param));
-
- PG_RETURN_VOID();
-}
-
-PG_FUNCTION_INFO_V1(my_compress);
-
-Datum
-my_compress(PG_FUNCTION_ARGS)
-{
- int int_param = 100;
- double real_param = 1.0;
- MyEnumType enum_param = MY_ENUM_ON;
- char *str_param = str_param_default;
-
- /*
- * Normally, when opclass contains 'options' method, then options are always
- * passed to support functions. However, if you add 'options' method to
- * existing opclass, previously defined indexes have no options, so the
- * check is required.
- */
- if (PG_HAS_OPCLASS_OPTIONS())
- {
- MyOptionsStruct *options = (MyOptionsStruct *) PG_GET_OPCLASS_OPTIONS();
-
- int_param = options->int_param;
- real_param = options->real_param;
- enum_param = options->enum_param;
- str_param = GET_STRING_RELOPTION(options, str_param);
- }
-
- /* the rest implementation of support function */
-}
-
-</programlisting>
- </para>
-
- <para>
- Since the representation of the key in <acronym>GiST</acronym> is
- flexible, it may depend on user-specified parameters. For instance,
- the length of key signature may be specified. See
- <literal>gtsvector_options()</literal> for example.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>sortsupport</function></term>
- <listitem>
- <para>
- Returns a comparator function to sort data in a way that preserves
- locality. It is used by <command>CREATE INDEX</command> and
- <command>REINDEX</command> commands. The quality of the created index
- depends on how well the sort order determined by the comparator function
- preserves locality of the inputs.
- </para>
- <para>
- The <function>sortsupport</function> method is optional. If it is not
- provided, <command>CREATE INDEX</command> builds the index by inserting
- each tuple to the tree using the <function>penalty</function> and
- <function>picksplit</function> functions, which is much slower.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like
- this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_sortsupport(internal)
-RETURNS void
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
-
- The argument is a pointer to a <structname>SortSupport</structname>
- struct. At a minimum, the function must fill in its comparator field.
- The comparator takes three arguments: two Datums to compare, and
- a pointer to the <structname>SortSupport</structname> struct. The
- Datums are the two indexed values in the format that they are stored
- in the index; that is, in the format returned by the
- <function>compress</function> method. The full API is defined in
- <filename>src/include/utils/sortsupport.h</filename>.
- </para>
-
- <para>
- The matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_sortsupport);
-
-static int
-my_fastcmp(Datum x, Datum y, SortSupport ssup)
-{
- /* establish order between x and y by computing some sorting value z */
-
- int z1 = ComputeSpatialCode(x);
- int z2 = ComputeSpatialCode(y);
-
- return z1 == z2 ? 0 : z1 > z2 ? 1 : -1;
-}
-
-Datum
-my_sortsupport(PG_FUNCTION_ARGS)
-{
- SortSupport ssup = (SortSupport) PG_GETARG_POINTER(0);
-
- ssup->comparator = my_fastcmp;
- PG_RETURN_VOID();
-}
-</programlisting>
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>stratnum</function></term>
- <listitem>
- <para>
- Given an <literal>RT*StrategyNumber</literal> value from
- <filename>src/include/access/stratnum.h</filename>, returns a strategy
- number used by this operator class for matching functionality. The
- function should return <literal>InvalidStrategy</literal> if the
- operator class has no matching strategy.
- </para>
-
- <para>
- This is used for temporal index constraints (i.e., <literal>PRIMARY
- KEY</literal> and <literal>UNIQUE</literal>). If the operator class
- provides this function and it returns results for
- <literal>RTEqualStrategyNumber</literal>, it can be used in the
- non-<literal>WITHOUT OVERLAPS</literal> part(s) of an index constraint.
- If it returns results for <literal>RTOverlapStrategyNumber</literal>,
- the operator class can be used in the <literal>WITHOUT
- OVERLAPS</literal> part of an index constraint.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like
- this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_stratnum(integer)
-RETURNS integer
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
- </para>
-
- <para>
- The matching code in the C module could then follow this skeleton:
-
-<programlisting>
-PG_FUNCTION_INFO_V1(my_stratnum);
-
-Datum
-my_stratnum(PG_FUNCTION_ARGS)
-{
- StrategyNumber strategy = (StrategyNumber) PG_GETARG_UINT16(1);
- StrategyNumber ret = InvalidStrategy;
-
- switch (strategy)
- {
- case RTEqualStrategyNumber:
- ret = BTEqualStrategyNumber;
- }
-
- PG_RETURN_UINT16(ret);
-}
-</programlisting>
- </para>
-
- <para>
- One translation function is provided by
- <productname>PostgreSQL</productname>:
- <literal>gist_stratnum_identity</literal> is for operator classes that
- already use the <literal>RT*StrategyNumber</literal> constants. It
- returns whatever is passed to it. The <literal>btree_gist</literal>
- extension defines a second translation function,
- <literal>gist_stratnum_btree</literal>, for operator classes that use
- the <literal>BT*StrategyNumber</literal> constants.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <para>
- All the GiST support methods are normally called in short-lived memory
- contexts; that is, <varname>CurrentMemoryContext</varname> will get reset after
- each tuple is processed. It is therefore not very important to worry about
- pfree'ing everything you palloc. However, in some cases it's useful for a
- support method to cache data across repeated calls. To do that, allocate
- the longer-lived data in <literal>fcinfo->flinfo->fn_mcxt</literal>, and
- keep a pointer to it in <literal>fcinfo->flinfo->fn_extra</literal>. Such
- data will survive for the life of the index operation (e.g., a single GiST
- index scan, index build, or index tuple insertion). Be careful to pfree
- the previous value when replacing a <literal>fn_extra</literal> value, or the leak
- will accumulate for the duration of the operation.
- </para>
-
-</sect1>
-
-<sect1 id="gist-implementation">
- <title>Implementation</title>
-
- <sect2 id="gist-buffering-build">
- <title>GiST Index Build Methods</title>
-
- <para>
- The simplest way to build a GiST index is just to insert all the entries,
- one by one. This tends to be slow for large indexes, because if the
- index tuples are scattered across the index and the index is large enough
- to not fit in cache, a lot of random I/O will be
- needed. <productname>PostgreSQL</productname> supports two alternative
- methods for initial build of a GiST index: <firstterm>sorted</firstterm>
- and <firstterm>buffered</firstterm> modes.
- </para>
-
- <para>
- The sorted method is only available if each of the opclasses used by the
- index provides a <function>sortsupport</function> function, as described
- in <xref linkend="gist-extensibility"/>. If they do, this method is
- usually the best, so it is used by default.
- </para>
-
- <para>
- The buffered method works by not inserting tuples directly into the index
- right away. It can dramatically reduce the amount of random I/O needed
- for non-ordered data sets. For well-ordered data sets the benefit is
- smaller or non-existent, because only a small number of pages receive new
- tuples at a time, and those pages fit in cache even if the index as a
- whole does not.
- </para>
-
- <para>
- The buffered method needs to call the <function>penalty</function>
- function more often than the simple method does, which consumes some
- extra CPU resources. Also, the buffers need temporary disk space, up to
- the size of the resulting index. Buffering can also influence the quality
- of the resulting index, in both positive and negative directions. That
- influence depends on various factors, like the distribution of the input
- data and the operator class implementation.
- </para>
-
- <para>
- If sorting is not possible, then by default a GiST index build switches
- to the buffering method when the index size reaches
- <xref linkend="guc-effective-cache-size"/>. Buffering can be manually
- forced or prevented by the <literal>buffering</literal> parameter to the
- CREATE INDEX command. The default behavior is good for most cases, but
- turning buffering off might speed up the build somewhat if the input data
- is ordered.
- </para>
-
- </sect2>
-</sect1>
-
-<sect1 id="gist-examples">
- <title>Examples</title>
-
- <para>
- The <productname>PostgreSQL</productname> source distribution includes
- several examples of index methods implemented using
- <acronym>GiST</acronym>. The core system currently provides text search
- support (indexing for <type>tsvector</type> and <type>tsquery</type>) as well as
- R-Tree equivalent functionality for some of the built-in geometric data types
- (see <filename>src/backend/access/gist/gistproc.c</filename>). The following
- <filename>contrib</filename> modules also contain <acronym>GiST</acronym>
- operator classes:
-
- <variablelist>
- <varlistentry>
- <term><filename>btree_gist</filename></term>
- <listitem>
- <para>B-tree equivalent functionality for several data types</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>cube</filename></term>
- <listitem>
- <para>Indexing for multidimensional cubes</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>hstore</filename></term>
- <listitem>
- <para>Module for storing (key, value) pairs</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>intarray</filename></term>
- <listitem>
- <para>RD-Tree for one-dimensional array of int4 values</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>ltree</filename></term>
- <listitem>
- <para>Indexing for tree-like structures</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>pg_trgm</filename></term>
- <listitem>
- <para>Text similarity using trigram matching</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><filename>seg</filename></term>
- <listitem>
- <para>Indexing for <quote>float ranges</quote></para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
-</sect1>
-
-</chapter>
diff --git a/doc/src/sgml/hash.sgml b/doc/src/sgml/hash.sgml
deleted file mode 100644
index e35911ebf8..0000000000
--- a/doc/src/sgml/hash.sgml
+++ /dev/null
@@ -1,162 +0,0 @@
-<!-- doc/src/sgml/hash.sgml -->
-
-<chapter id="hash-index">
-<title>Hash Indexes</title>
-
- <indexterm>
- <primary>index</primary>
- <secondary>Hash</secondary>
- </indexterm>
-
-<sect1 id="hash-intro">
- <title>Overview</title>
-
- <para>
- <productname>PostgreSQL</productname>
- includes an implementation of persistent on-disk hash indexes,
- which are fully crash recoverable. Any data type can be indexed by a
- hash index, including data types that do not have a well-defined linear
- ordering. Hash indexes store only the hash value of the data being
- indexed, thus there are no restrictions on the size of the data column
- being indexed.
- </para>
-
- <para>
- Hash indexes support only single-column indexes and do not allow
- uniqueness checking.
- </para>
-
- <para>
- Hash indexes support only the <literal>=</literal> operator,
- so WHERE clauses that specify range operations will not be able to take
- advantage of hash indexes.
- </para>
-
- <para>
- Each hash index tuple stores just the 4-byte hash value, not the actual
- column value. As a result, hash indexes may be much smaller than B-trees
- when indexing longer data items such as UUIDs, URLs, etc. The absence of
- the column value also makes all hash index scans lossy. Hash indexes may
- take part in bitmap index scans and backward scans.
- </para>
-
- <para>
- Hash indexes are best optimized for SELECT and UPDATE-heavy workloads
- that use equality scans on larger tables. In a B-tree index, searches must
- descend through the tree until the leaf page is found. In tables with
- millions of rows, this descent can increase access time to data. The
- equivalent of a leaf page in a hash index is referred to as a bucket page. In
- contrast, a hash index allows accessing the bucket pages directly,
- thereby potentially reducing index access time in larger tables. This
- reduction in "logical I/O" becomes even more pronounced on indexes/data
- larger than shared_buffers/RAM.
- </para>
-
- <para>
- Hash indexes have been designed to cope with uneven distributions of
- hash values. Direct access to the bucket pages works well if the hash
- values are evenly distributed. When inserts mean that the bucket page
- becomes full, additional overflow pages are chained to that specific
- bucket page, locally expanding the storage for index tuples that match
- that hash value. When scanning a hash bucket during queries, we need to
- scan through all of the overflow pages. Thus an unbalanced hash index
- might actually be worse than a B-tree in terms of number of block
- accesses required, for some data.
- </para>
-
- <para>
- As a result of the overflow cases, we can say that hash indexes are
- most suitable for unique, nearly unique data or data with a low number
- of rows per hash bucket.
- One possible way to avoid problems is to exclude highly non-unique
- values from the index using a partial index condition, but this may
- not be suitable in many cases.
- </para>
-
- <para>
- Like B-Trees, hash indexes perform simple index tuple deletion. This
- is a deferred maintenance operation that deletes index tuples that are
- known to be safe to delete (those whose item identifier's LP_DEAD bit
- is already set). If an insert finds no space is available on a page we
- try to avoid creating a new overflow page by attempting to remove dead
- index tuples. Removal cannot occur if the page is pinned at that time.
- Deletion of dead index pointers also occurs during VACUUM.
- </para>
-
- <para>
- If it can, VACUUM will also try to squeeze the index tuples onto as
- few overflow pages as possible, minimizing the overflow chain. If an
- overflow page becomes empty, overflow pages can be recycled for reuse
- in other buckets, though we never return them to the operating system.
- There is currently no provision to shrink a hash index, other than by
- rebuilding it with REINDEX.
- There is no provision for reducing the number of buckets, either.
- </para>
-
- <para>
- Hash indexes may expand the number of bucket pages as the number of
- rows indexed grows. The hash key-to-bucket-number mapping is chosen so that
- the index can be incrementally expanded. When a new bucket is to be added to
- the index, exactly one existing bucket will need to be "split", with some of
- its tuples being transferred to the new bucket according to the updated
- key-to-bucket-number mapping.
- </para>
-
- <para>
- The expansion occurs in the foreground, which could increase execution
- time for user inserts. Thus, hash indexes may not be suitable for tables
- with rapidly increasing number of rows.
- </para>
-
-</sect1>
-
-<sect1 id="hash-implementation">
- <title>Implementation</title>
-
- <para>
- There are four kinds of pages in a hash index: the meta page (page zero),
- which contains statically allocated control information; primary bucket
- pages; overflow pages; and bitmap pages, which keep track of overflow
- pages that have been freed and are available for re-use. For addressing
- purposes, bitmap pages are regarded as a subset of the overflow pages.
- </para>
-
- <para>
- Both scanning the index and inserting tuples require locating the bucket
- where a given tuple ought to be located. To do this, we need the bucket
- count, highmask, and lowmask from the metapage; however, it's undesirable
- for performance reasons to have to have to lock and pin the metapage for
- every such operation. Instead, we retain a cached copy of the metapage
- in each backend's relcache entry. This will produce the correct bucket
- mapping as long as the target bucket hasn't been split since the last
- cache refresh.
- </para>
-
- <para>
- Primary bucket pages and overflow pages are allocated independently since
- any given index might need more or fewer overflow pages relative to its
- number of buckets. The hash code uses an interesting set of addressing
- rules to support a variable number of overflow pages while not having to
- move primary bucket pages around after they are created.
- </para>
-
- <para>
- Each row in the table indexed is represented by a single index tuple in
- the hash index. Hash index tuples are stored in bucket pages, and if
- they exist, overflow pages. We speed up searches by keeping the index entries
- in any one index page sorted by hash code, thus allowing binary search to be
- used within an index page. Note however that there is *no* assumption about
- the relative ordering of hash codes across different index pages of a bucket.
- </para>
-
- <para>
- The bucket splitting algorithms to expand the hash index are too complex to
- be worthy of mention here, though are described in more detail in
- <filename>src/backend/access/hash/README</filename>.
- The split algorithm is crash safe and can be restarted if not completed
- successfully.
- </para>
-
-</sect1>
-
-</chapter>
diff --git a/doc/src/sgml/indextypes.sgml b/doc/src/sgml/indextypes.sgml
new file mode 100644
index 0000000000..7aad8d94ed
--- /dev/null
+++ b/doc/src/sgml/indextypes.sgml
@@ -0,0 +1,5597 @@
+ <!-- doc/src/sgml/indextypes.sgml -->
+
+<chapter id="indextypes">
+<title>Built-in Index Access Methods</title>
+
+ <sect1 id="btree">
+ <title>B-Tree Indexes</title>
+
+ <indexterm>
+ <primary>index</primary>
+ <secondary>B-Tree</secondary>
+ </indexterm>
+
+ <sect2 id="btree-intro">
+ <title>Introduction</title>
+
+ <para>
+ <productname>PostgreSQL</productname> includes an implementation of the
+ standard <acronym>btree</acronym> (multi-way balanced tree) index data
+ structure. Any data type that can be sorted into a well-defined linear
+ order can be indexed by a btree index. The only limitation is that an
+ index entry cannot exceed approximately one-third of a page (after TOAST
+ compression, if applicable).
+ </para>
+
+ <para>
+ Because each btree operator class imposes a sort order on its data type,
+ btree operator classes (or, really, operator families) have come to be
+ used as <productname>PostgreSQL</productname>'s general representation
+ and understanding of sorting semantics. Therefore, they've acquired
+ some features that go beyond what would be needed just to support btree
+ indexes, and parts of the system that are quite distant from the
+ btree <acronym>AM</acronym> make use of them.
+ </para>
+
+ </sect2>
+
+ <sect2 id="btree-behavior">
+ <title>Behavior of B-Tree Operator Classes</title>
+
+ <para>
+ As shown in <xref linkend="xindex-btree-strat-table"/>, a btree operator
+ class must provide five comparison operators,
+ <literal><</literal>,
+ <literal><=</literal>,
+ <literal>=</literal>,
+ <literal>>=</literal> and
+ <literal>></literal>.
+ One might expect that <literal><></literal> should also be part of
+ the operator class, but it is not, because it would almost never be
+ useful to use a <literal><></literal> WHERE clause in an index
+ search. (For some purposes, the planner treats <literal><></literal>
+ as associated with a btree operator class; but it finds that operator via
+ the <literal>=</literal> operator's negator link, rather than
+ from <structname>pg_amop</structname>.)
+ </para>
+
+ <para>
+ When several data types share near-identical sorting semantics, their
+ operator classes can be grouped into an operator family. Doing so is
+ advantageous because it allows the planner to make deductions about
+ cross-type comparisons. Each operator class within the family should
+ contain the single-type operators (and associated support functions)
+ for its input data type, while cross-type comparison operators and
+ support functions are <quote>loose</quote> in the family. It is
+ recommendable that a complete set of cross-type operators be included
+ in the family, thus ensuring that the planner can represent any
+ comparison conditions that it deduces from transitivity.
+ </para>
+
+ <para>
+ There are some basic assumptions that a btree operator family must
+ satisfy:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ An <literal>=</literal> operator must be an equivalence relation; that
+ is, for all non-null values <replaceable>A</replaceable>,
+ <replaceable>B</replaceable>, <replaceable>C</replaceable> of the
+ data type:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <replaceable>A</replaceable> <literal>=</literal>
+ <replaceable>A</replaceable> is true
+ (<firstterm>reflexive law</firstterm>)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ if <replaceable>A</replaceable> <literal>=</literal>
+ <replaceable>B</replaceable>,
+ then <replaceable>B</replaceable> <literal>=</literal>
+ <replaceable>A</replaceable>
+ (<firstterm>symmetric law</firstterm>)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ if <replaceable>A</replaceable> <literal>=</literal>
+ <replaceable>B</replaceable> and <replaceable>B</replaceable>
+ <literal>=</literal> <replaceable>C</replaceable>,
+ then <replaceable>A</replaceable> <literal>=</literal>
+ <replaceable>C</replaceable>
+ (<firstterm>transitive law</firstterm>)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ A <literal><</literal> operator must be a strong ordering relation;
+ that is, for all non-null values <replaceable>A</replaceable>,
+ <replaceable>B</replaceable>, <replaceable>C</replaceable>:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <replaceable>A</replaceable> <literal><</literal>
+ <replaceable>A</replaceable> is false
+ (<firstterm>irreflexive law</firstterm>)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ if <replaceable>A</replaceable> <literal><</literal>
+ <replaceable>B</replaceable>
+ and <replaceable>B</replaceable> <literal><</literal>
+ <replaceable>C</replaceable>,
+ then <replaceable>A</replaceable> <literal><</literal>
+ <replaceable>C</replaceable>
+ (<firstterm>transitive law</firstterm>)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Furthermore, the ordering is total; that is, for all non-null
+ values <replaceable>A</replaceable>, <replaceable>B</replaceable>:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ exactly one of <replaceable>A</replaceable> <literal><</literal>
+ <replaceable>B</replaceable>, <replaceable>A</replaceable>
+ <literal>=</literal> <replaceable>B</replaceable>, and
+ <replaceable>B</replaceable> <literal><</literal>
+ <replaceable>A</replaceable> is true
+ (<firstterm>trichotomy law</firstterm>)
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ (The trichotomy law justifies the definition of the comparison support
+ function, of course.)
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ The other three operators are defined in terms of <literal>=</literal>
+ and <literal><</literal> in the obvious way, and must act consistently
+ with them.
+ </para>
+
+ <para>
+ For an operator family supporting multiple data types, the above laws must
+ hold when <replaceable>A</replaceable>, <replaceable>B</replaceable>,
+ <replaceable>C</replaceable> are taken from any data types in the family.
+ The transitive laws are the trickiest to ensure, as in cross-type
+ situations they represent statements that the behaviors of two or three
+ different operators are consistent.
+ As an example, it would not work to put <type>float8</type>
+ and <type>numeric</type> into the same operator family, at least not with
+ the current semantics that <type>numeric</type> values are converted
+ to <type>float8</type> for comparison to a <type>float8</type>. Because
+ of the limited accuracy of <type>float8</type>, this means there are
+ distinct <type>numeric</type> values that will compare equal to the
+ same <type>float8</type> value, and thus the transitive law would fail.
+ </para>
+
+ <para>
+ Another requirement for a multiple-data-type family is that any implicit
+ or binary-coercion casts that are defined between data types included in
+ the operator family must not change the associated sort ordering.
+ </para>
+
+ <para>
+ It should be fairly clear why a btree index requires these laws to hold
+ within a single data type: without them there is no ordering to arrange
+ the keys with. Also, index searches using a comparison key of a
+ different data type require comparisons to behave sanely across two
+ data types. The extensions to three or more data types within a family
+ are not strictly required by the btree index mechanism itself, but the
+ planner relies on them for optimization purposes.
+ </para>
+
+ </sect2>
+
+ <sect2 id="btree-support-funcs">
+ <title>B-Tree Support Functions</title>
+
+ <para>
+ As shown in <xref linkend="xindex-btree-support-table"/>, btree defines
+ one required and four optional support functions. The five
+ user-defined methods are:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term><function>order</function></term>
+ <listitem>
+ <para>
+ For each combination of data types that a btree operator family
+ provides comparison operators for, it must provide a comparison
+ support function, registered in
+ <structname>pg_amproc</structname> with support function number 1
+ and
+ <structfield>amproclefttype</structfield>/<structfield>amprocrighttype</structfield>
+ equal to the left and right data types for the comparison (i.e.,
+ the same data types that the matching operators are registered
+ with in <structname>pg_amop</structname>). The comparison
+ function must take two non-null values
+ <replaceable>A</replaceable> and <replaceable>B</replaceable> and
+ return an <type>int32</type> value that is
+ <literal><</literal> <literal>0</literal>,
+ <literal>0</literal>, or <literal>></literal>
+ <literal>0</literal> when <replaceable>A</replaceable>
+ <literal><</literal> <replaceable>B</replaceable>,
+ <replaceable>A</replaceable> <literal>=</literal>
+ <replaceable>B</replaceable>, or <replaceable>A</replaceable>
+ <literal>></literal> <replaceable>B</replaceable>,
+ respectively. A null result is disallowed: all values of the
+ data type must be comparable. See
+ <filename>src/backend/access/nbtree/nbtcompare.c</filename> for
+ examples.
+ </para>
+
+ <para>
+ If the compared values are of a collatable data type, the
+ appropriate collation OID will be passed to the comparison
+ support function, using the standard
+ <function>PG_GET_COLLATION()</function> mechanism.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><function>sortsupport</function></term>
+ <listitem>
+ <para>
+ Optionally, a btree operator family may provide <firstterm>sort
+ support</firstterm> function(s), registered under support
+ function number 2. These functions allow implementing
+ comparisons for sorting purposes in a more efficient way than
+ naively calling the comparison support function. The APIs
+ involved in this are defined in
+ <filename>src/include/utils/sortsupport.h</filename>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><function>in_range</function></term>
+ <listitem>
+ <indexterm>
+ <primary>in_range support functions</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>support functions</primary>
+ <secondary>in_range</secondary>
+ </indexterm>
+ <para>
+ Optionally, a btree operator family may provide
+ <firstterm>in_range</firstterm> support function(s), registered
+ under support function number 3. These are not used during btree
+ index operations; rather, they extend the semantics of the
+ operator family so that it can support window clauses containing
+ the <literal>RANGE</literal> <replaceable>offset</replaceable>
+ <literal>PRECEDING</literal> and <literal>RANGE</literal>
+ <replaceable>offset</replaceable> <literal>FOLLOWING</literal>
+ frame bound types (see <xref
+ linkend="syntax-window-functions"/>). Fundamentally, the extra
+ information provided is how to add or subtract an
+ <replaceable>offset</replaceable> value in a way that is
+ compatible with the family's data ordering.
+ </para>
+
+ <para>
+ An <function>in_range</function> function must have the signature
+ <synopsis>
+ in_range(<replaceable>val</replaceable> type1, <replaceable>base</replaceable> type1, <replaceable>offset</replaceable> type2, <replaceable>sub</replaceable> bool, <replaceable>less</replaceable> bool)
+ returns bool
+ </synopsis>
+ <replaceable>val</replaceable> and
+ <replaceable>base</replaceable> must be of the same type, which
+ is one of the types supported by the operator family (i.e., a
+ type for which it provides an ordering). However,
+ <replaceable>offset</replaceable> could be of a different type,
+ which might be one otherwise unsupported by the family. An
+ example is that the built-in <literal>time_ops</literal> family
+ provides an <function>in_range</function> function that has
+ <replaceable>offset</replaceable> of type <type>interval</type>.
+ A family can provide <function>in_range</function> functions for
+ any of its supported types and one or more
+ <replaceable>offset</replaceable> types. Each
+ <function>in_range</function> function should be entered in
+ <structname>pg_amproc</structname> with
+ <structfield>amproclefttype</structfield> equal to
+ <type>type1</type> and <structfield>amprocrighttype</structfield>
+ equal to <type>type2</type>.
+ </para>
+
+ <para>
+ The essential semantics of an <function>in_range</function>
+ function depend on the two Boolean flag parameters. It should
+ add or subtract <replaceable>base</replaceable> and
+ <replaceable>offset</replaceable>, then compare
+ <replaceable>val</replaceable> to the result, as follows:
+ <itemizedlist>
+ <listitem>
+ <para>
+ if <literal>!</literal><replaceable>sub</replaceable> and
+ <literal>!</literal><replaceable>less</replaceable>, return
+ <replaceable>val</replaceable> <literal>>=</literal>
+ (<replaceable>base</replaceable> <literal>+</literal>
+ <replaceable>offset</replaceable>)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ if <literal>!</literal><replaceable>sub</replaceable> and
+ <replaceable>less</replaceable>, return
+ <replaceable>val</replaceable> <literal><=</literal>
+ (<replaceable>base</replaceable> <literal>+</literal>
+ <replaceable>offset</replaceable>)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ if <replaceable>sub</replaceable> and
+ <literal>!</literal><replaceable>less</replaceable>, return
+ <replaceable>val</replaceable> <literal>>=</literal>
+ (<replaceable>base</replaceable> <literal>-</literal>
+ <replaceable>offset</replaceable>)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ if <replaceable>sub</replaceable> and
+ <replaceable>less</replaceable>, return
+ <replaceable>val</replaceable> <literal><=</literal>
+ (<replaceable>base</replaceable> <literal>-</literal>
+ <replaceable>offset</replaceable>)
+ </para>
+ </listitem>
+ </itemizedlist>
+ Before doing so, the function should check the sign of
+ <replaceable>offset</replaceable>: if it is less than zero, raise
+ error
+ <literal>ERRCODE_INVALID_PRECEDING_OR_FOLLOWING_SIZE</literal>
+ (22013) with error text like <quote>invalid preceding or
+ following size in window function</quote>. (This is required by
+ the SQL standard, although nonstandard operator families might
+ perhaps choose to ignore this restriction, since there seems to
+ be little semantic necessity for it.) This requirement is
+ delegated to the <function>in_range</function> function so that
+ the core code needn't understand what <quote>less than
+ zero</quote> means for a particular data type.
+ </para>
+
+ <para>
+ An additional expectation is that <function>in_range</function>
+ functions should, if practical, avoid throwing an error if
+ <replaceable>base</replaceable> <literal>+</literal>
+ <replaceable>offset</replaceable> or
+ <replaceable>base</replaceable> <literal>-</literal>
+ <replaceable>offset</replaceable> would overflow. The correct
+ comparison result can be determined even if that value would be
+ out of the data type's range. Note that if the data type
+ includes concepts such as <quote>infinity</quote> or
+ <quote>NaN</quote>, extra care may be needed to ensure that
+ <function>in_range</function>'s results agree with the normal
+ sort order of the operator family.
+ </para>
+
+ <para>
+ The results of the <function>in_range</function> function must be
+ consistent with the sort ordering imposed by the operator family.
+ To be precise, given any fixed values of
+ <replaceable>offset</replaceable> and
+ <replaceable>sub</replaceable>, then:
+ <itemizedlist>
+ <listitem>
+ <para>
+ If <function>in_range</function> with
+ <replaceable>less</replaceable> = true is true for some
+ <replaceable>val1</replaceable> and
+ <replaceable>base</replaceable>, it must be true for every
+ <replaceable>val2</replaceable> <literal><=</literal>
+ <replaceable>val1</replaceable> with the same
+ <replaceable>base</replaceable>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ If <function>in_range</function> with
+ <replaceable>less</replaceable> = true is false for some
+ <replaceable>val1</replaceable> and
+ <replaceable>base</replaceable>, it must be false for every
+ <replaceable>val2</replaceable> <literal>>=</literal>
+ <replaceable>val1</replaceable> with the same
+ <replaceable>base</replaceable>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ If <function>in_range</function> with
+ <replaceable>less</replaceable> = true is true for some
+ <replaceable>val</replaceable> and
+ <replaceable>base1</replaceable>, it must be true for every
+ <replaceable>base2</replaceable> <literal>>=</literal>
+ <replaceable>base1</replaceable> with the same
+ <replaceable>val</replaceable>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ If <function>in_range</function> with
+ <replaceable>less</replaceable> = true is false for some
+ <replaceable>val</replaceable> and
+ <replaceable>base1</replaceable>, it must be false for every
+ <replaceable>base2</replaceable> <literal><=</literal>
+ <replaceable>base1</replaceable> with the same
+ <replaceable>val</replaceable>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ Analogous statements with inverted conditions hold when
+ <replaceable>less</replaceable> = false.
+ </para>
+
+ <para>
+ If the type being ordered (<type>type1</type>) is collatable, the
+ appropriate collation OID will be passed to the
+ <function>in_range</function> function, using the standard
+ PG_GET_COLLATION() mechanism.
+ </para>
+
+ <para>
+ <function>in_range</function> functions need not handle NULL
+ inputs, and typically will be marked strict.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><function>equalimage</function></term>
+ <listitem>
+ <para>
+ Optionally, a btree operator family may provide
+ <function>equalimage</function> (<quote>equality implies image
+ equality</quote>) support functions, registered under support
+ function number 4. These functions allow the core code to
+ determine when it is safe to apply the btree deduplication
+ optimization. Currently, <function>equalimage</function>
+ functions are only called when building or rebuilding an index.
+ </para>
+ <para>
+ An <function>equalimage</function> function must have the
+ signature
+ <synopsis>
+ equalimage(<replaceable>opcintype</replaceable> <type>oid</type>) returns bool
+ </synopsis>
+ The return value is static information about an operator class
+ and collation. Returning <literal>true</literal> indicates that
+ the <function>order</function> function for the operator class is
+ guaranteed to only return <literal>0</literal> (<quote>arguments
+ are equal</quote>) when its <replaceable>A</replaceable> and
+ <replaceable>B</replaceable> arguments are also interchangeable
+ without any loss of semantic information. Not registering an
+ <function>equalimage</function> function or returning
+ <literal>false</literal> indicates that this condition cannot be
+ assumed to hold.
+ </para>
+ <para>
+ The <replaceable>opcintype</replaceable> argument is the
+ <literal><structname>pg_type</structname>.oid</literal> of the
+ data type that the operator class indexes. This is a convenience
+ that allows reuse of the same underlying
+ <function>equalimage</function> function across operator classes.
+ If <replaceable>opcintype</replaceable> is a collatable data
+ type, the appropriate collation OID will be passed to the
+ <function>equalimage</function> function, using the standard
+ <function>PG_GET_COLLATION()</function> mechanism.
+ </para>
+ <para>
+ As far as the operator class is concerned, returning
+ <literal>true</literal> indicates that deduplication is safe (or
+ safe for the collation whose OID was passed to its
+ <function>equalimage</function> function). However, the core
+ code will only deem deduplication safe for an index when
+ <emphasis>every</emphasis> indexed column uses an operator class
+ that registers an <function>equalimage</function> function, and
+ each function actually returns <literal>true</literal> when
+ called.
+ </para>
+ <para>
+ Image equality is <emphasis>almost</emphasis> the same condition
+ as simple bitwise equality. There is one subtle difference: When
+ indexing a varlena data type, the on-disk representation of two
+ image equal datums may not be bitwise equal due to inconsistent
+ application of <acronym>TOAST</acronym> compression on input.
+ Formally, when an operator class's
+ <function>equalimage</function> function returns
+ <literal>true</literal>, it is safe to assume that the
+ <literal>datum_image_eq()</literal> C function will always agree
+ with the operator class's <function>order</function> function
+ (provided that the same collation OID is passed to both the
+ <function>equalimage</function> and <function>order</function>
+ functions).
+ </para>
+ <para>
+ The core code is fundamentally unable to deduce anything about
+ the <quote>equality implies image equality</quote> status of an
+ operator class within a multiple-data-type family based on
+ details from other operator classes in the same family. Also, it
+ is not sensible for an operator family to register a cross-type
+ <function>equalimage</function> function, and attempting to do so
+ will result in an error. This is because <quote>equality implies
+ image equality</quote> status does not just depend on
+ sorting/equality semantics, which are more or less defined at the
+ operator family level. In general, the semantics that one
+ particular data type implements must be considered separately.
+ </para>
+ <para>
+ The convention followed by the operator classes included with the
+ core <productname>PostgreSQL</productname> distribution is to
+ register a stock, generic <function>equalimage</function>
+ function. Most operator classes register
+ <function>btequalimage()</function>, which indicates that
+ deduplication is safe unconditionally. Operator classes for
+ collatable data types such as <type>text</type> register
+ <function>btvarstrequalimage()</function>, which indicates that
+ deduplication is safe with deterministic collations. Best
+ practice for third-party extensions is to register their own
+ custom function to retain control.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><function>options</function></term>
+ <listitem>
+ <para>
+ Optionally, a B-tree operator family may provide
+ <function>options</function> (<quote>operator class specific
+ options</quote>) support functions, registered under support
+ function number 5. These functions define a set of user-visible
+ parameters that control operator class behavior.
+ </para>
+ <para>
+ An <function>options</function> support function must have the
+ signature
+ <synopsis>
+ options(<replaceable>relopts</replaceable> <type>local_relopts *</type>) returns void
+ </synopsis>
+ The function is passed a pointer to a <structname>local_relopts</structname>
+ struct, which needs to be filled with a set of operator class
+ specific options. The options can be accessed from other support
+ functions using the <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
+ <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
+ </para>
+ <para>
+ Currently, no B-Tree operator class has an <function>options</function>
+ support function. B-tree doesn't allow flexible representation of keys
+ like GiST, SP-GiST, GIN and BRIN do. So, <function>options</function>
+ probably doesn't have much application in the current B-tree index
+ access method. Nevertheless, this support function was added to B-tree
+ for uniformity, and will probably find uses during further
+ evolution of B-tree in <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ </sect2>
+
+ <sect2 id="btree-implementation">
+ <title>Implementation</title>
+
+ <para>
+ This section covers B-Tree index implementation details that may be
+ of use to advanced users. See
+ <filename>src/backend/access/nbtree/README</filename> in the source
+ distribution for a much more detailed, internals-focused description
+ of the B-Tree implementation.
+ </para>
+ <sect3 id="btree-structure">
+ <title>B-Tree Structure</title>
+ <para>
+ <productname>PostgreSQL</productname> B-Tree indexes are
+ multi-level tree structures, where each level of the tree can be
+ used as a doubly-linked list of pages. A single metapage is stored
+ in a fixed position at the start of the first segment file of the
+ index. All other pages are either leaf pages or internal pages.
+ Leaf pages are the pages on the lowest level of the tree. All
+ other levels consist of internal pages. Each leaf page contains
+ tuples that point to table rows. Each internal page contains
+ tuples that point to the next level down in the tree. Typically,
+ over 99% of all pages are leaf pages. Both internal pages and leaf
+ pages use the standard page format described in <xref
+ linkend="storage-page-layout"/>.
+ </para>
+ <para>
+ New leaf pages are added to a B-Tree index when an existing leaf
+ page cannot fit an incoming tuple. A <firstterm>page
+ split</firstterm> operation makes room for items that originally
+ belonged on the overflowing page by moving a portion of the items
+ to a new page. Page splits must also insert a new
+ <firstterm>downlink</firstterm> to the new page in the parent page,
+ which may cause the parent to split in turn. Page splits
+ <quote>cascade upwards</quote> in a recursive fashion. When the
+ root page finally cannot fit a new downlink, a <firstterm>root page
+ split</firstterm> operation takes place. This adds a new level to
+ the tree structure by creating a new root page that is one level
+ above the original root page.
+ </para>
+ </sect3>
+
+ <sect3 id="btree-deletion">
+ <title>Bottom-up Index Deletion</title>
+ <para>
+ B-Tree indexes are not directly aware that under MVCC, there might
+ be multiple extant versions of the same logical table row; to an
+ index, each tuple is an independent object that needs its own index
+ entry. <quote>Version churn</quote> tuples may sometimes
+ accumulate and adversely affect query latency and throughput. This
+ typically occurs with <command>UPDATE</command>-heavy workloads
+ where most individual updates cannot apply the
+ <link linkend="storage-hot"><acronym>HOT</acronym> optimization.</link>
+ Changing the value of only
+ one column covered by one index during an <command>UPDATE</command>
+ <emphasis>always</emphasis> necessitates a new set of index tuples
+ — one for <emphasis>each and every</emphasis> index on the
+ table. Note in particular that this includes indexes that were not
+ <quote>logically modified</quote> by the <command>UPDATE</command>.
+ All indexes will need a successor physical index tuple that points
+ to the latest version in the table. Each new tuple within each
+ index will generally need to coexist with the original
+ <quote>updated</quote> tuple for a short period of time (typically
+ until shortly after the <command>UPDATE</command> transaction
+ commits).
+ </para>
+ <para>
+ B-Tree indexes incrementally delete version churn index tuples by
+ performing <firstterm>bottom-up index deletion</firstterm> passes.
+ Each deletion pass is triggered in reaction to an anticipated
+ <quote>version churn page split</quote>. This only happens with
+ indexes that are not logically modified by
+ <command>UPDATE</command> statements, where concentrated build up
+ of obsolete versions in particular pages would occur otherwise. A
+ page split will usually be avoided, though it's possible that
+ certain implementation-level heuristics will fail to identify and
+ delete even one garbage index tuple (in which case a page split or
+ deduplication pass resolves the issue of an incoming new tuple not
+ fitting on a leaf page). The worst-case number of versions that
+ any index scan must traverse (for any single logical row) is an
+ important contributor to overall system responsiveness and
+ throughput. A bottom-up index deletion pass targets suspected
+ garbage tuples in a single leaf page based on
+ <emphasis>qualitative</emphasis> distinctions involving logical
+ rows and versions. This contrasts with the <quote>top-down</quote>
+ index cleanup performed by autovacuum workers, which is triggered
+ when certain <emphasis>quantitative</emphasis> table-level
+ thresholds are exceeded (see <xref linkend="autovacuum"/>).
+ </para>
+ <note>
+ <para>
+ Not all deletion operations that are performed within B-Tree
+ indexes are bottom-up deletion operations. There is a distinct
+ category of index tuple deletion: <firstterm>simple index tuple
+ deletion</firstterm>. This is a deferred maintenance operation
+ that deletes index tuples that are known to be safe to delete
+ (those whose item identifier's <literal>LP_DEAD</literal> bit is
+ already set). Like bottom-up index deletion, simple index
+ deletion takes place at the point that a page split is anticipated
+ as a way of avoiding the split.
+ </para>
+ <para>
+ Simple deletion is opportunistic in the sense that it can only
+ take place when recent index scans set the
+ <literal>LP_DEAD</literal> bits of affected items in passing.
+ Prior to <productname>PostgreSQL</productname> 14, the only
+ category of B-Tree deletion was simple deletion. The main
+ differences between it and bottom-up deletion are that only the
+ former is opportunistically driven by the activity of passing
+ index scans, while only the latter specifically targets version
+ churn from <command>UPDATE</command>s that do not logically modify
+ indexed columns.
+ </para>
+ </note>
+ <para>
+ Bottom-up index deletion performs the vast majority of all garbage
+ index tuple cleanup for particular indexes with certain workloads.
+ This is expected with any B-Tree index that is subject to
+ significant version churn from <command>UPDATE</command>s that
+ rarely or never logically modify the columns that the index covers.
+ The average and worst-case number of versions per logical row can
+ be kept low purely through targeted incremental deletion passes.
+ It's quite possible that the on-disk size of certain indexes will
+ never increase by even one single page/block despite
+ <emphasis>constant</emphasis> version churn from
+ <command>UPDATE</command>s. Even then, an exhaustive <quote>clean
+ sweep</quote> by a <command>VACUUM</command> operation (typically
+ run in an autovacuum worker process) will eventually be required as
+ a part of <emphasis>collective</emphasis> cleanup of the table and
+ each of its indexes.
+ </para>
+ <para>
+ Unlike <command>VACUUM</command>, bottom-up index deletion does not
+ provide any strong guarantees about how old the oldest garbage
+ index tuple may be. No index can be permitted to retain
+ <quote>floating garbage</quote> index tuples that became dead prior
+ to a conservative cutoff point shared by the table and all of its
+ indexes collectively. This fundamental table-level invariant makes
+ it safe to recycle table <acronym>TID</acronym>s. This is how it
+ is possible for distinct logical rows to reuse the same table
+ <acronym>TID</acronym> over time (though this can never happen with
+ two logical rows whose lifetimes span the same
+ <command>VACUUM</command> cycle).
+ </para>
+ </sect3>
+
+ <sect3 id="btree-deduplication">
+ <title>Deduplication</title>
+ <para>
+ A duplicate is a leaf page tuple (a tuple that points to a table
+ row) where <emphasis>all</emphasis> indexed key columns have values
+ that match corresponding column values from at least one other leaf
+ page tuple in the same index. Duplicate tuples are quite common in
+ practice. B-Tree indexes can use a special, space-efficient
+ representation for duplicates when an optional technique is
+ enabled: <firstterm>deduplication</firstterm>.
+ </para>
+ <para>
+ Deduplication works by periodically merging groups of duplicate
+ tuples together, forming a single <firstterm>posting list</firstterm> tuple for each
+ group. The column key value(s) only appear once in this
+ representation. This is followed by a sorted array of
+ <acronym>TID</acronym>s that point to rows in the table. This
+ significantly reduces the storage size of indexes where each value
+ (or each distinct combination of column values) appears several
+ times on average. The latency of queries can be reduced
+ significantly. Overall query throughput may increase
+ significantly. The overhead of routine index vacuuming may also be
+ reduced significantly.
+ </para>
+ <note>
+ <para>
+ B-Tree deduplication is just as effective with
+ <quote>duplicates</quote> that contain a NULL value, even though
+ NULL values are never equal to each other according to the
+ <literal>=</literal> member of any B-Tree operator class. As far
+ as any part of the implementation that understands the on-disk
+ B-Tree structure is concerned, NULL is just another value from the
+ domain of indexed values.
+ </para>
+ </note>
+ <para>
+ The deduplication process occurs lazily, when a new item is
+ inserted that cannot fit on an existing leaf page, though only when
+ index tuple deletion could not free sufficient space for the new
+ item (typically deletion is briefly considered and then skipped
+ over). Unlike GIN posting list tuples, B-Tree posting list tuples
+ do not need to expand every time a new duplicate is inserted; they
+ are merely an alternative physical representation of the original
+ logical contents of the leaf page. This design prioritizes
+ consistent performance with mixed read-write workloads. Most
+ client applications will at least see a moderate performance
+ benefit from using deduplication. Deduplication is enabled by
+ default.
+ </para>
+ <para>
+ <command>CREATE INDEX</command> and <command>REINDEX</command>
+ apply deduplication to create posting list tuples, though the
+ strategy they use is slightly different. Each group of duplicate
+ ordinary tuples encountered in the sorted input taken from the
+ table is merged into a posting list tuple
+ <emphasis>before</emphasis> being added to the current pending leaf
+ page. Individual posting list tuples are packed with as many
+ <acronym>TID</acronym>s as possible. Leaf pages are written out in
+ the usual way, without any separate deduplication pass. This
+ strategy is well-suited to <command>CREATE INDEX</command> and
+ <command>REINDEX</command> because they are once-off batch
+ operations.
+ </para>
+ <para>
+ Write-heavy workloads that don't benefit from deduplication due to
+ having few or no duplicate values in indexes will incur a small,
+ fixed performance penalty (unless deduplication is explicitly
+ disabled). The <literal>deduplicate_items</literal> storage
+ parameter can be used to disable deduplication within individual
+ indexes. There is never any performance penalty with read-only
+ workloads, since reading posting list tuples is at least as
+ efficient as reading the standard tuple representation. Disabling
+ deduplication isn't usually helpful.
+ </para>
+ <para>
+ It is sometimes possible for unique indexes (as well as unique
+ constraints) to use deduplication. This allows leaf pages to
+ temporarily <quote>absorb</quote> extra version churn duplicates.
+ Deduplication in unique indexes augments bottom-up index deletion,
+ especially in cases where a long-running transaction holds a
+ snapshot that blocks garbage collection. The goal is to buy time
+ for the bottom-up index deletion strategy to become effective
+ again. Delaying page splits until a single long-running
+ transaction naturally goes away can allow a bottom-up deletion pass
+ to succeed where an earlier deletion pass failed.
+ </para>
+ <tip>
+ <para>
+ A special heuristic is applied to determine whether a
+ deduplication pass in a unique index should take place. It can
+ often skip straight to splitting a leaf page, avoiding a
+ performance penalty from wasting cycles on unhelpful deduplication
+ passes. If you're concerned about the overhead of deduplication,
+ consider setting <literal>deduplicate_items = off</literal>
+ selectively. Leaving deduplication enabled in unique indexes has
+ little downside.
+ </para>
+ </tip>
+ <para>
+ Deduplication cannot be used in all cases due to
+ implementation-level restrictions. Deduplication safety is
+ determined when <command>CREATE INDEX</command> or
+ <command>REINDEX</command> is run.
+ </para>
+ <para>
+ Note that deduplication is deemed unsafe and cannot be used in the
+ following cases involving semantically significant differences
+ among equal datums:
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <type>text</type>, <type>varchar</type>, and <type>char</type>
+ cannot use deduplication when a
+ <emphasis>nondeterministic</emphasis> collation is used. Case
+ and accent differences must be preserved among equal datums.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <type>numeric</type> cannot use deduplication. Numeric display
+ scale must be preserved among equal datums.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <type>jsonb</type> cannot use deduplication, since the
+ <type>jsonb</type> B-Tree operator class uses
+ <type>numeric</type> internally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <type>float4</type> and <type>float8</type> cannot use
+ deduplication. These types have distinct representations for
+ <literal>-0</literal> and <literal>0</literal>, which are
+ nevertheless considered equal. This difference must be
+ preserved.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ There is one further implementation-level restriction that may be
+ lifted in a future version of
+ <productname>PostgreSQL</productname>:
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Container types (such as composite types, arrays, or range
+ types) cannot use deduplication.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ There is one further implementation-level restriction that applies
+ regardless of the operator class or collation used:
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>INCLUDE</literal> indexes can never use deduplication.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ </sect3>
+ </sect2>
+
+ </sect1>
+
+ <sect1 id="gist">
+ <title>GiST Indexes</title>
+
+ <indexterm>
+ <primary>index</primary>
+ <secondary>GiST</secondary>
+ </indexterm>
+
+ <sect2 id="gist-intro">
+ <title>Introduction</title>
+
+ <para>
+ <acronym>GiST</acronym> stands for Generalized Search Tree. It is a
+ balanced, tree-structured access method, that acts as a base template in
+ which to implement arbitrary indexing schemes. B-trees, R-trees and many
+ other indexing schemes can be implemented in <acronym>GiST</acronym>.
+ </para>
+
+ <para>
+ One advantage of <acronym>GiST</acronym> is that it allows the development
+ of custom data types with the appropriate access methods, by
+ an expert in the domain of the data type, rather than a database expert.
+ </para>
+
+ <para>
+ Some of the information here is derived from the University of California
+ at Berkeley's GiST Indexing Project
+ <ulink url="http://gist.cs.berkeley.edu/">web site</ulink> and
+ Marcel Kornacker's thesis,
+ <ulink url="http://www.sai.msu.su/~megera/postgres/gist/papers/concurrency/access-methods-for-next-generation.pdf.gz">
+ Access Methods for Next-Generation Database Systems</ulink>.
+ The <acronym>GiST</acronym>
+ implementation in <productname>PostgreSQL</productname> is primarily
+ maintained by Teodor Sigaev and Oleg Bartunov, and there is more
+ information on their
+ <ulink url="http://www.sai.msu.su/~megera/postgres/gist/">web site</ulink>.
+ </para>
+
+ </sect2>
+
+ <sect2 id="gist-builtin-opclasses">
+ <title>Built-in Operator Classes</title>
+
+ <para>
+ The core <productname>PostgreSQL</productname> distribution
+ includes the <acronym>GiST</acronym> operator classes shown in
+ <xref linkend="gist-builtin-opclasses-table"/>.
+ (Some of the optional modules described in <xref linkend="contrib"/>
+ provide additional <acronym>GiST</acronym> operator classes.)
+ </para>
+
+ <table id="gist-builtin-opclasses-table">
+ <title>Built-in <acronym>GiST</acronym> Operator Classes</title>
+ <tgroup cols="3">
+ <colspec colname="col1" colwidth="2*"/>
+ <colspec colname="col2" colwidth="3*"/>
+ <colspec colname="col3" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Indexable Operators</entry>
+ <entry>Ordering Operators</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry valign="middle" morerows="13"><literal>box_ops</literal></entry>
+ <entry><literal><< (box, box)</literal></entry>
+ <entry valign="middle" morerows="13"><literal><-> (box, point)</literal></entry>
+ </row>
+ <row><entry><literal>&< (box, box)</literal></entry></row>
+ <row><entry><literal>&& (box, box)</literal></entry></row>
+ <row><entry><literal>&> (box, box)</literal></entry></row>
+ <row><entry><literal>>> (box, box)</literal></entry></row>
+ <row><entry><literal>~= (box, box)</literal></entry></row>
+ <row><entry><literal>@> (box, box)</literal></entry></row>
+ <row><entry><literal><@ (box, box)</literal></entry></row>
+ <row><entry><literal>&<| (box, box)</literal></entry></row>
+ <row><entry><literal><<| (box, box)</literal></entry></row>
+ <row><entry><literal>|>> (box, box)</literal></entry></row>
+ <row><entry><literal>|&> (box, box)</literal></entry></row>
+ <row><entry><literal>~ (box, box)</literal></entry></row>
+ <row><entry><literal>@ (box, box)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="13"><literal>circle_ops</literal></entry>
+ <entry><literal><< (circle, circle)</literal></entry>
+ <entry valign="middle" morerows="13"><literal><-> (circle, point)</literal></entry>
+ </row>
+ <row><entry><literal>&< (circle, circle)</literal></entry></row>
+ <row><entry><literal>&> (circle, circle)</literal></entry></row>
+ <row><entry><literal>>> (circle, circle)</literal></entry></row>
+ <row><entry><literal><@ (circle, circle)</literal></entry></row>
+ <row><entry><literal>@> (circle, circle)</literal></entry></row>
+ <row><entry><literal>~= (circle, circle)</literal></entry></row>
+ <row><entry><literal>&& (circle, circle)</literal></entry></row>
+ <row><entry><literal>|>> (circle, circle)</literal></entry></row>
+ <row><entry><literal><<| (circle, circle)</literal></entry></row>
+ <row><entry><literal>&<| (circle, circle)</literal></entry></row>
+ <row><entry><literal>|&> (circle, circle)</literal></entry></row>
+ <row><entry><literal>@ (circle, circle)</literal></entry></row>
+ <row><entry><literal>~ (circle, circle)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="10"><literal>inet_ops</literal></entry>
+ <entry><literal><< (inet, inet)</literal></entry>
+ <entry valign="middle" morerows="10"></entry>
+ </row>
+ <row><entry><literal><<= (inet, inet)</literal></entry></row>
+ <row><entry><literal>>> (inet, inet)</literal></entry></row>
+ <row><entry><literal>>>= (inet, inet)</literal></entry></row>
+ <row><entry><literal>= (inet, inet)</literal></entry></row>
+ <row><entry><literal><> (inet, inet)</literal></entry></row>
+ <row><entry><literal>< (inet, inet)</literal></entry></row>
+ <row><entry><literal><= (inet, inet)</literal></entry></row>
+ <row><entry><literal>> (inet, inet)</literal></entry></row>
+ <row><entry><literal>>= (inet, inet)</literal></entry></row>
+ <row><entry><literal>&& (inet, inet)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="17"><literal>multirange_ops</literal></entry>
+ <entry><literal>= (anymultirange, anymultirange)</literal></entry>
+ <entry valign="middle" morerows="17"></entry>
+ </row>
+ <row><entry><literal>&& (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal>&& (anymultirange, anyrange)</literal></entry></row>
+ <row><entry><literal>@> (anymultirange, anyelement)</literal></entry></row>
+ <row><entry><literal>@> (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal>@> (anymultirange, anyrange)</literal></entry></row>
+ <row><entry><literal><@ (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal><@ (anymultirange, anyrange)</literal></entry></row>
+ <row><entry><literal><< (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal><< (anymultirange, anyrange)</literal></entry></row>
+ <row><entry><literal>>> (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal>>> (anymultirange, anyrange)</literal></entry></row>
+ <row><entry><literal>&< (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal>&< (anymultirange, anyrange)</literal></entry></row>
+ <row><entry><literal>&> (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal>&> (anymultirange, anyrange)</literal></entry></row>
+ <row><entry><literal>-|- (anymultirange, anymultirange)</literal></entry></row>
+ <row><entry><literal>-|- (anymultirange, anyrange)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="7"><literal>point_ops</literal></entry>
+ <entry><literal>|>> (point, point)</literal></entry>
+ <entry valign="middle" morerows="7"><literal><-> (point, point)</literal></entry>
+ </row>
+ <row><entry><literal><< (point, point)</literal></entry></row>
+ <row><entry><literal>>> (point, point)</literal></entry></row>
+ <row><entry><literal><<| (point, point)</literal></entry></row>
+ <row><entry><literal>~= (point, point)</literal></entry></row>
+ <row><entry><literal><@ (point, box)</literal></entry></row>
+ <row><entry><literal><@ (point, polygon)</literal></entry></row>
+ <row><entry><literal><@ (point, circle)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="13"><literal>poly_ops</literal></entry>
+ <entry><literal><< (polygon, polygon)</literal></entry>
+ <entry valign="middle" morerows="13"><literal><-> (polygon, point)</literal></entry>
+ </row>
+ <row><entry><literal>&< (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>&> (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>>> (polygon, polygon)</literal></entry></row>
+ <row><entry><literal><@ (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>@> (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>~= (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>&& (polygon, polygon)</literal></entry></row>
+ <row><entry><literal><<| (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>&<| (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>|&> (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>|>> (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>@ (polygon, polygon)</literal></entry></row>
+ <row><entry><literal>~ (polygon, polygon)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="17"><literal>range_ops</literal></entry>
+ <entry><literal>= (anyrange, anyrange)</literal></entry>
+ <entry valign="middle" morerows="17"></entry>
+ </row>
+ <row><entry><literal>&& (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal>&& (anyrange, anymultirange)</literal></entry></row>
+ <row><entry><literal>@> (anyrange, anyelement)</literal></entry></row>
+ <row><entry><literal>@> (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal>@> (anyrange, anymultirange)</literal></entry></row>
+ <row><entry><literal><@ (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal><@ (anyrange, anymultirange)</literal></entry></row>
+ <row><entry><literal><< (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal><< (anyrange, anymultirange)</literal></entry></row>
+ <row><entry><literal>>> (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal>>> (anyrange, anymultirange)</literal></entry></row>
+ <row><entry><literal>&< (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal>&< (anyrange, anymultirange)</literal></entry></row>
+ <row><entry><literal>&> (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal>&> (anyrange, anymultirange)</literal></entry></row>
+ <row><entry><literal>-|- (anyrange, anyrange)</literal></entry></row>
+ <row><entry><literal>-|- (anyrange, anymultirange)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="1"><literal>tsquery_ops</literal></entry>
+ <entry><literal><@ (tsquery, tsquery)</literal></entry>
+ <entry valign="middle" morerows="1"></entry>
+ </row>
+ <row><entry><literal>@> (tsquery, tsquery)</literal></entry></row>
+ <row>
+ <entry valign="middle"><literal>tsvector_ops</literal></entry>
+ <entry><literal>@@ (tsvector, tsquery)</literal></entry>
+ <entry></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ For historical reasons, the <literal>inet_ops</literal> operator class is
+ not the default class for types <type>inet</type> and <type>cidr</type>.
+ To use it, mention the class name in <command>CREATE INDEX</command>,
+ for example
+<programlisting>
+CREATE INDEX ON my_table USING GIST (my_inet_column inet_ops);
+</programlisting>
+ </para>
+
+ </sect2>
+
+ <sect2 id="gist-extensibility">
+ <title>Extensibility</title>
+
+ <para>
+ Traditionally, implementing a new index access method meant a lot of
+ difficult work. It was necessary to understand the inner workings of the
+ database, such as the lock manager and Write-Ahead Log. The
+ <acronym>GiST</acronym> interface has a high level of abstraction,
+ requiring the access method implementer only to implement the semantics of
+ the data type being accessed. The <acronym>GiST</acronym> layer itself
+ takes care of concurrency, logging and searching the tree structure.
+ </para>
+
+ <para>
+ This extensibility should not be confused with the extensibility of the
+ other standard search trees in terms of the data they can handle. For
+ example, <productname>PostgreSQL</productname> supports extensible B-trees
+ and hash indexes. That means that you can use
+ <productname>PostgreSQL</productname> to build a B-tree or hash over any
+ data type you want. But B-trees only support range predicates
+ (<literal><</literal>, <literal>=</literal>, <literal>></literal>),
+ and hash indexes only support equality queries.
+ </para>
+
+ <para>
+ So if you index, say, an image collection with a
+ <productname>PostgreSQL</productname> B-tree, you can only issue queries
+ such as <quote>is imagex equal to imagey</quote>, <quote>is imagex less
+ than imagey</quote> and <quote>is imagex greater than imagey</quote>.
+ Depending on how you define <quote>equals</quote>, <quote>less than</quote>
+ and <quote>greater than</quote> in this context, this could be useful.
+ However, by using a <acronym>GiST</acronym> based index, you could create
+ ways to ask domain-specific questions, perhaps <quote>find all images of
+ horses</quote> or <quote>find all over-exposed images</quote>.
+ </para>
+
+ <para>
+ All it takes to get a <acronym>GiST</acronym> access method up and running
+ is to implement several user-defined methods, which define the behavior of
+ keys in the tree. Of course these methods have to be pretty fancy to
+ support fancy queries, but for all the standard queries (B-trees,
+ R-trees, etc.) they're relatively straightforward. In short,
+ <acronym>GiST</acronym> combines extensibility along with generality, code
+ reuse, and a clean interface.
+ </para>
+
+ <para>
+ There are five methods that an index operator class for
+ <acronym>GiST</acronym> must provide, and seven that are optional.
+ Correctness of the index is ensured
+ by proper implementation of the <function>same</function>, <function>consistent</function>
+ and <function>union</function> methods, while efficiency (size and speed) of the
+ index will depend on the <function>penalty</function> and <function>picksplit</function>
+ methods.
+ Two optional methods are <function>compress</function> and
+ <function>decompress</function>, which allow an index to have internal tree data of
+ a different type than the data it indexes. The leaves are to be of the
+ indexed data type, while the other tree nodes can be of any C struct (but
+ you still have to follow <productname>PostgreSQL</productname> data type rules here,
+ see about <literal>varlena</literal> for variable sized data). If the tree's
+ internal data type exists at the SQL level, the <literal>STORAGE</literal> option
+ of the <command>CREATE OPERATOR CLASS</command> command can be used.
+ The optional eighth method is <function>distance</function>, which is needed
+ if the operator class wishes to support ordered scans (nearest-neighbor
+ searches). The optional ninth method <function>fetch</function> is needed if the
+ operator class wishes to support index-only scans, except when the
+ <function>compress</function> method is omitted. The optional tenth method
+ <function>options</function> is needed if the operator class has
+ user-specified parameters.
+ The optional eleventh method <function>sortsupport</function> is used to
+ speed up building a <acronym>GiST</acronym> index.
+ The optional twelfth method <function>stratnum</function> is used to
+ translate well-known <literal>RT*StrategyNumber</literal>s (from
+ <filename>src/include/access/stratnum.h</filename>) into strategy numbers
+ used by the operator class. This lets the core code look up operators for
+ temporal constraint indexes.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><function>consistent</function></term>
+ <listitem>
+ <para>
+ Given an index entry <literal>p</literal> and a query value <literal>q</literal>,
+ this function determines whether the index entry is
+ <quote>consistent</quote> with the query; that is, could the predicate
+ <quote><replaceable>indexed_column</replaceable>
+ <replaceable>indexable_operator</replaceable> <literal>q</literal></quote> be true for
+ any row represented by the index entry? For a leaf index entry this is
+ equivalent to testing the indexable condition, while for an internal
+ tree node this determines whether it is necessary to scan the subtree
+ of the index represented by the tree node. When the result is
+ <literal>true</literal>, a <literal>recheck</literal> flag must also be returned.
+ This indicates whether the predicate is certainly true or only possibly
+ true. If <literal>recheck</literal> = <literal>false</literal> then the index has
+ tested the predicate condition exactly, whereas if <literal>recheck</literal>
+ = <literal>true</literal> the row is only a candidate match. In that case the
+ system will automatically evaluate the
+ <replaceable>indexable_operator</replaceable> against the actual row value to see
+ if it is really a match. This convention allows
+ <acronym>GiST</acronym> to support both lossless and lossy index
+ structures.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_consistent(internal, data_type, smallint, oid, internal)
+RETURNS bool
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_consistent);
+
+Datum
+my_consistent(PG_FUNCTION_ARGS)
+{
+ GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
+ data_type *query = PG_GETARG_DATA_TYPE_P(1);
+ StrategyNumber strategy = (StrategyNumber) PG_GETARG_UINT16(2);
+ /* Oid subtype = PG_GETARG_OID(3); */
+ bool *recheck = (bool *) PG_GETARG_POINTER(4);
+ data_type *key = DatumGetDataType(entry->key);
+ bool retval;
+
+ /*
+ * determine return value as a function of strategy, key and query.
+ *
+ * Use GIST_LEAF(entry) to know where you're called in the index tree,
+ * which comes handy when supporting the = operator for example (you could
+ * check for non empty union() in non-leaf nodes and equality in leaf
+ * nodes).
+ */
+
+ *recheck = true; /* or false if check is exact */
+
+ PG_RETURN_BOOL(retval);
+}
+</programlisting>
+
+ Here, <varname>key</varname> is an element in the index and <varname>query</varname>
+ the value being looked up in the index. The <literal>StrategyNumber</literal>
+ parameter indicates which operator of your operator class is being
+ applied — it matches one of the operator numbers in the
+ <command>CREATE OPERATOR CLASS</command> command.
+ </para>
+
+ <para>
+ Depending on which operators you have included in the class, the data
+ type of <varname>query</varname> could vary with the operator, since it will
+ be whatever type is on the right-hand side of the operator, which might
+ be different from the indexed data type appearing on the left-hand side.
+ (The above code skeleton assumes that only one type is possible; if
+ not, fetching the <varname>query</varname> argument value would have to depend
+ on the operator.) It is recommended that the SQL declaration of
+ the <function>consistent</function> function use the opclass's indexed data
+ type for the <varname>query</varname> argument, even though the actual type
+ might be something else depending on the operator.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>union</function></term>
+ <listitem>
+ <para>
+ This method consolidates information in the tree. Given a set of
+ entries, this function generates a new index entry that represents
+ all the given entries.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_union(internal, internal)
+RETURNS storage_type
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_union);
+
+Datum
+my_union(PG_FUNCTION_ARGS)
+{
+ GistEntryVector *entryvec = (GistEntryVector *) PG_GETARG_POINTER(0);
+ GISTENTRY *ent = entryvec->vector;
+ data_type *out,
+ *tmp,
+ *old;
+ int numranges,
+ i = 0;
+
+ numranges = entryvec->n;
+ tmp = DatumGetDataType(ent[0].key);
+ out = tmp;
+
+ if (numranges == 1)
+ {
+ out = data_type_deep_copy(tmp);
+
+ PG_RETURN_DATA_TYPE_P(out);
+ }
+
+ for (i = 1; i < numranges; i++)
+ {
+ old = out;
+ tmp = DatumGetDataType(ent[i].key);
+ out = my_union_implementation(out, tmp);
+ }
+
+ PG_RETURN_DATA_TYPE_P(out);
+}
+</programlisting>
+ </para>
+
+ <para>
+ As you can see, in this skeleton we're dealing with a data type
+ where <literal>union(X, Y, Z) = union(union(X, Y), Z)</literal>. It's easy
+ enough to support data types where this is not the case, by
+ implementing the proper union algorithm in this
+ <acronym>GiST</acronym> support method.
+ </para>
+
+ <para>
+ The result of the <function>union</function> function must be a value of the
+ index's storage type, whatever that is (it might or might not be
+ different from the indexed column's type). The <function>union</function>
+ function should return a pointer to newly <function>palloc()</function>ed
+ memory. You can't just return the input value as-is, even if there is
+ no type change.
+ </para>
+
+ <para>
+ As shown above, the <function>union</function> function's
+ first <type>internal</type> argument is actually
+ a <structname>GistEntryVector</structname> pointer. The second argument is a
+ pointer to an integer variable, which can be ignored. (It used to be
+ required that the <function>union</function> function store the size of its
+ result value into that variable, but this is no longer necessary.)
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>compress</function></term>
+ <listitem>
+ <para>
+ Converts a data item into a format suitable for physical storage in
+ an index page.
+ If the <function>compress</function> method is omitted, data items are stored
+ in the index without modification.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_compress(internal)
+RETURNS internal
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_compress);
+
+Datum
+my_compress(PG_FUNCTION_ARGS)
+{
+ GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
+ GISTENTRY *retval;
+
+ if (entry->leafkey)
+ {
+ /* replace entry->key with a compressed version */
+ compressed_data_type *compressed_data = palloc(sizeof(compressed_data_type));
+
+ /* fill *compressed_data from entry->key ... */
+
+ retval = palloc(sizeof(GISTENTRY));
+ gistentryinit(*retval, PointerGetDatum(compressed_data),
+ entry->rel, entry->page, entry->offset, FALSE);
+ }
+ else
+ {
+ /* typically we needn't do anything with non-leaf entries */
+ retval = entry;
+ }
+
+ PG_RETURN_POINTER(retval);
+}
+</programlisting>
+ </para>
+
+ <para>
+ You have to adapt <replaceable>compressed_data_type</replaceable> to the specific
+ type you're converting to in order to compress your leaf nodes, of
+ course.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>decompress</function></term>
+ <listitem>
+ <para>
+ Converts the stored representation of a data item into a format that
+ can be manipulated by the other GiST methods in the operator class.
+ If the <function>decompress</function> method is omitted, it is assumed that
+ the other GiST methods can work directly on the stored data format.
+ (<function>decompress</function> is not necessarily the reverse of
+ the <function>compress</function> method; in particular,
+ if <function>compress</function> is lossy then it's impossible
+ for <function>decompress</function> to exactly reconstruct the original
+ data. <function>decompress</function> is not necessarily equivalent
+ to <function>fetch</function>, either, since the other GiST methods might not
+ require full reconstruction of the data.)
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_decompress(internal)
+RETURNS internal
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_decompress);
+
+Datum
+my_decompress(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(PG_GETARG_POINTER(0));
+}
+</programlisting>
+
+ The above skeleton is suitable for the case where no decompression
+ is needed. (But, of course, omitting the method altogether is even
+ easier, and is recommended in such cases.)
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>penalty</function></term>
+ <listitem>
+ <para>
+ Returns a value indicating the <quote>cost</quote> of inserting the new
+ entry into a particular branch of the tree. Items will be inserted
+ down the path of least <function>penalty</function> in the tree.
+ Values returned by <function>penalty</function> should be non-negative.
+ If a negative value is returned, it will be treated as zero.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_penalty(internal, internal, internal)
+RETURNS internal
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT; -- in some cases penalty functions need not be strict
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_penalty);
+
+Datum
+my_penalty(PG_FUNCTION_ARGS)
+{
+ GISTENTRY *origentry = (GISTENTRY *) PG_GETARG_POINTER(0);
+ GISTENTRY *newentry = (GISTENTRY *) PG_GETARG_POINTER(1);
+ float *penalty = (float *) PG_GETARG_POINTER(2);
+ data_type *orig = DatumGetDataType(origentry->key);
+ data_type *new = DatumGetDataType(newentry->key);
+
+ *penalty = my_penalty_implementation(orig, new);
+ PG_RETURN_POINTER(penalty);
+}
+</programlisting>
+
+ For historical reasons, the <function>penalty</function> function doesn't
+ just return a <type>float</type> result; instead it has to store the value
+ at the location indicated by the third argument. The return
+ value per se is ignored, though it's conventional to pass back the
+ address of that argument.
+ </para>
+
+ <para>
+ The <function>penalty</function> function is crucial to good performance of
+ the index. It'll get used at insertion time to determine which branch
+ to follow when choosing where to add the new entry in the tree. At
+ query time, the more balanced the index, the quicker the lookup.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>picksplit</function></term>
+ <listitem>
+ <para>
+ When an index page split is necessary, this function decides which
+ entries on the page are to stay on the old page, and which are to move
+ to the new page.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_picksplit(internal, internal)
+RETURNS internal
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_picksplit);
+
+Datum
+my_picksplit(PG_FUNCTION_ARGS)
+{
+ GistEntryVector *entryvec = (GistEntryVector *) PG_GETARG_POINTER(0);
+ GIST_SPLITVEC *v = (GIST_SPLITVEC *) PG_GETARG_POINTER(1);
+ OffsetNumber maxoff = entryvec->n - 1;
+ GISTENTRY *ent = entryvec->vector;
+ int i,
+ nbytes;
+ OffsetNumber *left,
+ *right;
+ data_type *tmp_union;
+ data_type *unionL;
+ data_type *unionR;
+ GISTENTRY **raw_entryvec;
+
+ maxoff = entryvec->n - 1;
+ nbytes = (maxoff + 1) * sizeof(OffsetNumber);
+
+ v->spl_left = (OffsetNumber *) palloc(nbytes);
+ left = v->spl_left;
+ v->spl_nleft = 0;
+
+ v->spl_right = (OffsetNumber *) palloc(nbytes);
+ right = v->spl_right;
+ v->spl_nright = 0;
+
+ unionL = NULL;
+ unionR = NULL;
+
+ /* Initialize the raw entry vector. */
+ raw_entryvec = (GISTENTRY **) malloc(entryvec->n * sizeof(void *));
+ for (i = FirstOffsetNumber; i <= maxoff; i = OffsetNumberNext(i))
+ raw_entryvec[i] = &(entryvec->vector[i]);
+
+ for (i = FirstOffsetNumber; i <= maxoff; i = OffsetNumberNext(i))
+ {
+ int real_index = raw_entryvec[i] - entryvec->vector;
+
+ tmp_union = DatumGetDataType(entryvec->vector[real_index].key);
+ Assert(tmp_union != NULL);
+
+ /*
+ * Choose where to put the index entries and update unionL and unionR
+ * accordingly. Append the entries to either v->spl_left or
+ * v->spl_right, and care about the counters.
+ */
+
+ if (my_choice_is_left(unionL, curl, unionR, curr))
+ {
+ if (unionL == NULL)
+ unionL = tmp_union;
+ else
+ unionL = my_union_implementation(unionL, tmp_union);
+
+ *left = real_index;
+ ++left;
+ ++(v->spl_nleft);
+ }
+ else
+ {
+ /*
+ * Same on the right
+ */
+ }
+ }
+
+ v->spl_ldatum = DataTypeGetDatum(unionL);
+ v->spl_rdatum = DataTypeGetDatum(unionR);
+ PG_RETURN_POINTER(v);
+}
+</programlisting>
+
+ Notice that the <function>picksplit</function> function's result is delivered
+ by modifying the passed-in <structname>v</structname> structure. The return
+ value per se is ignored, though it's conventional to pass back the
+ address of <structname>v</structname>.
+ </para>
+
+ <para>
+ Like <function>penalty</function>, the <function>picksplit</function> function
+ is crucial to good performance of the index. Designing suitable
+ <function>penalty</function> and <function>picksplit</function> implementations
+ is where the challenge of implementing well-performing
+ <acronym>GiST</acronym> indexes lies.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>same</function></term>
+ <listitem>
+ <para>
+ Returns true if two index entries are identical, false otherwise.
+ (An <quote>index entry</quote> is a value of the index's storage type,
+ not necessarily the original indexed column's type.)
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_same(storage_type, storage_type, internal)
+RETURNS internal
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_same);
+
+Datum
+my_same(PG_FUNCTION_ARGS)
+{
+ prefix_range *v1 = PG_GETARG_PREFIX_RANGE_P(0);
+ prefix_range *v2 = PG_GETARG_PREFIX_RANGE_P(1);
+ bool *result = (bool *) PG_GETARG_POINTER(2);
+
+ *result = my_eq(v1, v2);
+ PG_RETURN_POINTER(result);
+}
+</programlisting>
+
+ For historical reasons, the <function>same</function> function doesn't
+ just return a Boolean result; instead it has to store the flag
+ at the location indicated by the third argument. The return
+ value per se is ignored, though it's conventional to pass back the
+ address of that argument.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>distance</function></term>
+ <listitem>
+ <para>
+ Given an index entry <literal>p</literal> and a query value <literal>q</literal>,
+ this function determines the index entry's
+ <quote>distance</quote> from the query value. This function must be
+ supplied if the operator class contains any ordering operators.
+ A query using the ordering operator will be implemented by returning
+ index entries with the smallest <quote>distance</quote> values first,
+ so the results must be consistent with the operator's semantics.
+ For a leaf index entry the result just represents the distance to
+ the index entry; for an internal tree node, the result must be the
+ smallest distance that any child entry could have.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_distance(internal, data_type, smallint, oid, internal)
+RETURNS float8
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ And the matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_distance);
+
+Datum
+my_distance(PG_FUNCTION_ARGS)
+{
+ GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
+ data_type *query = PG_GETARG_DATA_TYPE_P(1);
+ StrategyNumber strategy = (StrategyNumber) PG_GETARG_UINT16(2);
+ /* Oid subtype = PG_GETARG_OID(3); */
+ /* bool *recheck = (bool *) PG_GETARG_POINTER(4); */
+ data_type *key = DatumGetDataType(entry->key);
+ double retval;
+
+ /*
+ * determine return value as a function of strategy, key and query.
+ */
+
+ PG_RETURN_FLOAT8(retval);
+}
+</programlisting>
+
+ The arguments to the <function>distance</function> function are identical to
+ the arguments of the <function>consistent</function> function.
+ </para>
+
+ <para>
+ Some approximation is allowed when determining the distance, so long
+ as the result is never greater than the entry's actual distance. Thus,
+ for example, distance to a bounding box is usually sufficient in
+ geometric applications. For an internal tree node, the distance
+ returned must not be greater than the distance to any of the child
+ nodes. If the returned distance is not exact, the function must set
+ <literal>*recheck</literal> to true. (This is not necessary for internal tree
+ nodes; for them, the calculation is always assumed to be inexact.) In
+ this case the executor will calculate the accurate distance after
+ fetching the tuple from the heap, and reorder the tuples if necessary.
+ </para>
+
+ <para>
+ If the distance function returns <literal>*recheck = true</literal> for any
+ leaf node, the original ordering operator's return type must
+ be <type>float8</type> or <type>float4</type>, and the distance function's
+ result values must be comparable to those of the original ordering
+ operator, since the executor will sort using both distance function
+ results and recalculated ordering-operator results. Otherwise, the
+ distance function's result values can be any finite <type>float8</type>
+ values, so long as the relative order of the result values matches the
+ order returned by the ordering operator. (Infinity and minus infinity
+ are used internally to handle cases such as nulls, so it is not
+ recommended that <function>distance</function> functions return these values.)
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>fetch</function></term>
+ <listitem>
+ <para>
+ Converts the compressed index representation of a data item into the
+ original data type, for index-only scans. The returned data must be an
+ exact, non-lossy copy of the originally indexed value.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_fetch(internal)
+RETURNS internal
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ The argument is a pointer to a <structname>GISTENTRY</structname> struct. On
+ entry, its <structfield>key</structfield> field contains a non-NULL leaf datum in
+ compressed form. The return value is another <structname>GISTENTRY</structname>
+ struct, whose <structfield>key</structfield> field contains the same datum in its
+ original, uncompressed form. If the opclass's compress function does
+ nothing for leaf entries, the <function>fetch</function> method can return the
+ argument as-is. Or, if the opclass does not have a compress function,
+ the <function>fetch</function> method can be omitted as well, since it would
+ necessarily be a no-op.
+ </para>
+
+ <para>
+ The matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_fetch);
+
+Datum
+my_fetch(PG_FUNCTION_ARGS)
+{
+ GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
+ input_data_type *in = DatumGetPointer(entry->key);
+ fetched_data_type *fetched_data;
+ GISTENTRY *retval;
+
+ retval = palloc(sizeof(GISTENTRY));
+ fetched_data = palloc(sizeof(fetched_data_type));
+
+ /*
+ * Convert 'fetched_data' into the a Datum of the original datatype.
+ */
+
+ /* fill *retval from fetched_data. */
+ gistentryinit(*retval, PointerGetDatum(converted_datum),
+ entry->rel, entry->page, entry->offset, FALSE);
+
+ PG_RETURN_POINTER(retval);
+}
+</programlisting>
+ </para>
+
+ <para>
+ If the compress method is lossy for leaf entries, the operator class
+ cannot support index-only scans, and must not define
+ a <function>fetch</function> function.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>options</function></term>
+ <listitem>
+ <para>
+ Allows definition of user-visible parameters that control operator
+ class behavior.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_options(internal)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+ </para>
+
+ <para>
+ The function is passed a pointer to a <structname>local_relopts</structname>
+ struct, which needs to be filled with a set of operator class
+ specific options. The options can be accessed from other support
+ functions using the <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
+ <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
+ </para>
+
+ <para>
+ An example implementation of my_options() and parameters use
+ from other support functions are given below:
+
+<programlisting>
+typedef enum MyEnumType
+{
+ MY_ENUM_ON,
+ MY_ENUM_OFF,
+ MY_ENUM_AUTO
+} MyEnumType;
+
+typedef struct
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ int int_param; /* integer parameter */
+ double real_param; /* real parameter */
+ MyEnumType enum_param; /* enum parameter */
+ int str_param; /* string parameter */
+} MyOptionsStruct;
+
+/* String representation of enum values */
+static relopt_enum_elt_def myEnumValues[] =
+{
+ {"on", MY_ENUM_ON},
+ {"off", MY_ENUM_OFF},
+ {"auto", MY_ENUM_AUTO},
+ {(const char *) NULL} /* list terminator */
+};
+
+static char *str_param_default = "default";
+
+/*
+ * Sample validator: checks that string is not longer than 8 bytes.
+ */
+static void
+validate_my_string_relopt(const char *value)
+{
+ if (strlen(value) > 8)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("str_param must be at most 8 bytes")));
+}
+
+/*
+ * Sample filler: switches characters to lower case.
+ */
+static Size
+fill_my_string_relopt(const char *value, void *ptr)
+{
+ char *tmp = str_tolower(value, strlen(value), DEFAULT_COLLATION_OID);
+ int len = strlen(tmp);
+
+ if (ptr)
+ strcpy((char *) ptr, tmp);
+
+ pfree(tmp);
+ return len + 1;
+}
+
+PG_FUNCTION_INFO_V1(my_options);
+
+Datum
+my_options(PG_FUNCTION_ARGS)
+{
+ local_relopts *relopts = (local_relopts *) PG_GETARG_POINTER(0);
+
+ init_local_reloptions(relopts, sizeof(MyOptionsStruct));
+ add_local_int_reloption(relopts, "int_param", "integer parameter",
+ 100, 0, 1000000,
+ offsetof(MyOptionsStruct, int_param));
+ add_local_real_reloption(relopts, "real_param", "real parameter",
+ 1.0, 0.0, 1000000.0,
+ offsetof(MyOptionsStruct, real_param));
+ add_local_enum_reloption(relopts, "enum_param", "enum parameter",
+ myEnumValues, MY_ENUM_ON,
+ "Valid values are: \"on\", \"off\" and \"auto\".",
+ offsetof(MyOptionsStruct, enum_param));
+ add_local_string_reloption(relopts, "str_param", "string parameter",
+ str_param_default,
+ &validate_my_string_relopt,
+ &fill_my_string_relopt,
+ offsetof(MyOptionsStruct, str_param));
+
+ PG_RETURN_VOID();
+}
+
+PG_FUNCTION_INFO_V1(my_compress);
+
+Datum
+my_compress(PG_FUNCTION_ARGS)
+{
+ int int_param = 100;
+ double real_param = 1.0;
+ MyEnumType enum_param = MY_ENUM_ON;
+ char *str_param = str_param_default;
+
+ /*
+ * Normally, when opclass contains 'options' method, then options are always
+ * passed to support functions. However, if you add 'options' method to
+ * existing opclass, previously defined indexes have no options, so the
+ * check is required.
+ */
+ if (PG_HAS_OPCLASS_OPTIONS())
+ {
+ MyOptionsStruct *options = (MyOptionsStruct *) PG_GET_OPCLASS_OPTIONS();
+
+ int_param = options->int_param;
+ real_param = options->real_param;
+ enum_param = options->enum_param;
+ str_param = GET_STRING_RELOPTION(options, str_param);
+ }
+
+ /* the rest implementation of support function */
+}
+
+</programlisting>
+ </para>
+
+ <para>
+ Since the representation of the key in <acronym>GiST</acronym> is
+ flexible, it may depend on user-specified parameters. For instance,
+ the length of key signature may be specified. See
+ <literal>gtsvector_options()</literal> for example.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>sortsupport</function></term>
+ <listitem>
+ <para>
+ Returns a comparator function to sort data in a way that preserves
+ locality. It is used by <command>CREATE INDEX</command> and
+ <command>REINDEX</command> commands. The quality of the created index
+ depends on how well the sort order determined by the comparator function
+ preserves locality of the inputs.
+ </para>
+ <para>
+ The <function>sortsupport</function> method is optional. If it is not
+ provided, <command>CREATE INDEX</command> builds the index by inserting
+ each tuple to the tree using the <function>penalty</function> and
+ <function>picksplit</function> functions, which is much slower.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like
+ this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_sortsupport(internal)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+
+ The argument is a pointer to a <structname>SortSupport</structname>
+ struct. At a minimum, the function must fill in its comparator field.
+ The comparator takes three arguments: two Datums to compare, and
+ a pointer to the <structname>SortSupport</structname> struct. The
+ Datums are the two indexed values in the format that they are stored
+ in the index; that is, in the format returned by the
+ <function>compress</function> method. The full API is defined in
+ <filename>src/include/utils/sortsupport.h</filename>.
+ </para>
+
+ <para>
+ The matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_sortsupport);
+
+static int
+my_fastcmp(Datum x, Datum y, SortSupport ssup)
+{
+ /* establish order between x and y by computing some sorting value z */
+
+ int z1 = ComputeSpatialCode(x);
+ int z2 = ComputeSpatialCode(y);
+
+ return z1 == z2 ? 0 : z1 > z2 ? 1 : -1;
+}
+
+Datum
+my_sortsupport(PG_FUNCTION_ARGS)
+{
+ SortSupport ssup = (SortSupport) PG_GETARG_POINTER(0);
+
+ ssup->comparator = my_fastcmp;
+ PG_RETURN_VOID();
+}
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>stratnum</function></term>
+ <listitem>
+ <para>
+ Given an <literal>RT*StrategyNumber</literal> value from
+ <filename>src/include/access/stratnum.h</filename>, returns a strategy
+ number used by this operator class for matching functionality. The
+ function should return <literal>InvalidStrategy</literal> if the
+ operator class has no matching strategy.
+ </para>
+
+ <para>
+ This is used for temporal index constraints (i.e., <literal>PRIMARY
+ KEY</literal> and <literal>UNIQUE</literal>). If the operator class
+ provides this function and it returns results for
+ <literal>RTEqualStrategyNumber</literal>, it can be used in the
+ non-<literal>WITHOUT OVERLAPS</literal> part(s) of an index constraint.
+ If it returns results for <literal>RTOverlapStrategyNumber</literal>,
+ the operator class can be used in the <literal>WITHOUT
+ OVERLAPS</literal> part of an index constraint.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like
+ this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_stratnum(integer)
+RETURNS integer
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+ </para>
+
+ <para>
+ The matching code in the C module could then follow this skeleton:
+
+<programlisting>
+PG_FUNCTION_INFO_V1(my_stratnum);
+
+Datum
+my_stratnum(PG_FUNCTION_ARGS)
+{
+ StrategyNumber strategy = (StrategyNumber) PG_GETARG_UINT16(1);
+ StrategyNumber ret = InvalidStrategy;
+
+ switch (strategy)
+ {
+ case RTEqualStrategyNumber:
+ ret = BTEqualStrategyNumber;
+ }
+
+ PG_RETURN_UINT16(ret);
+}
+ </programlisting>
+ </para>
+
+ <para>
+ One translation function is provided by
+ <productname>PostgreSQL</productname>:
+ <literal>gist_stratnum_identity</literal> is for operator classes that
+ already use the <literal>RT*StrategyNumber</literal> constants. It
+ returns whatever is passed to it. The <literal>btree_gist</literal>
+ extension defines a second translation function,
+ <literal>gist_stratnum_btree</literal>, for operator classes that use
+ the <literal>BT*StrategyNumber</literal> constants.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ All the GiST support methods are normally called in short-lived memory
+ contexts; that is, <varname>CurrentMemoryContext</varname> will get reset after
+ each tuple is processed. It is therefore not very important to worry about
+ pfree'ing everything you palloc. However, in some cases it's useful for a
+ support method to cache data across repeated calls. To do that, allocate
+ the longer-lived data in <literal>fcinfo->flinfo->fn_mcxt</literal>, and
+ keep a pointer to it in <literal>fcinfo->flinfo->fn_extra</literal>. Such
+ data will survive for the life of the index operation (e.g., a single GiST
+ index scan, index build, or index tuple insertion). Be careful to pfree
+ the previous value when replacing a <literal>fn_extra</literal> value, or the leak
+ will accumulate for the duration of the operation.
+ </para>
+
+ </sect2>
+
+ <sect2 id="gist-implementation">
+ <title>Implementation</title>
+
+ <sect3 id="gist-buffering-build">
+ <title>GiST Index Build Methods</title>
+
+ <para>
+ The simplest way to build a GiST index is just to insert all the entries,
+ one by one. This tends to be slow for large indexes, because if the
+ index tuples are scattered across the index and the index is large enough
+ to not fit in cache, a lot of random I/O will be
+ needed. <productname>PostgreSQL</productname> supports two alternative
+ methods for initial build of a GiST index: <firstterm>sorted</firstterm>
+ and <firstterm>buffered</firstterm> modes.
+ </para>
+
+ <para>
+ The sorted method is only available if each of the opclasses used by the
+ index provides a <function>sortsupport</function> function, as described
+ in <xref linkend="gist-extensibility"/>. If they do, this method is
+ usually the best, so it is used by default.
+ </para>
+
+ <para>
+ The buffered method works by not inserting tuples directly into the index
+ right away. It can dramatically reduce the amount of random I/O needed
+ for non-ordered data sets. For well-ordered data sets the benefit is
+ smaller or non-existent, because only a small number of pages receive new
+ tuples at a time, and those pages fit in cache even if the index as a
+ whole does not.
+ </para>
+
+ <para>
+ The buffered method needs to call the <function>penalty</function>
+ function more often than the simple method does, which consumes some
+ extra CPU resources. Also, the buffers need temporary disk space, up to
+ the size of the resulting index. Buffering can also influence the quality
+ of the resulting index, in both positive and negative directions. That
+ influence depends on various factors, like the distribution of the input
+ data and the operator class implementation.
+ </para>
+
+ <para>
+ If sorting is not possible, then by default a GiST index build switches
+ to the buffering method when the index size reaches
+ <xref linkend="guc-effective-cache-size"/>. Buffering can be manually
+ forced or prevented by the <literal>buffering</literal> parameter to the
+ CREATE INDEX command. The default behavior is good for most cases, but
+ turning buffering off might speed up the build somewhat if the input data
+ is ordered.
+ </para>
+
+ </sect3>
+ </sect2>
+
+ <sect2 id="gist-examples">
+ <title>Examples</title>
+
+ <para>
+ The <productname>PostgreSQL</productname> source distribution includes
+ several examples of index methods implemented using
+ <acronym>GiST</acronym>. The core system currently provides text search
+ support (indexing for <type>tsvector</type> and <type>tsquery</type>) as well as
+ R-Tree equivalent functionality for some of the built-in geometric data types
+ (see <filename>src/backend/access/gist/gistproc.c</filename>). The following
+ <filename>contrib</filename> modules also contain <acronym>GiST</acronym>
+ operator classes:
+
+ <variablelist>
+ <varlistentry>
+ <term><filename>btree_gist</filename></term>
+ <listitem>
+ <para>B-tree equivalent functionality for several data types</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>cube</filename></term>
+ <listitem>
+ <para>Indexing for multidimensional cubes</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>hstore</filename></term>
+ <listitem>
+ <para>Module for storing (key, value) pairs</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>intarray</filename></term>
+ <listitem>
+ <para>RD-Tree for one-dimensional array of int4 values</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>ltree</filename></term>
+ <listitem>
+ <para>Indexing for tree-like structures</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>pg_trgm</filename></term>
+ <listitem>
+ <para>Text similarity using trigram matching</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>seg</filename></term>
+ <listitem>
+ <para>Indexing for <quote>float ranges</quote></para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ </sect2>
+
+ </sect1>
+
+ <sect1 id="spgist">
+ <title>SP-GiST Indexes</title>
+
+ <indexterm>
+ <primary>index</primary>
+ <secondary>SP-GiST</secondary>
+ </indexterm>
+
+ <sect2 id="spgist-intro">
+ <title>Introduction</title>
+
+ <para>
+ <acronym>SP-GiST</acronym> is an abbreviation for space-partitioned
+ <acronym>GiST</acronym>. <acronym>SP-GiST</acronym> supports partitioned
+ search trees, which facilitate development of a wide range of different
+ non-balanced data structures, such as quad-trees, k-d trees, and radix
+ trees (tries). The common feature of these structures is that they
+ repeatedly divide the search space into partitions that need not be
+ of equal size. Searches that are well matched to the partitioning rule
+ can be very fast.
+ </para>
+
+ <para>
+ These popular data structures were originally developed for in-memory
+ usage. In main memory, they are usually designed as a set of dynamically
+ allocated nodes linked by pointers. This is not suitable for direct
+ storing on disk, since these chains of pointers can be rather long which
+ would require too many disk accesses. In contrast, disk-based data
+ structures should have a high fanout to minimize I/O. The challenge
+ addressed by <acronym>SP-GiST</acronym> is to map search tree nodes to
+ disk pages in such a way that a search need access only a few disk pages,
+ even if it traverses many nodes.
+ </para>
+
+ <para>
+ Like <acronym>GiST</acronym>, <acronym>SP-GiST</acronym> is meant to allow
+ the development of custom data types with the appropriate access methods,
+ by an expert in the domain of the data type, rather than a database expert.
+ </para>
+
+ <para>
+ Some of the information here is derived from Purdue University's
+ SP-GiST Indexing Project
+ <ulink url="https://www.cs.purdue.edu/spgist/">web site</ulink>.
+ The <acronym>SP-GiST</acronym> implementation in
+ <productname>PostgreSQL</productname> is primarily maintained by Teodor
+ Sigaev and Oleg Bartunov, and there is more information on their
+ <!-- URL will be changed -->
+ <ulink url="http://www.sai.msu.su/~megera/wiki/spgist_dev">web site</ulink>.
+ </para>
+
+ </sect2>
+
+ <sect2 id="spgist-builtin-opclasses">
+ <title>Built-in Operator Classes</title>
+
+ <para>
+ The core <productname>PostgreSQL</productname> distribution
+ includes the <acronym>SP-GiST</acronym> operator classes shown in
+ <xref linkend="spgist-builtin-opclasses-table"/>.
+ </para>
+
+ <table id="spgist-builtin-opclasses-table">
+ <title>Built-in <acronym>SP-GiST</acronym> Operator Classes</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Indexable Operators</entry>
+ <entry>Ordering Operators</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry valign="middle" morerows="11"><literal>box_ops</literal></entry>
+ <entry><literal><< (box,box)</literal></entry>
+ <entry valign="middle" morerows="11"><literal><-> (box,point)</literal></entry>
+ </row>
+ <row><entry><literal>&< (box,box)</literal></entry></row>
+ <row><entry><literal>&> (box,box)</literal></entry></row>
+ <row><entry><literal>>> (box,box)</literal></entry></row>
+ <row><entry><literal><@ (box,box)</literal></entry></row>
+ <row><entry><literal>@> (box,box)</literal></entry></row>
+ <row><entry><literal>~= (box,box)</literal></entry></row>
+ <row><entry><literal>&& (box,box)</literal></entry></row>
+ <row><entry><literal><<| (box,box)</literal></entry></row>
+ <row><entry><literal>&<| (box,box)</literal></entry></row>
+ <row><entry><literal>|&> (box,box)</literal></entry></row>
+ <row><entry><literal>|>> (box,box)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="10"><literal>inet_ops</literal></entry>
+ <entry><literal><< (inet,inet)</literal></entry>
+ <entry valign="middle" morerows="10"></entry>
+ </row>
+ <row><entry><literal><<= (inet,inet)</literal></entry></row>
+ <row><entry><literal>>> (inet,inet)</literal></entry></row>
+ <row><entry><literal>>>= (inet,inet)</literal></entry></row>
+ <row><entry><literal>= (inet,inet)</literal></entry></row>
+ <row><entry><literal><> (inet,inet)</literal></entry></row>
+ <row><entry><literal>< (inet,inet)</literal></entry></row>
+ <row><entry><literal><= (inet,inet)</literal></entry></row>
+ <row><entry><literal>> (inet,inet)</literal></entry></row>
+ <row><entry><literal>>= (inet,inet)</literal></entry></row>
+ <row><entry><literal>&& (inet,inet)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="5"><literal>kd_point_ops</literal></entry>
+ <entry><literal>|>> (point,point)</literal></entry>
+ <entry valign="middle" morerows="5"><literal><-> (point,point)</literal></entry>
+ </row>
+ <row><entry><literal><< (point,point)</literal></entry></row>
+ <row><entry><literal>>> (point,point)</literal></entry></row>
+ <row><entry><literal><<| (point,point)</literal></entry></row>
+ <row><entry><literal>~= (point,point)</literal></entry></row>
+ <row><entry><literal><@ (point,box)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="11"><literal>poly_ops</literal></entry>
+ <entry><literal><< (polygon,polygon)</literal></entry>
+ <entry valign="middle" morerows="11"><literal><-> (polygon,point)</literal></entry>
+ </row>
+ <row><entry><literal>&< (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>&> (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>>> (polygon,polygon)</literal></entry></row>
+ <row><entry><literal><@ (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>@> (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>~= (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>&& (polygon,polygon)</literal></entry></row>
+ <row><entry><literal><<| (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>&<| (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>|>> (polygon,polygon)</literal></entry></row>
+ <row><entry><literal>|&> (polygon,polygon)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="5"><literal>quad_point_ops</literal></entry>
+ <entry><literal>|>> (point,point)</literal></entry>
+ <entry valign="middle" morerows="5"><literal><-> (point,point)</literal></entry>
+ </row>
+ <row><entry><literal><< (point,point)</literal></entry></row>
+ <row><entry><literal>>> (point,point)</literal></entry></row>
+ <row><entry><literal><<| (point,point)</literal></entry></row>
+ <row><entry><literal>~= (point,point)</literal></entry></row>
+ <row><entry><literal><@ (point,box)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="9"><literal>range_ops</literal></entry>
+ <entry><literal>= (anyrange,anyrange)</literal></entry>
+ <entry valign="middle" morerows="9"></entry>
+ </row>
+ <row><entry><literal>&& (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>@> (anyrange,anyelement)</literal></entry></row>
+ <row><entry><literal>@> (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal><@ (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal><< (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>>> (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>&< (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>&> (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>-|- (anyrange,anyrange)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="9"><literal>text_ops</literal></entry>
+ <entry><literal>= (text,text)</literal></entry>
+ <entry valign="middle" morerows="9"></entry>
+ </row>
+ <row><entry><literal>< (text,text)</literal></entry></row>
+ <row><entry><literal><= (text,text)</literal></entry></row>
+ <row><entry><literal>> (text,text)</literal></entry></row>
+ <row><entry><literal>>= (text,text)</literal></entry></row>
+ <row><entry><literal>~<~ (text,text)</literal></entry></row>
+ <row><entry><literal>~<=~ (text,text)</literal></entry></row>
+ <row><entry><literal>~>=~ (text,text)</literal></entry></row>
+ <row><entry><literal>~>~ (text,text)</literal></entry></row>
+ <row><entry><literal>^@ (text,text)</literal></entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ Of the two operator classes for type <type>point</type>,
+ <literal>quad_point_ops</literal> is the default. <literal>kd_point_ops</literal>
+ supports the same operators but uses a different index data structure that
+ may offer better performance in some applications.
+ </para>
+ <para>
+ The <literal>quad_point_ops</literal>, <literal>kd_point_ops</literal> and
+ <literal>poly_ops</literal> operator classes support the <literal><-></literal>
+ ordering operator, which enables the k-nearest neighbor (<literal>k-NN</literal>)
+ search over indexed point or polygon data sets.
+ </para>
+
+ </sect2>
+
+ <sect2 id="spgist-extensibility">
+ <title>Extensibility</title>
+
+ <para>
+ <acronym>SP-GiST</acronym> offers an interface with a high level of
+ abstraction, requiring the access method developer to implement only
+ methods specific to a given data type. The <acronym>SP-GiST</acronym> core
+ is responsible for efficient disk mapping and searching the tree structure.
+ It also takes care of concurrency and logging considerations.
+ </para>
+
+ <para>
+ Leaf tuples of an <acronym>SP-GiST</acronym> tree usually contain values
+ of the same data type as the indexed column, although it is also possible
+ for them to contain lossy representations of the indexed column.
+ Leaf tuples stored at the root level will directly represent
+ the original indexed data value, but leaf tuples at lower
+ levels might contain only a partial value, such as a suffix.
+ In that case the operator class support functions must be able to
+ reconstruct the original value using information accumulated from the
+ inner tuples that are passed through to reach the leaf level.
+ </para>
+
+ <para>
+ When an <acronym>SP-GiST</acronym> index is created with
+ <literal>INCLUDE</literal> columns, the values of those columns are also
+ stored in leaf tuples. The <literal>INCLUDE</literal> columns are of no
+ concern to the <acronym>SP-GiST</acronym> operator class, so they are
+ not discussed further here.
+ </para>
+
+ <para>
+ Inner tuples are more complex, since they are branching points in the
+ search tree. Each inner tuple contains a set of one or more
+ <firstterm>nodes</firstterm>, which represent groups of similar leaf values.
+ A node contains a downlink that leads either to another, lower-level inner
+ tuple, or to a short list of leaf tuples that all lie on the same index page.
+ Each node normally has a <firstterm>label</firstterm> that describes it; for example,
+ in a radix tree the node label could be the next character of the string
+ value. (Alternatively, an operator class can omit the node labels, if it
+ works with a fixed set of nodes for all inner tuples;
+ see <xref linkend="spgist-null-labels"/>.)
+ Optionally, an inner tuple can have a <firstterm>prefix</firstterm> value
+ that describes all its members. In a radix tree this could be the common
+ prefix of the represented strings. The prefix value is not necessarily
+ really a prefix, but can be any data needed by the operator class;
+ for example, in a quad-tree it can store the central point that the four
+ quadrants are measured with respect to. A quad-tree inner tuple would
+ then also contain four nodes corresponding to the quadrants around this
+ central point.
+ </para>
+
+ <para>
+ Some tree algorithms require knowledge of level (or depth) of the current
+ tuple, so the <acronym>SP-GiST</acronym> core provides the possibility for
+ operator classes to manage level counting while descending the tree.
+ There is also support for incrementally reconstructing the represented
+ value when that is needed, and for passing down additional data (called
+ <firstterm>traverse values</firstterm>) during a tree descent.
+ </para>
+
+ <note>
+ <para>
+ The <acronym>SP-GiST</acronym> core code takes care of null entries.
+ Although <acronym>SP-GiST</acronym> indexes do store entries for nulls
+ in indexed columns, this is hidden from the index operator class code:
+ no null index entries or search conditions will ever be passed to the
+ operator class methods. (It is assumed that <acronym>SP-GiST</acronym>
+ operators are strict and so cannot succeed for null values.) Null values
+ are therefore not discussed further here.
+ </para>
+ </note>
+
+ <para>
+ There are five user-defined methods that an index operator class for
+ <acronym>SP-GiST</acronym> must provide, and two are optional. All five
+ mandatory methods follow the convention of accepting two <type>internal</type>
+ arguments, the first of which is a pointer to a C struct containing input
+ values for the support method, while the second argument is a pointer to a
+ C struct where output values must be placed. Four of the mandatory methods just
+ return <type>void</type>, since all their results appear in the output struct; but
+ <function>leaf_consistent</function> returns a <type>boolean</type> result.
+ The methods must not modify any fields of their input structs. In all
+ cases, the output struct is initialized to zeroes before calling the
+ user-defined method. The optional sixth method <function>compress</function>
+ accepts a <type>datum</type> to be indexed as the only argument and returns a value suitable
+ for physical storage in a leaf tuple. The optional seventh method
+ <function>options</function> accepts an <type>internal</type> pointer to a C struct, where
+ opclass-specific parameters should be placed, and returns <type>void</type>.
+ </para>
+
+ <para>
+ The five mandatory user-defined methods are:
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><function>config</function></term>
+ <listitem>
+ <para>
+ Returns static information about the index implementation, including
+ the data type OIDs of the prefix and node label data types.
+ </para>
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+<programlisting>
+CREATE FUNCTION my_config(internal, internal) RETURNS void ...
+</programlisting>
+ The first argument is a pointer to a <structname>spgConfigIn</structname>
+ C struct, containing input data for the function.
+ The second argument is a pointer to a <structname>spgConfigOut</structname>
+ C struct, which the function must fill with result data.
+<programlisting>
+typedef struct spgConfigIn
+{
+ Oid attType; /* Data type to be indexed */
+} spgConfigIn;
+
+typedef struct spgConfigOut
+{
+ Oid prefixType; /* Data type of inner-tuple prefixes */
+ Oid labelType; /* Data type of inner-tuple node labels */
+ Oid leafType; /* Data type of leaf-tuple values */
+ bool canReturnData; /* Opclass can reconstruct original data */
+ bool longValuesOK; /* Opclass can cope with values > 1 page */
+} spgConfigOut;
+</programlisting>
+
+ <structfield>attType</structfield> is passed in order to support polymorphic
+ index operator classes; for ordinary fixed-data-type operator classes, it
+ will always have the same value and so can be ignored.
+ </para>
+
+ <para>
+ For operator classes that do not use prefixes,
+ <structfield>prefixType</structfield> can be set to <literal>VOIDOID</literal>.
+ Likewise, for operator classes that do not use node labels,
+ <structfield>labelType</structfield> can be set to <literal>VOIDOID</literal>.
+ <structfield>canReturnData</structfield> should be set true if the operator class
+ is capable of reconstructing the originally-supplied index value.
+ <structfield>longValuesOK</structfield> should be set true only when the
+ <structfield>attType</structfield> is of variable length and the operator
+ class is capable of segmenting long values by repeated suffixing
+ (see <xref linkend="spgist-limits"/>).
+ </para>
+
+ <para>
+ <structfield>leafType</structfield> should match the index storage type
+ defined by the operator class's <structfield>opckeytype</structfield>
+ catalog entry.
+ (Note that <structfield>opckeytype</structfield> can be zero,
+ implying the storage type is the same as the operator class's input
+ type, which is the most common situation.)
+ For reasons of backward compatibility, the <function>config</function>
+ method can set <structfield>leafType</structfield> to some other value,
+ and that value will be used; but this is deprecated since the index
+ contents are then incorrectly identified in the catalogs.
+ Also, it's permissible to
+ leave <structfield>leafType</structfield> uninitialized (zero);
+ that is interpreted as meaning the index storage type derived from
+ <structfield>opckeytype</structfield>.
+ </para>
+
+ <para>
+ When <structfield>attType</structfield>
+ and <structfield>leafType</structfield> are different, the optional
+ method <function>compress</function> must be provided.
+ Method <function>compress</function> is responsible
+ for transformation of datums to be indexed from <structfield>attType</structfield>
+ to <structfield>leafType</structfield>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>choose</function></term>
+ <listitem>
+ <para>
+ Chooses a method for inserting a new value into an inner tuple.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+<programlisting>
+CREATE FUNCTION my_choose(internal, internal) RETURNS void ...
+</programlisting>
+ The first argument is a pointer to a <structname>spgChooseIn</structname>
+ C struct, containing input data for the function.
+ The second argument is a pointer to a <structname>spgChooseOut</structname>
+ C struct, which the function must fill with result data.
+<programlisting>
+typedef struct spgChooseIn
+{
+ Datum datum; /* original datum to be indexed */
+ Datum leafDatum; /* current datum to be stored at leaf */
+ int level; /* current level (counting from zero) */
+
+ /* Data from current inner tuple */
+ bool allTheSame; /* tuple is marked all-the-same? */
+ bool hasPrefix; /* tuple has a prefix? */
+ Datum prefixDatum; /* if so, the prefix value */
+ int nNodes; /* number of nodes in the inner tuple */
+ Datum *nodeLabels; /* node label values (NULL if none) */
+} spgChooseIn;
+
+typedef enum spgChooseResultType
+{
+ spgMatchNode = 1, /* descend into existing node */
+ spgAddNode, /* add a node to the inner tuple */
+ spgSplitTuple /* split inner tuple (change its prefix) */
+} spgChooseResultType;
+
+typedef struct spgChooseOut
+{
+ spgChooseResultType resultType; /* action code, see above */
+ union
+ {
+ struct /* results for spgMatchNode */
+ {
+ int nodeN; /* descend to this node (index from 0) */
+ int levelAdd; /* increment level by this much */
+ Datum restDatum; /* new leaf datum */
+ } matchNode;
+ struct /* results for spgAddNode */
+ {
+ Datum nodeLabel; /* new node's label */
+ int nodeN; /* where to insert it (index from 0) */
+ } addNode;
+ struct /* results for spgSplitTuple */
+ {
+ /* Info to form new upper-level inner tuple with one child tuple */
+ bool prefixHasPrefix; /* tuple should have a prefix? */
+ Datum prefixPrefixDatum; /* if so, its value */
+ int prefixNNodes; /* number of nodes */
+ Datum *prefixNodeLabels; /* their labels (or NULL for
+ * no labels) */
+ int childNodeN; /* which node gets child tuple */
+
+ /* Info to form new lower-level inner tuple with all old nodes */
+ bool postfixHasPrefix; /* tuple should have a prefix? */
+ Datum postfixPrefixDatum; /* if so, its value */
+ } splitTuple;
+ } result;
+} spgChooseOut;
+</programlisting>
+
+ <structfield>datum</structfield> is the original datum of
+ <structname>spgConfigIn</structname>.<structfield>attType</structfield>
+ type that was to be inserted into the index.
+ <structfield>leafDatum</structfield> is a value of
+ <structname>spgConfigOut</structname>.<structfield>leafType</structfield>
+ type, which is initially a result of method
+ <function>compress</function> applied to <structfield>datum</structfield>
+ when method <function>compress</function> is provided, or the same value as
+ <structfield>datum</structfield> otherwise.
+ <structfield>leafDatum</structfield> can change at lower levels of the tree
+ if the <function>choose</function> or <function>picksplit</function>
+ methods change it. When the insertion search reaches a leaf page,
+ the current value of <structfield>leafDatum</structfield> is what will be stored
+ in the newly created leaf tuple.
+ <structfield>level</structfield> is the current inner tuple's level, starting at
+ zero for the root level.
+ <structfield>allTheSame</structfield> is true if the current inner tuple is
+ marked as containing multiple equivalent nodes
+ (see <xref linkend="spgist-all-the-same"/>).
+ <structfield>hasPrefix</structfield> is true if the current inner tuple contains
+ a prefix; if so,
+ <structfield>prefixDatum</structfield> is its value.
+ <structfield>nNodes</structfield> is the number of child nodes contained in the
+ inner tuple, and
+ <structfield>nodeLabels</structfield> is an array of their label values, or
+ NULL if there are no labels.
+ </para>
+
+ <para>
+ The <function>choose</function> function can determine either that
+ the new value matches one of the existing child nodes, or that a new
+ child node must be added, or that the new value is inconsistent with
+ the tuple prefix and so the inner tuple must be split to create a
+ less restrictive prefix.
+ </para>
+
+ <para>
+ If the new value matches one of the existing child nodes,
+ set <structfield>resultType</structfield> to <literal>spgMatchNode</literal>.
+ Set <structfield>nodeN</structfield> to the index (from zero) of that node in
+ the node array.
+ Set <structfield>levelAdd</structfield> to the increment in
+ <structfield>level</structfield> caused by descending through that node,
+ or leave it as zero if the operator class does not use levels.
+ Set <structfield>restDatum</structfield> to equal <structfield>leafDatum</structfield>
+ if the operator class does not modify datums from one level to the
+ next, or otherwise set it to the modified value to be used as
+ <structfield>leafDatum</structfield> at the next level.
+ </para>
+
+ <para>
+ If a new child node must be added,
+ set <structfield>resultType</structfield> to <literal>spgAddNode</literal>.
+ Set <structfield>nodeLabel</structfield> to the label to be used for the new
+ node, and set <structfield>nodeN</structfield> to the index (from zero) at which
+ to insert the node in the node array.
+ After the node has been added, the <function>choose</function>
+ function will be called again with the modified inner tuple;
+ that call should result in an <literal>spgMatchNode</literal> result.
+ </para>
+
+ <para>
+ If the new value is inconsistent with the tuple prefix,
+ set <structfield>resultType</structfield> to <literal>spgSplitTuple</literal>.
+ This action moves all the existing nodes into a new lower-level
+ inner tuple, and replaces the existing inner tuple with a tuple
+ having a single downlink pointing to the new lower-level inner tuple.
+ Set <structfield>prefixHasPrefix</structfield> to indicate whether the new
+ upper tuple should have a prefix, and if so set
+ <structfield>prefixPrefixDatum</structfield> to the prefix value. This new
+ prefix value must be sufficiently less restrictive than the original
+ to accept the new value to be indexed.
+ Set <structfield>prefixNNodes</structfield> to the number of nodes needed in the
+ new tuple, and set <structfield>prefixNodeLabels</structfield> to a palloc'd array
+ holding their labels, or to NULL if node labels are not required.
+ Note that the total size of the new upper tuple must be no more
+ than the total size of the tuple it is replacing; this constrains
+ the lengths of the new prefix and new labels.
+ Set <structfield>childNodeN</structfield> to the index (from zero) of the node
+ that will downlink to the new lower-level inner tuple.
+ Set <structfield>postfixHasPrefix</structfield> to indicate whether the new
+ lower-level inner tuple should have a prefix, and if so set
+ <structfield>postfixPrefixDatum</structfield> to the prefix value. The
+ combination of these two prefixes and the downlink node's label
+ (if any) must have the same meaning as the original prefix, because
+ there is no opportunity to alter the node labels that are moved to
+ the new lower-level tuple, nor to change any child index entries.
+ After the node has been split, the <function>choose</function>
+ function will be called again with the replacement inner tuple.
+ That call may return an <literal>spgAddNode</literal> result, if no suitable
+ node was created by the <literal>spgSplitTuple</literal> action. Eventually
+ <function>choose</function> must return <literal>spgMatchNode</literal> to
+ allow the insertion to descend to the next level.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>picksplit</function></term>
+ <listitem>
+ <para>
+ Decides how to create a new inner tuple over a set of leaf tuples.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+<programlisting>
+CREATE FUNCTION my_picksplit(internal, internal) RETURNS void ...
+</programlisting>
+ The first argument is a pointer to a <structname>spgPickSplitIn</structname>
+ C struct, containing input data for the function.
+ The second argument is a pointer to a <structname>spgPickSplitOut</structname>
+ C struct, which the function must fill with result data.
+<programlisting>
+typedef struct spgPickSplitIn
+{
+ int nTuples; /* number of leaf tuples */
+ Datum *datums; /* their datums (array of length nTuples) */
+ int level; /* current level (counting from zero) */
+} spgPickSplitIn;
+
+typedef struct spgPickSplitOut
+{
+ bool hasPrefix; /* new inner tuple should have a prefix? */
+ Datum prefixDatum; /* if so, its value */
+
+ int nNodes; /* number of nodes for new inner tuple */
+ Datum *nodeLabels; /* their labels (or NULL for no labels) */
+
+ int *mapTuplesToNodes; /* node index for each leaf tuple */
+ Datum *leafTupleDatums; /* datum to store in each new leaf tuple */
+} spgPickSplitOut;
+</programlisting>
+
+ <structfield>nTuples</structfield> is the number of leaf tuples provided.
+ <structfield>datums</structfield> is an array of their datum values of
+ <structname>spgConfigOut</structname>.<structfield>leafType</structfield>
+ type.
+ <structfield>level</structfield> is the current level that all the leaf tuples
+ share, which will become the level of the new inner tuple.
+ </para>
+
+ <para>
+ Set <structfield>hasPrefix</structfield> to indicate whether the new inner
+ tuple should have a prefix, and if so set
+ <structfield>prefixDatum</structfield> to the prefix value.
+ Set <structfield>nNodes</structfield> to indicate the number of nodes that
+ the new inner tuple will contain, and
+ set <structfield>nodeLabels</structfield> to an array of their label values,
+ or to NULL if node labels are not required.
+ Set <structfield>mapTuplesToNodes</structfield> to an array that gives the index
+ (from zero) of the node that each leaf tuple should be assigned to.
+ Set <structfield>leafTupleDatums</structfield> to an array of the values to
+ be stored in the new leaf tuples (these will be the same as the
+ input <structfield>datums</structfield> if the operator class does not modify
+ datums from one level to the next).
+ Note that the <function>picksplit</function> function is
+ responsible for palloc'ing the
+ <structfield>nodeLabels</structfield>, <structfield>mapTuplesToNodes</structfield> and
+ <structfield>leafTupleDatums</structfield> arrays.
+ </para>
+
+ <para>
+ If more than one leaf tuple is supplied, it is expected that the
+ <function>picksplit</function> function will classify them into more than
+ one node; otherwise it is not possible to split the leaf tuples
+ across multiple pages, which is the ultimate purpose of this
+ operation. Therefore, if the <function>picksplit</function> function
+ ends up placing all the leaf tuples in the same node, the core
+ SP-GiST code will override that decision and generate an inner
+ tuple in which the leaf tuples are assigned at random to several
+ identically-labeled nodes. Such a tuple is marked
+ <literal>allTheSame</literal> to signify that this has happened. The
+ <function>choose</function> and <function>inner_consistent</function> functions
+ must take suitable care with such inner tuples.
+ See <xref linkend="spgist-all-the-same"/> for more information.
+ </para>
+
+ <para>
+ <function>picksplit</function> can be applied to a single leaf tuple only
+ in the case that the <function>config</function> function set
+ <structfield>longValuesOK</structfield> to true and a larger-than-a-page input
+ value has been supplied. In this case the point of the operation is
+ to strip off a prefix and produce a new, shorter leaf datum value.
+ The call will be repeated until a leaf datum short enough to fit on
+ a page has been produced. See <xref linkend="spgist-limits"/> for
+ more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>inner_consistent</function></term>
+ <listitem>
+ <para>
+ Returns set of nodes (branches) to follow during tree search.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+<programlisting>
+CREATE FUNCTION my_inner_consistent(internal, internal) RETURNS void ...
+</programlisting>
+ The first argument is a pointer to a <structname>spgInnerConsistentIn</structname>
+ C struct, containing input data for the function.
+ The second argument is a pointer to a <structname>spgInnerConsistentOut</structname>
+ C struct, which the function must fill with result data.
+
+<programlisting>
+typedef struct spgInnerConsistentIn
+{
+ ScanKey scankeys; /* array of operators and comparison values */
+ ScanKey orderbys; /* array of ordering operators and comparison
+ * values */
+ int nkeys; /* length of scankeys array */
+ int norderbys; /* length of orderbys array */
+
+ Datum reconstructedValue; /* value reconstructed at parent */
+ void *traversalValue; /* opclass-specific traverse value */
+ MemoryContext traversalMemoryContext; /* put new traverse values here */
+ int level; /* current level (counting from zero) */
+ bool returnData; /* original data must be returned? */
+
+ /* Data from current inner tuple */
+ bool allTheSame; /* tuple is marked all-the-same? */
+ bool hasPrefix; /* tuple has a prefix? */
+ Datum prefixDatum; /* if so, the prefix value */
+ int nNodes; /* number of nodes in the inner tuple */
+ Datum *nodeLabels; /* node label values (NULL if none) */
+} spgInnerConsistentIn;
+
+typedef struct spgInnerConsistentOut
+{
+ int nNodes; /* number of child nodes to be visited */
+ int *nodeNumbers; /* their indexes in the node array */
+ int *levelAdds; /* increment level by this much for each */
+ Datum *reconstructedValues; /* associated reconstructed values */
+ void **traversalValues; /* opclass-specific traverse values */
+ double **distances; /* associated distances */
+} spgInnerConsistentOut;
+</programlisting>
+
+ The array <structfield>scankeys</structfield>, of length <structfield>nkeys</structfield>,
+ describes the index search condition(s). These conditions are
+ combined with AND — only index entries that satisfy all of
+ them are interesting. (Note that <structfield>nkeys</structfield> = 0 implies
+ that all index entries satisfy the query.) Usually the consistent
+ function only cares about the <structfield>sk_strategy</structfield> and
+ <structfield>sk_argument</structfield> fields of each array entry, which
+ respectively give the indexable operator and comparison value.
+ In particular it is not necessary to check <structfield>sk_flags</structfield> to
+ see if the comparison value is NULL, because the SP-GiST core code
+ will filter out such conditions.
+ The array <structfield>orderbys</structfield>, of length <structfield>norderbys</structfield>,
+ describes ordering operators (if any) in the same manner.
+ <structfield>reconstructedValue</structfield> is the value reconstructed for the
+ parent tuple; it is <literal>(Datum) 0</literal> at the root level or if the
+ <function>inner_consistent</function> function did not provide a value at the
+ parent level.
+ <structfield>traversalValue</structfield> is a pointer to any traverse data
+ passed down from the previous call of <function>inner_consistent</function>
+ on the parent index tuple, or NULL at the root level.
+ <structfield>traversalMemoryContext</structfield> is the memory context in which
+ to store output traverse values (see below).
+ <structfield>level</structfield> is the current inner tuple's level, starting at
+ zero for the root level.
+ <structfield>returnData</structfield> is <literal>true</literal> if reconstructed data is
+ required for this query; this will only be so if the
+ <function>config</function> function asserted <structfield>canReturnData</structfield>.
+ <structfield>allTheSame</structfield> is true if the current inner tuple is
+ marked <quote>all-the-same</quote>; in this case all the nodes have the
+ same label (if any) and so either all or none of them match the query
+ (see <xref linkend="spgist-all-the-same"/>).
+ <structfield>hasPrefix</structfield> is true if the current inner tuple contains
+ a prefix; if so,
+ <structfield>prefixDatum</structfield> is its value.
+ <structfield>nNodes</structfield> is the number of child nodes contained in the
+ inner tuple, and
+ <structfield>nodeLabels</structfield> is an array of their label values, or
+ NULL if the nodes do not have labels.
+ </para>
+
+ <para>
+ <structfield>nNodes</structfield> must be set to the number of child nodes that
+ need to be visited by the search, and
+ <structfield>nodeNumbers</structfield> must be set to an array of their indexes.
+ If the operator class keeps track of levels, set
+ <structfield>levelAdds</structfield> to an array of the level increments
+ required when descending to each node to be visited. (Often these
+ increments will be the same for all the nodes, but that's not
+ necessarily so, so an array is used.)
+ If value reconstruction is needed, set
+ <structfield>reconstructedValues</structfield> to an array of the values
+ reconstructed for each child node to be visited; otherwise, leave
+ <structfield>reconstructedValues</structfield> as NULL.
+ The reconstructed values are assumed to be of type
+ <structname>spgConfigOut</structname>.<structfield>leafType</structfield>.
+ (However, since the core system will do nothing with them except
+ possibly copy them, it is sufficient for them to have the
+ same <literal>typlen</literal> and <literal>typbyval</literal>
+ properties as <structfield>leafType</structfield>.)
+ If ordered search is performed, set <structfield>distances</structfield>
+ to an array of distance values according to <structfield>orderbys</structfield>
+ array (nodes with lowest distances will be processed first). Leave it
+ NULL otherwise.
+ If it is desired to pass down additional out-of-band information
+ (<quote>traverse values</quote>) to lower levels of the tree search,
+ set <structfield>traversalValues</structfield> to an array of the appropriate
+ traverse values, one for each child node to be visited; otherwise,
+ leave <structfield>traversalValues</structfield> as NULL.
+ Note that the <function>inner_consistent</function> function is
+ responsible for palloc'ing the
+ <structfield>nodeNumbers</structfield>, <structfield>levelAdds</structfield>,
+ <structfield>distances</structfield>,
+ <structfield>reconstructedValues</structfield>, and
+ <structfield>traversalValues</structfield> arrays in the current memory context.
+ However, any output traverse values pointed to by
+ the <structfield>traversalValues</structfield> array should be allocated
+ in <structfield>traversalMemoryContext</structfield>.
+ Each traverse value must be a single palloc'd chunk.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>leaf_consistent</function></term>
+ <listitem>
+ <para>
+ Returns true if a leaf tuple satisfies a query.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+<programlisting>
+CREATE FUNCTION my_leaf_consistent(internal, internal) RETURNS bool ...
+</programlisting>
+ The first argument is a pointer to a <structname>spgLeafConsistentIn</structname>
+ C struct, containing input data for the function.
+ The second argument is a pointer to a <structname>spgLeafConsistentOut</structname>
+ C struct, which the function must fill with result data.
+<programlisting>
+typedef struct spgLeafConsistentIn
+{
+ ScanKey scankeys; /* array of operators and comparison values */
+ ScanKey orderbys; /* array of ordering operators and comparison
+ * values */
+ int nkeys; /* length of scankeys array */
+ int norderbys; /* length of orderbys array */
+
+ Datum reconstructedValue; /* value reconstructed at parent */
+ void *traversalValue; /* opclass-specific traverse value */
+ int level; /* current level (counting from zero) */
+ bool returnData; /* original data must be returned? */
+
+ Datum leafDatum; /* datum in leaf tuple */
+} spgLeafConsistentIn;
+
+typedef struct spgLeafConsistentOut
+{
+ Datum leafValue; /* reconstructed original data, if any */
+ bool recheck; /* set true if operator must be rechecked */
+ bool recheckDistances; /* set true if distances must be rechecked */
+ double *distances; /* associated distances */
+} spgLeafConsistentOut;
+</programlisting>
+
+ The array <structfield>scankeys</structfield>, of length <structfield>nkeys</structfield>,
+ describes the index search condition(s). These conditions are
+ combined with AND — only index entries that satisfy all of
+ them satisfy the query. (Note that <structfield>nkeys</structfield> = 0 implies
+ that all index entries satisfy the query.) Usually the consistent
+ function only cares about the <structfield>sk_strategy</structfield> and
+ <structfield>sk_argument</structfield> fields of each array entry, which
+ respectively give the indexable operator and comparison value.
+ In particular it is not necessary to check <structfield>sk_flags</structfield> to
+ see if the comparison value is NULL, because the SP-GiST core code
+ will filter out such conditions.
+ The array <structfield>orderbys</structfield>, of length <structfield>norderbys</structfield>,
+ describes the ordering operators in the same manner.
+ <structfield>reconstructedValue</structfield> is the value reconstructed for the
+ parent tuple; it is <literal>(Datum) 0</literal> at the root level or if the
+ <function>inner_consistent</function> function did not provide a value at the
+ parent level.
+ <structfield>traversalValue</structfield> is a pointer to any traverse data
+ passed down from the previous call of <function>inner_consistent</function>
+ on the parent index tuple, or NULL at the root level.
+ <structfield>level</structfield> is the current leaf tuple's level, starting at
+ zero for the root level.
+ <structfield>returnData</structfield> is <literal>true</literal> if reconstructed data is
+ required for this query; this will only be so if the
+ <function>config</function> function asserted <structfield>canReturnData</structfield>.
+ <structfield>leafDatum</structfield> is the key value of
+ <structname>spgConfigOut</structname>.<structfield>leafType</structfield>
+ stored in the current leaf tuple.
+ </para>
+
+ <para>
+ The function must return <literal>true</literal> if the leaf tuple matches the
+ query, or <literal>false</literal> if not. In the <literal>true</literal> case,
+ if <structfield>returnData</structfield> is <literal>true</literal> then
+ <structfield>leafValue</structfield> must be set to the value (of type
+ <structname>spgConfigIn</structname>.<structfield>attType</structfield>)
+ originally supplied to be indexed for this leaf tuple. Also,
+ <structfield>recheck</structfield> may be set to <literal>true</literal> if the match
+ is uncertain and so the operator(s) must be re-applied to the actual
+ heap tuple to verify the match.
+ If ordered search is performed, set <structfield>distances</structfield>
+ to an array of distance values according to <structfield>orderbys</structfield>
+ array. Leave it NULL otherwise. If at least one of returned distances
+ is not exact, set <structfield>recheckDistances</structfield> to true.
+ In this case, the executor will calculate the exact distances after
+ fetching the tuple from the heap, and will reorder the tuples if needed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ The optional user-defined methods are:
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><function>Datum compress(Datum in)</function></term>
+ <listitem>
+ <para>
+ Converts a data item into a format suitable for physical storage in
+ a leaf tuple of the index. It accepts a value of type
+ <structname>spgConfigIn</structname>.<structfield>attType</structfield>
+ and returns a value of type
+ <structname>spgConfigOut</structname>.<structfield>leafType</structfield>.
+ The output value must not contain an out-of-line TOAST pointer.
+ </para>
+
+ <para>
+ Note: the <function>compress</function> method is only applied to
+ values to be stored. The consistent methods receive query
+ <structfield>scankeys</structfield> unchanged, without transformation
+ using <function>compress</function>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>options</function></term>
+ <listitem>
+ <para>
+ Defines a set of user-visible parameters that control operator class
+ behavior.
+ </para>
+
+ <para>
+ The <acronym>SQL</acronym> declaration of the function must look like this:
+
+<programlisting>
+CREATE OR REPLACE FUNCTION my_options(internal)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+</programlisting>
+ </para>
+
+ <para>
+ The function is passed a pointer to a <structname>local_relopts</structname>
+ struct, which needs to be filled with a set of operator class
+ specific options. The options can be accessed from other support
+ functions using the <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
+ <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
+ </para>
+
+ <para>
+ Since the representation of the key in <acronym>SP-GiST</acronym> is
+ flexible, it may depend on user-specified parameters.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ All the SP-GiST support methods are normally called in a short-lived
+ memory context; that is, <varname>CurrentMemoryContext</varname> will be reset
+ after processing of each tuple. It is therefore not very important to
+ worry about pfree'ing everything you palloc. (The <function>config</function>
+ method is an exception: it should try to avoid leaking memory. But
+ usually the <function>config</function> method need do nothing but assign
+ constants into the passed parameter struct.)
+ </para>
+
+ <para>
+ If the indexed column is of a collatable data type, the index collation
+ will be passed to all the support methods, using the standard
+ <function>PG_GET_COLLATION()</function> mechanism.
+ </para>
+
+ </sect2>
+
+ <sect2 id="spgist-implementation">
+ <title>Implementation</title>
+
+ <para>
+ This section covers implementation details and other tricks that are
+ useful for implementers of <acronym>SP-GiST</acronym> operator classes to
+ know.
+ </para>
+
+ <sect3 id="spgist-limits">
+ <title>SP-GiST Limits</title>
+
+ <para>
+ Individual leaf tuples and inner tuples must fit on a single index page
+ (8kB by default). Therefore, when indexing values of variable-length
+ data types, long values can only be supported by methods such as radix
+ trees, in which each level of the tree includes a prefix that is short
+ enough to fit on a page, and the final leaf level includes a suffix also
+ short enough to fit on a page. The operator class should set
+ <structfield>longValuesOK</structfield> to true only if it is prepared to arrange for
+ this to happen. Otherwise, the <acronym>SP-GiST</acronym> core will
+ reject any request to index a value that is too large to fit
+ on an index page.
+ </para>
+
+ <para>
+ Likewise, it is the operator class's responsibility that inner tuples
+ do not grow too large to fit on an index page; this limits the number
+ of child nodes that can be used in one inner tuple, as well as the
+ maximum size of a prefix value.
+ </para>
+
+ <para>
+ Another limitation is that when an inner tuple's node points to a set
+ of leaf tuples, those tuples must all be in the same index page.
+ (This is a design decision to reduce seeking and save space in the
+ links that chain such tuples together.) If the set of leaf tuples
+ grows too large for a page, a split is performed and an intermediate
+ inner tuple is inserted. For this to fix the problem, the new inner
+ tuple <emphasis>must</emphasis> divide the set of leaf values into more than one
+ node group. If the operator class's <function>picksplit</function> function
+ fails to do that, the <acronym>SP-GiST</acronym> core resorts to
+ extraordinary measures described in <xref linkend="spgist-all-the-same"/>.
+ </para>
+
+ <para>
+ When <structfield>longValuesOK</structfield> is true, it is expected
+ that successive levels of the <acronym>SP-GiST</acronym> tree will
+ absorb more and more information into the prefixes and node labels of
+ the inner tuples, making the required leaf datum smaller and smaller,
+ so that eventually it will fit on a page.
+ To prevent bugs in operator classes from causing infinite insertion
+ loops, the <acronym>SP-GiST</acronym> core will raise an error if the
+ leaf datum does not become any smaller within ten cycles
+ of <function>choose</function> method calls.
+ </para>
+ </sect3>
+
+ <sect3 id="spgist-null-labels">
+ <title>SP-GiST Without Node Labels</title>
+
+ <para>
+ Some tree algorithms use a fixed set of nodes for each inner tuple;
+ for example, in a quad-tree there are always exactly four nodes
+ corresponding to the four quadrants around the inner tuple's centroid
+ point. In such a case the code typically works with the nodes by
+ number, and there is no need for explicit node labels. To suppress
+ node labels (and thereby save some space), the <function>picksplit</function>
+ function can return NULL for the <structfield>nodeLabels</structfield> array,
+ and likewise the <function>choose</function> function can return NULL for
+ the <structfield>prefixNodeLabels</structfield> array during
+ a <literal>spgSplitTuple</literal> action.
+ This will in turn result in <structfield>nodeLabels</structfield> being NULL during
+ subsequent calls to <function>choose</function> and <function>inner_consistent</function>.
+ In principle, node labels could be used for some inner tuples and omitted
+ for others in the same index.
+ </para>
+
+ <para>
+ When working with an inner tuple having unlabeled nodes, it is an error
+ for <function>choose</function> to return <literal>spgAddNode</literal>, since the set
+ of nodes is supposed to be fixed in such cases.
+ </para>
+ </sect3>
+
+ <sect3 id="spgist-all-the-same">
+ <title><quote>All-the-Same</quote> Inner Tuples</title>
+
+ <para>
+ The <acronym>SP-GiST</acronym> core can override the results of the
+ operator class's <function>picksplit</function> function when
+ <function>picksplit</function> fails to divide the supplied leaf values into
+ at least two node categories. When this happens, the new inner tuple
+ is created with multiple nodes that each have the same label (if any)
+ that <function>picksplit</function> gave to the one node it did use, and the
+ leaf values are divided at random among these equivalent nodes.
+ The <literal>allTheSame</literal> flag is set on the inner tuple to warn the
+ <function>choose</function> and <function>inner_consistent</function> functions that the
+ tuple does not have the node set that they might otherwise expect.
+ </para>
+
+ <para>
+ When dealing with an <literal>allTheSame</literal> tuple, a <function>choose</function>
+ result of <literal>spgMatchNode</literal> is interpreted to mean that the new
+ value can be assigned to any of the equivalent nodes; the core code will
+ ignore the supplied <structfield>nodeN</structfield> value and descend into one
+ of the nodes at random (so as to keep the tree balanced). It is an
+ error for <function>choose</function> to return <literal>spgAddNode</literal>, since
+ that would make the nodes not all equivalent; the
+ <literal>spgSplitTuple</literal> action must be used if the value to be inserted
+ doesn't match the existing nodes.
+ </para>
+
+ <para>
+ When dealing with an <literal>allTheSame</literal> tuple, the
+ <function>inner_consistent</function> function should return either all or none
+ of the nodes as targets for continuing the index search, since they are
+ all equivalent. This may or may not require any special-case code,
+ depending on how much the <function>inner_consistent</function> function normally
+ assumes about the meaning of the nodes.
+ </para>
+ </sect3>
+
+ </sect2>
+
+ <sect2 id="spgist-examples">
+ <title>Examples</title>
+
+ <para>
+ The <productname>PostgreSQL</productname> source distribution includes
+ several examples of index operator classes for <acronym>SP-GiST</acronym>,
+ as described in <xref linkend="spgist-builtin-opclasses-table"/>. Look
+ into <filename>src/backend/access/spgist/</filename>
+ and <filename>src/backend/utils/adt/</filename> to see the code.
+ </para>
+
+ </sect2>
+
+ </sect1>
+
+ <sect1 id="gin">
+ <title>GIN Indexes</title>
+
+ <indexterm>
+ <primary>index</primary>
+ <secondary>GIN</secondary>
+ </indexterm>
+
+ <sect2 id="gin-intro">
+ <title>Introduction</title>
+
+ <para>
+ <acronym>GIN</acronym> stands for Generalized Inverted Index.
+ <acronym>GIN</acronym> is designed for handling cases where the items
+ to be indexed are composite values, and the queries to be handled by
+ the index need to search for element values that appear within
+ the composite items. For example, the items could be documents,
+ and the queries could be searches for documents containing specific words.
+ </para>
+
+ <para>
+ We use the word <firstterm>item</firstterm> to refer to a composite value that
+ is to be indexed, and the word <firstterm>key</firstterm> to refer to an element
+ value. <acronym>GIN</acronym> always stores and searches for keys,
+ not item values per se.
+ </para>
+
+ <para>
+ A <acronym>GIN</acronym> index stores a set of (key, posting list) pairs,
+ where a <firstterm>posting list</firstterm> is a set of row IDs in which the key
+ occurs. The same row ID can appear in multiple posting lists, since
+ an item can contain more than one key. Each key value is stored only
+ once, so a <acronym>GIN</acronym> index is very compact for cases
+ where the same key appears many times.
+ </para>
+
+ <para>
+ <acronym>GIN</acronym> is generalized in the sense that the
+ <acronym>GIN</acronym> access method code does not need to know the
+ specific operations that it accelerates.
+ Instead, it uses custom strategies defined for particular data types.
+ The strategy defines how keys are extracted from indexed items and
+ query conditions, and how to determine whether a row that contains
+ some of the key values in a query actually satisfies the query.
+ </para>
+
+ <para>
+ One advantage of <acronym>GIN</acronym> is that it allows the development
+ of custom data types with the appropriate access methods, by
+ an expert in the domain of the data type, rather than a database expert.
+ This is much the same advantage as using <acronym>GiST</acronym>.
+ </para>
+
+ <para>
+ The <acronym>GIN</acronym>
+ implementation in <productname>PostgreSQL</productname> is primarily
+ maintained by Teodor Sigaev and Oleg Bartunov. There is more
+ information about <acronym>GIN</acronym> on their
+ <ulink url="http://www.sai.msu.su/~megera/wiki/Gin">website</ulink>.
+ </para>
+ </sect2>
+
+ <sect2 id="gin-builtin-opclasses">
+ <title>Built-in Operator Classes</title>
+
+ <para>
+ The core <productname>PostgreSQL</productname> distribution
+ includes the <acronym>GIN</acronym> operator classes shown in
+ <xref linkend="gin-builtin-opclasses-table"/>.
+ (Some of the optional modules described in <xref linkend="contrib"/>
+ provide additional <acronym>GIN</acronym> operator classes.)
+ </para>
+
+ <table id="gin-builtin-opclasses-table">
+ <title>Built-in <acronym>GIN</acronym> Operator Classes</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Indexable Operators</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry morerows="3" valign="middle"><literal>array_ops</literal></entry>
+ <entry><literal>&& (anyarray,anyarray)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>@> (anyarray,anyarray)</literal></entry>
+ </row>
+ <row>
+ <entry><literal><@ (anyarray,anyarray)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>= (anyarray,anyarray)</literal></entry>
+ </row>
+ <row>
+ <entry morerows="5" valign="middle"><literal>jsonb_ops</literal></entry>
+ <entry><literal>@> (jsonb,jsonb)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>@? (jsonb,jsonpath)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>@@ (jsonb,jsonpath)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>? (jsonb,text)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>?| (jsonb,text[])</literal></entry>
+ </row>
+ <row>
+ <entry><literal>?& (jsonb,text[])</literal></entry>
+ </row>
+ <row>
+ <entry morerows="2" valign="middle"><literal>jsonb_path_ops</literal></entry>
+ <entry><literal>@> (jsonb,jsonb)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>@? (jsonb,jsonpath)</literal></entry>
+ </row>
+ <row>
+ <entry><literal>@@ (jsonb,jsonpath)</literal></entry>
+ </row>
+ <row>
+ <entry valign="middle"><literal>tsvector_ops</literal></entry>
+ <entry><literal>@@ (tsvector,tsquery)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ Of the two operator classes for type <type>jsonb</type>, <literal>jsonb_ops</literal>
+ is the default. <literal>jsonb_path_ops</literal> supports fewer operators but
+ offers better performance for those operators.
+ See <xref linkend="json-indexing"/> for details.
+ </para>
+
+ </sect2>
+
+ <sect2 id="gin-extensibility">
+ <title>Extensibility</title>
+
+ <para>
+ The <acronym>GIN</acronym> interface has a high level of abstraction,
+ requiring the access method implementer only to implement the semantics of
+ the data type being accessed. The <acronym>GIN</acronym> layer itself
+ takes care of concurrency, logging and searching the tree structure.
+ </para>
+
+ <para>
+ All it takes to get a <acronym>GIN</acronym> access method working is to
+ implement a few user-defined methods, which define the behavior of
+ keys in the tree and the relationships between keys, indexed items,
+ and indexable queries. In short, <acronym>GIN</acronym> combines
+ extensibility with generality, code reuse, and a clean interface.
+ </para>
+
+ <para>
+ There are two methods that an operator class for
+ <acronym>GIN</acronym> must provide:
+
+ <variablelist>
+ <varlistentry>
+ <term><function>Datum *extractValue(Datum itemValue, int32 *nkeys,
+ bool **nullFlags)</function></term>
+ <listitem>
+ <para>
+ Returns a palloc'd array of keys given an item to be indexed. The
+ number of returned keys must be stored into <literal>*nkeys</literal>.
+ If any of the keys can be null, also palloc an array of
+ <literal>*nkeys</literal> <type>bool</type> fields, store its address at
+ <literal>*nullFlags</literal>, and set these null flags as needed.
+ <literal>*nullFlags</literal> can be left <symbol>NULL</symbol> (its initial value)
+ if all keys are non-null.
+ The return value can be <symbol>NULL</symbol> if the item contains no keys.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>Datum *extractQuery(Datum query, int32 *nkeys,
+ StrategyNumber n, bool **pmatch, Pointer **extra_data,
+ bool **nullFlags, int32 *searchMode)</function></term>
+ <listitem>
+ <para>
+ Returns a palloc'd array of keys given a value to be queried; that is,
+ <literal>query</literal> is the value on the right-hand side of an
+ indexable operator whose left-hand side is the indexed column.
+ <literal>n</literal> is the strategy number of the operator within the
+ operator class (see <xref linkend="xindex-strategies"/>).
+ Often, <function>extractQuery</function> will need
+ to consult <literal>n</literal> to determine the data type of
+ <literal>query</literal> and the method it should use to extract key values.
+ The number of returned keys must be stored into <literal>*nkeys</literal>.
+ If any of the keys can be null, also palloc an array of
+ <literal>*nkeys</literal> <type>bool</type> fields, store its address at
+ <literal>*nullFlags</literal>, and set these null flags as needed.
+ <literal>*nullFlags</literal> can be left <symbol>NULL</symbol> (its initial value)
+ if all keys are non-null.
+ The return value can be <symbol>NULL</symbol> if the <literal>query</literal> contains no keys.
+ </para>
+
+ <para>
+ <literal>searchMode</literal> is an output argument that allows
+ <function>extractQuery</function> to specify details about how the search
+ will be done.
+ If <literal>*searchMode</literal> is set to
+ <literal>GIN_SEARCH_MODE_DEFAULT</literal> (which is the value it is
+ initialized to before call), only items that match at least one of
+ the returned keys are considered candidate matches.
+ If <literal>*searchMode</literal> is set to
+ <literal>GIN_SEARCH_MODE_INCLUDE_EMPTY</literal>, then in addition to items
+ containing at least one matching key, items that contain no keys at
+ all are considered candidate matches. (This mode is useful for
+ implementing is-subset-of operators, for example.)
+ If <literal>*searchMode</literal> is set to <literal>GIN_SEARCH_MODE_ALL</literal>,
+ then all non-null items in the index are considered candidate
+ matches, whether they match any of the returned keys or not. (This
+ mode is much slower than the other two choices, since it requires
+ scanning essentially the entire index, but it may be necessary to
+ implement corner cases correctly. An operator that needs this mode
+ in most cases is probably not a good candidate for a GIN operator
+ class.)
+ The symbols to use for setting this mode are defined in
+ <filename>access/gin.h</filename>.
+ </para>
+
+ <para>
+ <literal>pmatch</literal> is an output argument for use when partial match
+ is supported. To use it, <function>extractQuery</function> must allocate
+ an array of <literal>*nkeys</literal> <type>bool</type>s and store its address at
+ <literal>*pmatch</literal>. Each element of the array should be set to true
+ if the corresponding key requires partial match, false if not.
+ If <literal>*pmatch</literal> is set to <symbol>NULL</symbol> then GIN assumes partial match
+ is not required. The variable is initialized to <symbol>NULL</symbol> before call,
+ so this argument can simply be ignored by operator classes that do
+ not support partial match.
+ </para>
+
+ <para>
+ <literal>extra_data</literal> is an output argument that allows
+ <function>extractQuery</function> to pass additional data to the
+ <function>consistent</function> and <function>comparePartial</function> methods.
+ To use it, <function>extractQuery</function> must allocate
+ an array of <literal>*nkeys</literal> pointers and store its address at
+ <literal>*extra_data</literal>, then store whatever it wants to into the
+ individual pointers. The variable is initialized to <symbol>NULL</symbol> before
+ call, so this argument can simply be ignored by operator classes that
+ do not require extra data. If <literal>*extra_data</literal> is set, the
+ whole array is passed to the <function>consistent</function> method, and
+ the appropriate element to the <function>comparePartial</function> method.
+ </para>
+
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ An operator class must also provide a function to check if an indexed item
+ matches the query. It comes in two flavors, a Boolean <function>consistent</function>
+ function, and a ternary <function>triConsistent</function> function.
+ <function>triConsistent</function> covers the functionality of both, so providing
+ <function>triConsistent</function> alone is sufficient. However, if the Boolean
+ variant is significantly cheaper to calculate, it can be advantageous to
+ provide both. If only the Boolean variant is provided, some optimizations
+ that depend on refuting index items before fetching all the keys are
+ disabled.
+
+ <variablelist>
+ <varlistentry>
+ <term><function>bool consistent(bool check[], StrategyNumber n, Datum query,
+ int32 nkeys, Pointer extra_data[], bool *recheck,
+ Datum queryKeys[], bool nullFlags[])</function></term>
+ <listitem>
+ <para>
+ Returns true if an indexed item satisfies the query operator with
+ strategy number <literal>n</literal> (or might satisfy it, if the recheck
+ indication is returned). This function does not have direct access
+ to the indexed item's value, since <acronym>GIN</acronym> does not
+ store items explicitly. Rather, what is available is knowledge
+ about which key values extracted from the query appear in a given
+ indexed item. The <literal>check</literal> array has length
+ <literal>nkeys</literal>, which is the same as the number of keys previously
+ returned by <function>extractQuery</function> for this <literal>query</literal> datum.
+ Each element of the
+ <literal>check</literal> array is true if the indexed item contains the
+ corresponding query key, i.e., if (check[i] == true) the i-th key of the
+ <function>extractQuery</function> result array is present in the indexed item.
+ The original <literal>query</literal> datum is
+ passed in case the <function>consistent</function> method needs to consult it,
+ and so are the <literal>queryKeys[]</literal> and <literal>nullFlags[]</literal>
+ arrays previously returned by <function>extractQuery</function>.
+ <literal>extra_data</literal> is the extra-data array returned by
+ <function>extractQuery</function>, or <symbol>NULL</symbol> if none.
+ </para>
+
+ <para>
+ When <function>extractQuery</function> returns a null key in
+ <literal>queryKeys[]</literal>, the corresponding <literal>check[]</literal> element
+ is true if the indexed item contains a null key; that is, the
+ semantics of <literal>check[]</literal> are like <literal>IS NOT DISTINCT
+ FROM</literal>. The <function>consistent</function> function can examine the
+ corresponding <literal>nullFlags[]</literal> element if it needs to tell
+ the difference between a regular value match and a null match.
+ </para>
+
+ <para>
+ On success, <literal>*recheck</literal> should be set to true if the heap
+ tuple needs to be rechecked against the query operator, or false if
+ the index test is exact. That is, a false return value guarantees
+ that the heap tuple does not match the query; a true return value with
+ <literal>*recheck</literal> set to false guarantees that the heap tuple does
+ match the query; and a true return value with
+ <literal>*recheck</literal> set to true means that the heap tuple might match
+ the query, so it needs to be fetched and rechecked by evaluating the
+ query operator directly against the originally indexed item.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>GinTernaryValue triConsistent(GinTernaryValue check[], StrategyNumber n, Datum query,
+ int32 nkeys, Pointer extra_data[],
+ Datum queryKeys[], bool nullFlags[])</function></term>
+ <listitem>
+ <para>
+ <function>triConsistent</function> is similar to <function>consistent</function>,
+ but instead of Booleans in the <literal>check</literal> vector, there are
+ three possible values for each
+ key: <literal>GIN_TRUE</literal>, <literal>GIN_FALSE</literal> and
+ <literal>GIN_MAYBE</literal>. <literal>GIN_FALSE</literal> and <literal>GIN_TRUE</literal>
+ have the same meaning as regular Boolean values, while
+ <literal>GIN_MAYBE</literal> means that the presence of that key is not known.
+ When <literal>GIN_MAYBE</literal> values are present, the function should only
+ return <literal>GIN_TRUE</literal> if the item certainly matches whether or
+ not the index item contains the corresponding query keys. Likewise, the
+ function must return <literal>GIN_FALSE</literal> only if the item certainly
+ does not match, whether or not it contains the <literal>GIN_MAYBE</literal>
+ keys. If the result depends on the <literal>GIN_MAYBE</literal> entries, i.e.,
+ the match cannot be confirmed or refuted based on the known query keys,
+ the function must return <literal>GIN_MAYBE</literal>.
+ </para>
+ <para>
+ When there are no <literal>GIN_MAYBE</literal> values in the <literal>check</literal>
+ vector, a <literal>GIN_MAYBE</literal> return value is the equivalent of
+ setting the <literal>recheck</literal> flag in the
+ Boolean <function>consistent</function> function.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ In addition, GIN must have a way to sort the key values stored in the index.
+ The operator class can define the sort ordering by specifying a comparison
+ method:
+
+ <variablelist>
+ <varlistentry>
+ <term><function>int compare(Datum a, Datum b)</function></term>
+ <listitem>
+ <para>
+ Compares two keys (not indexed items!) and returns an integer less than
+ zero, zero, or greater than zero, indicating whether the first key is
+ less than, equal to, or greater than the second. Null keys are never
+ passed to this function.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ Alternatively, if the operator class does not provide a <function>compare</function>
+ method, GIN will look up the default btree operator class for the index
+ key data type, and use its comparison function. It is recommended to
+ specify the comparison function in a GIN operator class that is meant for
+ just one data type, as looking up the btree operator class costs a few
+ cycles. However, polymorphic GIN operator classes (such
+ as <literal>array_ops</literal>) typically cannot specify a single comparison
+ function.
+ </para>
+
+ <para>
+ An operator class for <acronym>GIN</acronym> can optionally supply the
+ following methods:
+
+ <variablelist>
+ <varlistentry>
+ <term><function>int comparePartial(Datum partial_key, Datum key, StrategyNumber n,
+ Pointer extra_data)</function></term>
+ <listitem>
+ <para>
+ Compare a partial-match query key to an index key. Returns an integer
+ whose sign indicates the result: less than zero means the index key
+ does not match the query, but the index scan should continue; zero
+ means that the index key does match the query; greater than zero
+ indicates that the index scan should stop because no more matches
+ are possible. The strategy number <literal>n</literal> of the operator
+ that generated the partial match query is provided, in case its
+ semantics are needed to determine when to end the scan. Also,
+ <literal>extra_data</literal> is the corresponding element of the extra-data
+ array made by <function>extractQuery</function>, or <symbol>NULL</symbol> if none.
+ Null keys are never passed to this function.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><function>void options(local_relopts *relopts)</function></term>
+ <listitem>
+ <para>
+ Defines a set of user-visible parameters that control operator class
+ behavior.
+ </para>
+
+ <para>
+ The <function>options</function> function is passed a pointer to a
+ <structname>local_relopts</structname> struct, which needs to be
+ filled with a set of operator class specific options. The options
+ can be accessed from other support functions using the
+ <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
+ <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
+ </para>
+
+ <para>
+ Since both key extraction of indexed values and representation of the
+ key in <acronym>GIN</acronym> are flexible, they may depend on
+ user-specified parameters.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ To support <quote>partial match</quote> queries, an operator class must
+ provide the <function>comparePartial</function> method, and its
+ <function>extractQuery</function> method must set the <literal>pmatch</literal>
+ parameter when a partial-match query is encountered. See
+ <xref linkend="gin-partial-match"/> for details.
+ </para>
+
+ <para>
+ The actual data types of the various <literal>Datum</literal> values mentioned
+ above vary depending on the operator class. The item values passed to
+ <function>extractValue</function> are always of the operator class's input type, and
+ all key values must be of the class's <literal>STORAGE</literal> type. The type of
+ the <literal>query</literal> argument passed to <function>extractQuery</function>,
+ <function>consistent</function> and <function>triConsistent</function> is whatever is the
+ right-hand input type of the class member operator identified by the
+ strategy number. This need not be the same as the indexed type, so long as
+ key values of the correct type can be extracted from it. However, it is
+ recommended that the SQL declarations of these three support functions use
+ the opclass's indexed data type for the <literal>query</literal> argument, even
+ though the actual type might be something else depending on the operator.
+ </para>
+
+ </sect2>
+
+ <sect2 id="gin-implementation">
+ <title>Implementation</title>
+
+ <para>
+ Internally, a <acronym>GIN</acronym> index contains a B-tree index
+ constructed over keys, where each key is an element of one or more indexed
+ items (a member of an array, for example) and where each tuple in a leaf
+ page contains either a pointer to a B-tree of heap pointers (a
+ <quote>posting tree</quote>), or a simple list of heap pointers (a <quote>posting
+ list</quote>) when the list is small enough to fit into a single index tuple along
+ with the key value. <xref linkend="gin-internals-figure"/> illustrates
+ these components of a GIN index.
+ </para>
+
+ <para>
+ As of <productname>PostgreSQL</productname> 9.1, null key values can be
+ included in the index. Also, placeholder nulls are included in the index
+ for indexed items that are null or contain no keys according to
+ <function>extractValue</function>. This allows searches that should find empty
+ items to do so.
+ </para>
+
+ <para>
+ Multicolumn <acronym>GIN</acronym> indexes are implemented by building
+ a single B-tree over composite values (column number, key value). The
+ key values for different columns can be of different types.
+ </para>
+
+ <figure id="gin-internals-figure">
+ <title>GIN Internals</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="images/gin.svg" format="SVG" width="100%"/>
+ </imageobject>
+ </mediaobject>
+ </figure>
+
+ <sect3 id="gin-fast-update">
+ <title>GIN Fast Update Technique</title>
+
+ <para>
+ Updating a <acronym>GIN</acronym> index tends to be slow because of the
+ intrinsic nature of inverted indexes: inserting or updating one heap row
+ can cause many inserts into the index (one for each key extracted
+ from the indexed item).
+ <acronym>GIN</acronym> is capable of postponing much of this work by inserting
+ new tuples into a temporary, unsorted list of pending entries.
+ When the table is vacuumed or autoanalyzed, or when
+ <function>gin_clean_pending_list</function> function is called, or if the
+ pending list becomes larger than
+ <xref linkend="guc-gin-pending-list-limit"/>, the entries are moved to the
+ main <acronym>GIN</acronym> data structure using the same bulk insert
+ techniques used during initial index creation. This greatly improves
+ <acronym>GIN</acronym> index update speed, even counting the additional
+ vacuum overhead. Moreover the overhead work can be done by a background
+ process instead of in foreground query processing.
+ </para>
+
+ <para>
+ The main disadvantage of this approach is that searches must scan the list
+ of pending entries in addition to searching the regular index, and so
+ a large list of pending entries will slow searches significantly.
+ Another disadvantage is that, while most updates are fast, an update
+ that causes the pending list to become <quote>too large</quote> will incur an
+ immediate cleanup cycle and thus be much slower than other updates.
+ Proper use of autovacuum can minimize both of these problems.
+ </para>
+
+ <para>
+ If consistent response time is more important than update speed,
+ use of pending entries can be disabled by turning off the
+ <literal>fastupdate</literal> storage parameter for a
+ <acronym>GIN</acronym> index. See <xref linkend="sql-createindex"/>
+ for details.
+ </para>
+ </sect3>
+
+ <sect3 id="gin-partial-match">
+ <title>Partial Match Algorithm</title>
+
+ <para>
+ GIN can support <quote>partial match</quote> queries, in which the query
+ does not determine an exact match for one or more keys, but the possible
+ matches fall within a reasonably narrow range of key values (within the
+ key sorting order determined by the <function>compare</function> support method).
+ The <function>extractQuery</function> method, instead of returning a key value
+ to be matched exactly, returns a key value that is the lower bound of
+ the range to be searched, and sets the <literal>pmatch</literal> flag true.
+ The key range is then scanned using the <function>comparePartial</function>
+ method. <function>comparePartial</function> must return zero for a matching
+ index key, less than zero for a non-match that is still within the range
+ to be searched, or greater than zero if the index key is past the range
+ that could match.
+ </para>
+ </sect3>
+
+ </sect2>
+
+ <sect2 id="gin-tips">
+ <title>GIN Tips and Tricks</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>Create vs. insert</term>
+ <listitem>
+ <para>
+ Insertion into a <acronym>GIN</acronym> index can be slow
+ due to the likelihood of many keys being inserted for each item.
+ So, for bulk insertions into a table it is advisable to drop the GIN
+ index and recreate it after finishing bulk insertion.
+ </para>
+
+ <para>
+ When <literal>fastupdate</literal> is enabled for <acronym>GIN</acronym>
+ (see <xref linkend="gin-fast-update"/> for details), the penalty is
+ less than when it is not. But for very large updates it may still be
+ best to drop and recreate the index.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><xref linkend="guc-maintenance-work-mem"/></term>
+ <listitem>
+ <para>
+ Build time for a <acronym>GIN</acronym> index is very sensitive to
+ the <varname>maintenance_work_mem</varname> setting; it doesn't pay to
+ skimp on work memory during index creation.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><xref linkend="guc-gin-pending-list-limit"/></term>
+ <listitem>
+ <para>
+ During a series of insertions into an existing <acronym>GIN</acronym>
+ index that has <literal>fastupdate</literal> enabled, the system will clean up
+ the pending-entry list whenever the list grows larger than
+ <varname>gin_pending_list_limit</varname>. To avoid fluctuations in observed
+ response time, it's desirable to have pending-list cleanup occur in the
+ background (i.e., via autovacuum). Foreground cleanup operations
+ can be avoided by increasing <varname>gin_pending_list_limit</varname>
+ or making autovacuum more aggressive.
+ However, enlarging the threshold of the cleanup operation means that
+ if a foreground cleanup does occur, it will take even longer.
+ </para>
+ <para>
+ <varname>gin_pending_list_limit</varname> can be overridden for individual
+ GIN indexes by changing storage parameters, which allows each
+ GIN index to have its own cleanup threshold.
+ For example, it's possible to increase the threshold only for the GIN
+ index which can be updated heavily, and decrease it otherwise.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><xref linkend="guc-gin-fuzzy-search-limit"/></term>
+ <listitem>
+ <para>
+ The primary goal of developing <acronym>GIN</acronym> indexes was
+ to create support for highly scalable full-text search in
+ <productname>PostgreSQL</productname>, and there are often situations when
+ a full-text search returns a very large set of results. Moreover, this
+ often happens when the query contains very frequent words, so that the
+ large result set is not even useful. Since reading many
+ tuples from the disk and sorting them could take a lot of time, this is
+ unacceptable for production. (Note that the index search itself is very
+ fast.)
+ </para>
+ <para>
+ To facilitate controlled execution of such queries,
+ <acronym>GIN</acronym> has a configurable soft upper limit on the
+ number of rows returned: the
+ <varname>gin_fuzzy_search_limit</varname> configuration parameter.
+ It is set to 0 (meaning no limit) by default.
+ If a non-zero limit is set, then the returned set is a subset of
+ the whole result set, chosen at random.
+ </para>
+ <para>
+ <quote>Soft</quote> means that the actual number of returned results
+ could differ somewhat from the specified limit, depending on the query
+ and the quality of the system's random number generator.
+ </para>
+ <para>
+ From experience, values in the thousands (e.g., 5000 — 20000)
+ work well.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ </sect2>
+
+ <sect2 id="gin-limit">
+ <title>Limitations</title>
+
+ <para>
+ <acronym>GIN</acronym> assumes that indexable operators are strict. This
+ means that <function>extractValue</function> will not be called at all on a null
+ item value (instead, a placeholder index entry is created automatically),
+ and <function>extractQuery</function> will not be called on a null query
+ value either (instead, the query is presumed to be unsatisfiable). Note
+ however that null key values contained within a non-null composite item
+ or query value are supported.
+ </para>
+ </sect2>
+
+ <sect2 id="gin-examples">
+ <title>Examples</title>
+
+ <para>
+ The core <productname>PostgreSQL</productname> distribution
+ includes the <acronym>GIN</acronym> operator classes previously shown in
+ <xref linkend="gin-builtin-opclasses-table"/>.
+ The following <filename>contrib</filename> modules also contain
+ <acronym>GIN</acronym> operator classes:
+
+ <variablelist>
+ <varlistentry>
+ <term><filename>btree_gin</filename></term>
+ <listitem>
+ <para>B-tree equivalent functionality for several data types</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>hstore</filename></term>
+ <listitem>
+ <para>Module for storing (key, value) pairs</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>intarray</filename></term>
+ <listitem>
+ <para>Enhanced support for <type>int[]</type></para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><filename>pg_trgm</filename></term>
+ <listitem>
+ <para>Text similarity using trigram matching</para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </sect2>
+
+ </sect1>
+
+ <sect1 id="brin">
+ <title>BRIN Indexes</title>
+
+ <indexterm>
+ <primary>index</primary>
+ <secondary>BRIN</secondary>
+ </indexterm>
+
+ <sect2 id="brin-intro">
+ <title>Introduction</title>
+
+ <para>
+ <acronym>BRIN</acronym> stands for Block Range Index.
+ <acronym>BRIN</acronym> is designed for handling very large tables
+ in which certain columns have some natural correlation with their
+ physical location within the table.
+ </para>
+
+ <para>
+ <acronym>BRIN</acronym> works in terms of <firstterm>block ranges</firstterm>
+ (or <quote>page ranges</quote>).
+ A block range is a group of pages that are physically
+ adjacent in the table; for each block range, some summary info is stored
+ by the index.
+ For example, a table storing a store's sale orders might have
+ a date column on which each order was placed, and most of the time
+ the entries for earlier orders will appear earlier in the table as well;
+ a table storing a ZIP code column might have all codes for a city
+ grouped together naturally.
+ </para>
+
+ <para>
+ <acronym>BRIN</acronym> indexes can satisfy queries via regular bitmap
+ index scans, and will return all tuples in all pages within each range if
+ the summary info stored by the index is <firstterm>consistent</firstterm> with the
+ query conditions.
+ The query executor is in charge of rechecking these tuples and discarding
+ those that do not match the query conditions — in other words, these
+ indexes are lossy.
+ Because a <acronym>BRIN</acronym> index is very small, scanning the index
+ adds little overhead compared to a sequential scan, but may avoid scanning
+ large parts of the table that are known not to contain matching tuples.
+ </para>
+
+ <para>
+ The specific data that a <acronym>BRIN</acronym> index will store,
+ as well as the specific queries that the index will be able to satisfy,
+ depend on the operator class selected for each column of the index.
+ Data types having a linear sort order can have operator classes that
+ store the minimum and maximum value within each block range, for instance;
+ geometrical types might store the bounding box for all the objects
+ in the block range.
+ </para>
+
+ <para>
+ The size of the block range is determined at index creation time by
+ the <literal>pages_per_range</literal> storage parameter. The number of index
+ entries will be equal to the size of the relation in pages divided by
+ the selected value for <literal>pages_per_range</literal>. Therefore, the smaller
+ the number, the larger the index becomes (because of the need to
+ store more index entries), but at the same time the summary data stored can
+ be more precise and more data blocks can be skipped during an index scan.
+ </para>
+
+ <sect3 id="brin-operation">
+ <title>Index Maintenance</title>
+
+ <para>
+ At the time of creation, all existing heap pages are scanned and a
+ summary index tuple is created for each range, including the
+ possibly-incomplete range at the end.
+ As new pages are filled with data, page ranges that are already
+ summarized will cause the summary information to be updated with data
+ from the new tuples.
+ When a new page is created that does not fall within the last
+ summarized range, the range that the new page belongs to
+ does not automatically acquire a summary tuple;
+ those tuples remain unsummarized until a summarization run is
+ invoked later, creating the initial summary for that range.
+ </para>
+
+ <para>
+ There are several ways to trigger the initial summarization of a page range.
+ If the table is vacuumed, either manually or by
+ <link linkend="autovacuum">autovacuum</link>, all existing unsummarized
+ page ranges are summarized.
+ Also, if the index's
+ <xref linkend="index-reloption-autosummarize"/> parameter is enabled,
+ which it isn't by default,
+ whenever autovacuum runs in that database, summarization will occur for all
+ unsummarized page ranges that have been filled,
+ regardless of whether the table itself is processed by autovacuum; see below.
+ </para>
+
+ <para>
+ Lastly, the following functions can be used (while these functions run,
+ <xref linkend="guc-search-path"/> is temporarily changed to
+ <literal>pg_catalog, pg_temp</literal>):
+ <simplelist>
+ <member>
+ <function>brin_summarize_new_values(regclass)</function>
+ which summarizes all unsummarized ranges;
+ </member>
+ <member>
+ <function>brin_summarize_range(regclass, bigint)</function>
+ which summarizes only the range containing the given page,
+ if it is unsummarized.
+ </member>
+ </simplelist>
+ </para>
+
+ <para>
+ When autosummarization is enabled, a request is sent to
+ <literal>autovacuum</literal> to execute a targeted summarization
+ for a block range when an insertion is detected for the first item
+ of the first page of the next block range,
+ to be fulfilled the next time an autovacuum
+ worker finishes running in the
+ same database. If the request queue is full, the request is not recorded
+ and a message is sent to the server log:
+ <screen>
+ LOG: request for BRIN range summarization for index "brin_wi_idx" page 128 was not recorded
+ </screen>
+ When this happens, the range will remain unsummarized until the next
+ regular vacuum run on the table, or one of the functions mentioned above
+ are invoked.
+ </para>
+
+ <para>
+ Conversely, a range can be de-summarized using the
+ <function>brin_desummarize_range(regclass, bigint)</function> function,
+ which is useful when the index tuple is no longer a very good
+ representation because the existing values have changed.
+ See <xref linkend="functions-admin-index"/> for details.
+ </para>
+
+ </sect3>
+ </sect2>
+
+ <sect2 id="brin-builtin-opclasses">
+ <title>Built-in Operator Classes</title>
+
+ <para>
+ The core <productname>PostgreSQL</productname> distribution
+ includes the <acronym>BRIN</acronym> operator classes shown in
+ <xref linkend="brin-builtin-opclasses-table"/>.
+ </para>
+
+ <para>
+ The <firstterm>minmax</firstterm>
+ operator classes store the minimum and the maximum values appearing
+ in the indexed column within the range. The <firstterm>inclusion</firstterm>
+ operator classes store a value which includes the values in the indexed
+ column within the range. The <firstterm>bloom</firstterm> operator
+ classes build a Bloom filter for all values in the range. The
+ <firstterm>minmax-multi</firstterm> operator classes store multiple
+ minimum and maximum values, representing values appearing in the indexed
+ column within the range.
+ </para>
+
+ <table id="brin-builtin-opclasses-table">
+ <title>Built-in <acronym>BRIN</acronym> Operator Classes</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Indexable Operators</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry valign="middle" morerows="4"><literal>bit_minmax_ops</literal></entry>
+ <entry><literal>= (bit,bit)</literal></entry>
+ </row>
+ <row><entry><literal>< (bit,bit)</literal></entry></row>
+ <row><entry><literal>> (bit,bit)</literal></entry></row>
+ <row><entry><literal><= (bit,bit)</literal></entry></row>
+ <row><entry><literal>>= (bit,bit)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="12"><literal>box_inclusion_ops</literal></entry>
+ <entry><literal>@> (box,point)</literal></entry>
+ </row>
+ <row><entry><literal><< (box,box)</literal></entry></row>
+ <row><entry><literal>&< (box,box)</literal></entry></row>
+ <row><entry><literal>&> (box,box)</literal></entry></row>
+ <row><entry><literal>>> (box,box)</literal></entry></row>
+ <row><entry><literal><@ (box,box)</literal></entry></row>
+ <row><entry><literal>@> (box,box)</literal></entry></row>
+ <row><entry><literal>~= (box,box)</literal></entry></row>
+ <row><entry><literal>&& (box,box)</literal></entry></row>
+ <row><entry><literal><<| (box,box)</literal></entry></row>
+ <row><entry><literal>&<| (box,box)</literal></entry></row>
+ <row><entry><literal>|&> (box,box)</literal></entry></row>
+ <row><entry><literal>|>> (box,box)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>bpchar_bloom_ops</literal></entry>
+ <entry><literal>= (character,character)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>bpchar_minmax_ops</literal></entry>
+ <entry><literal>= (character,character)</literal></entry>
+ </row>
+ <row><entry><literal>< (character,character)</literal></entry></row>
+ <row><entry><literal><= (character,character)</literal></entry></row>
+ <row><entry><literal>> (character,character)</literal></entry></row>
+ <row><entry><literal>>= (character,character)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>bytea_bloom_ops</literal></entry>
+ <entry><literal>= (bytea,bytea)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>bytea_minmax_ops</literal></entry>
+ <entry><literal>= (bytea,bytea)</literal></entry>
+ </row>
+ <row><entry><literal>< (bytea,bytea)</literal></entry></row>
+ <row><entry><literal><= (bytea,bytea)</literal></entry></row>
+ <row><entry><literal>> (bytea,bytea)</literal></entry></row>
+ <row><entry><literal>>= (bytea,bytea)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>char_bloom_ops</literal></entry>
+ <entry><literal>= ("char","char")</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>char_minmax_ops</literal></entry>
+ <entry><literal>= ("char","char")</literal></entry>
+ </row>
+ <row><entry><literal>< ("char","char")</literal></entry></row>
+ <row><entry><literal><= ("char","char")</literal></entry></row>
+ <row><entry><literal>> ("char","char")</literal></entry></row>
+ <row><entry><literal>>= ("char","char")</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>date_bloom_ops</literal></entry>
+ <entry><literal>= (date,date)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>date_minmax_ops</literal></entry>
+ <entry><literal>= (date,date)</literal></entry>
+ </row>
+ <row><entry><literal>< (date,date)</literal></entry></row>
+ <row><entry><literal><= (date,date)</literal></entry></row>
+ <row><entry><literal>> (date,date)</literal></entry></row>
+ <row><entry><literal>>= (date,date)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>date_minmax_multi_ops</literal></entry>
+ <entry><literal>= (date,date)</literal></entry>
+ </row>
+ <row><entry><literal>< (date,date)</literal></entry></row>
+ <row><entry><literal><= (date,date)</literal></entry></row>
+ <row><entry><literal>> (date,date)</literal></entry></row>
+ <row><entry><literal>>= (date,date)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>float4_bloom_ops</literal></entry>
+ <entry><literal>= (float4,float4)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>float4_minmax_ops</literal></entry>
+ <entry><literal>= (float4,float4)</literal></entry>
+ </row>
+ <row><entry><literal>< (float4,float4)</literal></entry></row>
+ <row><entry><literal>> (float4,float4)</literal></entry></row>
+ <row><entry><literal><= (float4,float4)</literal></entry></row>
+ <row><entry><literal>>= (float4,float4)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>float4_minmax_multi_ops</literal></entry>
+ <entry><literal>= (float4,float4)</literal></entry>
+ </row>
+ <row><entry><literal>< (float4,float4)</literal></entry></row>
+ <row><entry><literal>> (float4,float4)</literal></entry></row>
+ <row><entry><literal><= (float4,float4)</literal></entry></row>
+ <row><entry><literal>>= (float4,float4)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>float8_bloom_ops</literal></entry>
+ <entry><literal>= (float8,float8)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>float8_minmax_ops</literal></entry>
+ <entry><literal>= (float8,float8)</literal></entry>
+ </row>
+ <row><entry><literal>< (float8,float8)</literal></entry></row>
+ <row><entry><literal><= (float8,float8)</literal></entry></row>
+ <row><entry><literal>> (float8,float8)</literal></entry></row>
+ <row><entry><literal>>= (float8,float8)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>float8_minmax_multi_ops</literal></entry>
+ <entry><literal>= (float8,float8)</literal></entry>
+ </row>
+ <row><entry><literal>< (float8,float8)</literal></entry></row>
+ <row><entry><literal><= (float8,float8)</literal></entry></row>
+ <row><entry><literal>> (float8,float8)</literal></entry></row>
+ <row><entry><literal>>= (float8,float8)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="5"><literal>inet_inclusion_ops</literal></entry>
+ <entry><literal><< (inet,inet)</literal></entry>
+ </row>
+ <row><entry><literal><<= (inet,inet)</literal></entry></row>
+ <row><entry><literal>>> (inet,inet)</literal></entry></row>
+ <row><entry><literal>>>= (inet,inet)</literal></entry></row>
+ <row><entry><literal>= (inet,inet)</literal></entry></row>
+ <row><entry><literal>&& (inet,inet)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>inet_bloom_ops</literal></entry>
+ <entry><literal>= (inet,inet)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>inet_minmax_ops</literal></entry>
+ <entry><literal>= (inet,inet)</literal></entry>
+ </row>
+ <row><entry><literal>< (inet,inet)</literal></entry></row>
+ <row><entry><literal><= (inet,inet)</literal></entry></row>
+ <row><entry><literal>> (inet,inet)</literal></entry></row>
+ <row><entry><literal>>= (inet,inet)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>inet_minmax_multi_ops</literal></entry>
+ <entry><literal>= (inet,inet)</literal></entry>
+ </row>
+ <row><entry><literal>< (inet,inet)</literal></entry></row>
+ <row><entry><literal><= (inet,inet)</literal></entry></row>
+ <row><entry><literal>> (inet,inet)</literal></entry></row>
+ <row><entry><literal>>= (inet,inet)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>int2_bloom_ops</literal></entry>
+ <entry><literal>= (int2,int2)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>int2_minmax_ops</literal></entry>
+ <entry><literal>= (int2,int2)</literal></entry>
+ </row>
+ <row><entry><literal>< (int2,int2)</literal></entry></row>
+ <row><entry><literal>> (int2,int2)</literal></entry></row>
+ <row><entry><literal><= (int2,int2)</literal></entry></row>
+ <row><entry><literal>>= (int2,int2)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>int2_minmax_multi_ops</literal></entry>
+ <entry><literal>= (int2,int2)</literal></entry>
+ </row>
+ <row><entry><literal>< (int2,int2)</literal></entry></row>
+ <row><entry><literal>> (int2,int2)</literal></entry></row>
+ <row><entry><literal><= (int2,int2)</literal></entry></row>
+ <row><entry><literal>>= (int2,int2)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>int4_bloom_ops</literal></entry>
+ <entry><literal>= (int4,int4)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>int4_minmax_ops</literal></entry>
+ <entry><literal>= (int4,int4)</literal></entry>
+ </row>
+ <row><entry><literal>< (int4,int4)</literal></entry></row>
+ <row><entry><literal>> (int4,int4)</literal></entry></row>
+ <row><entry><literal><= (int4,int4)</literal></entry></row>
+ <row><entry><literal>>= (int4,int4)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>int4_minmax_multi_ops</literal></entry>
+ <entry><literal>= (int4,int4)</literal></entry>
+ </row>
+ <row><entry><literal>< (int4,int4)</literal></entry></row>
+ <row><entry><literal>> (int4,int4)</literal></entry></row>
+ <row><entry><literal><= (int4,int4)</literal></entry></row>
+ <row><entry><literal>>= (int4,int4)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>int8_bloom_ops</literal></entry>
+ <entry><literal>= (bigint,bigint)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>int8_minmax_ops</literal></entry>
+ <entry><literal>= (bigint,bigint)</literal></entry>
+ </row>
+ <row><entry><literal>< (bigint,bigint)</literal></entry></row>
+ <row><entry><literal>> (bigint,bigint)</literal></entry></row>
+ <row><entry><literal><= (bigint,bigint)</literal></entry></row>
+ <row><entry><literal>>= (bigint,bigint)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>int8_minmax_multi_ops</literal></entry>
+ <entry><literal>= (bigint,bigint)</literal></entry>
+ </row>
+ <row><entry><literal>< (bigint,bigint)</literal></entry></row>
+ <row><entry><literal>> (bigint,bigint)</literal></entry></row>
+ <row><entry><literal><= (bigint,bigint)</literal></entry></row>
+ <row><entry><literal>>= (bigint,bigint)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>interval_bloom_ops</literal></entry>
+ <entry><literal>= (interval,interval)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>interval_minmax_ops</literal></entry>
+ <entry><literal>= (interval,interval)</literal></entry>
+ </row>
+ <row><entry><literal>< (interval,interval)</literal></entry></row>
+ <row><entry><literal><= (interval,interval)</literal></entry></row>
+ <row><entry><literal>> (interval,interval)</literal></entry></row>
+ <row><entry><literal>>= (interval,interval)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>interval_minmax_multi_ops</literal></entry>
+ <entry><literal>= (interval,interval)</literal></entry>
+ </row>
+ <row><entry><literal>< (interval,interval)</literal></entry></row>
+ <row><entry><literal><= (interval,interval)</literal></entry></row>
+ <row><entry><literal>> (interval,interval)</literal></entry></row>
+ <row><entry><literal>>= (interval,interval)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>macaddr_bloom_ops</literal></entry>
+ <entry><literal>= (macaddr,macaddr)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>macaddr_minmax_ops</literal></entry>
+ <entry><literal>= (macaddr,macaddr)</literal></entry>
+ </row>
+ <row><entry><literal>< (macaddr,macaddr)</literal></entry></row>
+ <row><entry><literal><= (macaddr,macaddr)</literal></entry></row>
+ <row><entry><literal>> (macaddr,macaddr)</literal></entry></row>
+ <row><entry><literal>>= (macaddr,macaddr)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>macaddr_minmax_multi_ops</literal></entry>
+ <entry><literal>= (macaddr,macaddr)</literal></entry>
+ </row>
+ <row><entry><literal>< (macaddr,macaddr)</literal></entry></row>
+ <row><entry><literal><= (macaddr,macaddr)</literal></entry></row>
+ <row><entry><literal>> (macaddr,macaddr)</literal></entry></row>
+ <row><entry><literal>>= (macaddr,macaddr)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>macaddr8_bloom_ops</literal></entry>
+ <entry><literal>= (macaddr8,macaddr8)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>macaddr8_minmax_ops</literal></entry>
+ <entry><literal>= (macaddr8,macaddr8)</literal></entry>
+ </row>
+ <row><entry><literal>< (macaddr8,macaddr8)</literal></entry></row>
+ <row><entry><literal><= (macaddr8,macaddr8)</literal></entry></row>
+ <row><entry><literal>> (macaddr8,macaddr8)</literal></entry></row>
+ <row><entry><literal>>= (macaddr8,macaddr8)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>macaddr8_minmax_multi_ops</literal></entry>
+ <entry><literal>= (macaddr8,macaddr8)</literal></entry>
+ </row>
+ <row><entry><literal>< (macaddr8,macaddr8)</literal></entry></row>
+ <row><entry><literal><= (macaddr8,macaddr8)</literal></entry></row>
+ <row><entry><literal>> (macaddr8,macaddr8)</literal></entry></row>
+ <row><entry><literal>>= (macaddr8,macaddr8)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>name_bloom_ops</literal></entry>
+ <entry><literal>= (name,name)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>name_minmax_ops</literal></entry>
+ <entry><literal>= (name,name)</literal></entry>
+ </row>
+ <row><entry><literal>< (name,name)</literal></entry></row>
+ <row><entry><literal><= (name,name)</literal></entry></row>
+ <row><entry><literal>> (name,name)</literal></entry></row>
+ <row><entry><literal>>= (name,name)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>numeric_bloom_ops</literal></entry>
+ <entry><literal>= (numeric,numeric)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>numeric_minmax_ops</literal></entry>
+ <entry><literal>= (numeric,numeric)</literal></entry>
+ </row>
+ <row><entry><literal>< (numeric,numeric)</literal></entry></row>
+ <row><entry><literal><= (numeric,numeric)</literal></entry></row>
+ <row><entry><literal>> (numeric,numeric)</literal></entry></row>
+ <row><entry><literal>>= (numeric,numeric)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>numeric_minmax_multi_ops</literal></entry>
+ <entry><literal>= (numeric,numeric)</literal></entry>
+ </row>
+ <row><entry><literal>< (numeric,numeric)</literal></entry></row>
+ <row><entry><literal><= (numeric,numeric)</literal></entry></row>
+ <row><entry><literal>> (numeric,numeric)</literal></entry></row>
+ <row><entry><literal>>= (numeric,numeric)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>oid_bloom_ops</literal></entry>
+ <entry><literal>= (oid,oid)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>oid_minmax_ops</literal></entry>
+ <entry><literal>= (oid,oid)</literal></entry>
+ </row>
+ <row><entry><literal>< (oid,oid)</literal></entry></row>
+ <row><entry><literal>> (oid,oid)</literal></entry></row>
+ <row><entry><literal><= (oid,oid)</literal></entry></row>
+ <row><entry><literal>>= (oid,oid)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>oid_minmax_multi_ops</literal></entry>
+ <entry><literal>= (oid,oid)</literal></entry>
+ </row>
+ <row><entry><literal>< (oid,oid)</literal></entry></row>
+ <row><entry><literal>> (oid,oid)</literal></entry></row>
+ <row><entry><literal><= (oid,oid)</literal></entry></row>
+ <row><entry><literal>>= (oid,oid)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>pg_lsn_bloom_ops</literal></entry>
+ <entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>pg_lsn_minmax_ops</literal></entry>
+ <entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
+ </row>
+ <row><entry><literal>< (pg_lsn,pg_lsn)</literal></entry></row>
+ <row><entry><literal>> (pg_lsn,pg_lsn)</literal></entry></row>
+ <row><entry><literal><= (pg_lsn,pg_lsn)</literal></entry></row>
+ <row><entry><literal>>= (pg_lsn,pg_lsn)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>pg_lsn_minmax_multi_ops</literal></entry>
+ <entry><literal>= (pg_lsn,pg_lsn)</literal></entry>
+ </row>
+ <row><entry><literal>< (pg_lsn,pg_lsn)</literal></entry></row>
+ <row><entry><literal>> (pg_lsn,pg_lsn)</literal></entry></row>
+ <row><entry><literal><= (pg_lsn,pg_lsn)</literal></entry></row>
+ <row><entry><literal>>= (pg_lsn,pg_lsn)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="13"><literal>range_inclusion_ops</literal></entry>
+ <entry><literal>= (anyrange,anyrange)</literal></entry>
+ </row>
+ <row><entry><literal>< (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal><= (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>>= (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>> (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>&& (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>@> (anyrange,anyelement)</literal></entry></row>
+ <row><entry><literal>@> (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal><@ (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal><< (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>>> (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>&< (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>&> (anyrange,anyrange)</literal></entry></row>
+ <row><entry><literal>-|- (anyrange,anyrange)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>text_bloom_ops</literal></entry>
+ <entry><literal>= (text,text)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>text_minmax_ops</literal></entry>
+ <entry><literal>= (text,text)</literal></entry>
+ </row>
+ <row><entry><literal>< (text,text)</literal></entry></row>
+ <row><entry><literal><= (text,text)</literal></entry></row>
+ <row><entry><literal>> (text,text)</literal></entry></row>
+ <row><entry><literal>>= (text,text)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>tid_bloom_ops</literal></entry>
+ <entry><literal>= (tid,tid)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>tid_minmax_ops</literal></entry>
+ <entry><literal>= (tid,tid)</literal></entry>
+ </row>
+ <row><entry><literal>< (tid,tid)</literal></entry></row>
+ <row><entry><literal>> (tid,tid)</literal></entry></row>
+ <row><entry><literal><= (tid,tid)</literal></entry></row>
+ <row><entry><literal>>= (tid,tid)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>tid_minmax_multi_ops</literal></entry>
+ <entry><literal>= (tid,tid)</literal></entry>
+ </row>
+ <row><entry><literal>< (tid,tid)</literal></entry></row>
+ <row><entry><literal>> (tid,tid)</literal></entry></row>
+ <row><entry><literal><= (tid,tid)</literal></entry></row>
+ <row><entry><literal>>= (tid,tid)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>timestamp_bloom_ops</literal></entry>
+ <entry><literal>= (timestamp,timestamp)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>timestamp_minmax_ops</literal></entry>
+ <entry><literal>= (timestamp,timestamp)</literal></entry>
+ </row>
+ <row><entry><literal>< (timestamp,timestamp)</literal></entry></row>
+ <row><entry><literal><= (timestamp,timestamp)</literal></entry></row>
+ <row><entry><literal>> (timestamp,timestamp)</literal></entry></row>
+ <row><entry><literal>>= (timestamp,timestamp)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>timestamp_minmax_multi_ops</literal></entry>
+ <entry><literal>= (timestamp,timestamp)</literal></entry>
+ </row>
+ <row><entry><literal>< (timestamp,timestamp)</literal></entry></row>
+ <row><entry><literal><= (timestamp,timestamp)</literal></entry></row>
+ <row><entry><literal>> (timestamp,timestamp)</literal></entry></row>
+ <row><entry><literal>>= (timestamp,timestamp)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>timestamptz_bloom_ops</literal></entry>
+ <entry><literal>= (timestamptz,timestamptz)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>timestamptz_minmax_ops</literal></entry>
+ <entry><literal>= (timestamptz,timestamptz)</literal></entry>
+ </row>
+ <row><entry><literal>< (timestamptz,timestamptz)</literal></entry></row>
+ <row><entry><literal><= (timestamptz,timestamptz)</literal></entry></row>
+ <row><entry><literal>> (timestamptz,timestamptz)</literal></entry></row>
+ <row><entry><literal>>= (timestamptz,timestamptz)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>timestamptz_minmax_multi_ops</literal></entry>
+ <entry><literal>= (timestamptz,timestamptz)</literal></entry>
+ </row>
+ <row><entry><literal>< (timestamptz,timestamptz)</literal></entry></row>
+ <row><entry><literal><= (timestamptz,timestamptz)</literal></entry></row>
+ <row><entry><literal>> (timestamptz,timestamptz)</literal></entry></row>
+ <row><entry><literal>>= (timestamptz,timestamptz)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>time_bloom_ops</literal></entry>
+ <entry><literal>= (time,time)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>time_minmax_ops</literal></entry>
+ <entry><literal>= (time,time)</literal></entry>
+ </row>
+ <row><entry><literal>< (time,time)</literal></entry></row>
+ <row><entry><literal><= (time,time)</literal></entry></row>
+ <row><entry><literal>> (time,time)</literal></entry></row>
+ <row><entry><literal>>= (time,time)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>time_minmax_multi_ops</literal></entry>
+ <entry><literal>= (time,time)</literal></entry>
+ </row>
+ <row><entry><literal>< (time,time)</literal></entry></row>
+ <row><entry><literal><= (time,time)</literal></entry></row>
+ <row><entry><literal>> (time,time)</literal></entry></row>
+ <row><entry><literal>>= (time,time)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>timetz_bloom_ops</literal></entry>
+ <entry><literal>= (timetz,timetz)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>timetz_minmax_ops</literal></entry>
+ <entry><literal>= (timetz,timetz)</literal></entry>
+ </row>
+ <row><entry><literal>< (timetz,timetz)</literal></entry></row>
+ <row><entry><literal><= (timetz,timetz)</literal></entry></row>
+ <row><entry><literal>> (timetz,timetz)</literal></entry></row>
+ <row><entry><literal>>= (timetz,timetz)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>timetz_minmax_multi_ops</literal></entry>
+ <entry><literal>= (timetz,timetz)</literal></entry>
+ </row>
+ <row><entry><literal>< (timetz,timetz)</literal></entry></row>
+ <row><entry><literal><= (timetz,timetz)</literal></entry></row>
+ <row><entry><literal>> (timetz,timetz)</literal></entry></row>
+ <row><entry><literal>>= (timetz,timetz)</literal></entry></row>
+
+ <row>
+ <entry valign="middle"><literal>uuid_bloom_ops</literal></entry>
+ <entry><literal>= (uuid,uuid)</literal></entry>
+ </row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>uuid_minmax_ops</literal></entry>
+ <entry><literal>= (uuid,uuid)</literal></entry>
+ </row>
+ <row><entry><literal>< (uuid,uuid)</literal></entry></row>
+ <row><entry><literal>> (uuid,uuid)</literal></entry></row>
+ <row><entry><literal><= (uuid,uuid)</literal></entry></row>
+ <row><entry><literal>>= (uuid,uuid)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>uuid_minmax_multi_ops</literal></entry>
+ <entry><literal>= (uuid,uuid)</literal></entry>
+ </row>
+ <row><entry><literal>< (uuid,uuid)</literal></entry></row>
+ <row><entry><literal>> (uuid,uuid)</literal></entry></row>
+ <row><entry><literal><= (uuid,uuid)</literal></entry></row>
+ <row><entry><literal>>= (uuid,uuid)</literal></entry></row>
+
+ <row>
+ <entry valign="middle" morerows="4"><literal>varbit_minmax_ops</literal></entry>
+ <entry><literal>= (varbit,varbit)</literal></entry>
+ </row>
+ <row><entry><literal>< (varbit,varbit)</literal></entry></row>
+ <row><entry><literal>> (varbit,varbit)</literal></entry></row>
+ <row><entry><literal><= (varbit,varbit)</literal></entry></row>
+ <row><entry><literal>>= (varbit,varbit)</literal></entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <sect3 id="brin-builtin-opclasses--parameters">
+ <title>Operator Class Parameters</title>
+
+ <para>
+ Some of the built-in operator classes allow specifying parameters affecting
+ behavior of the operator class. Each operator class has its own set of
+ allowed parameters. Only the <literal>bloom</literal> and <literal>minmax-multi</literal>
+ operator classes allow specifying parameters:
+ </para>
+
+ <para>
+ bloom operator classes accept these parameters:
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>n_distinct_per_range</literal></term>
+ <listitem>
+ <para>
+ Defines the estimated number of distinct non-null values in the block
+ range, used by <acronym>BRIN</acronym> bloom indexes for sizing of the
+ Bloom filter. It behaves similarly to <literal>n_distinct</literal> option
+ for <xref linkend="sql-altertable"/>. When set to a positive value,
+ each block range is assumed to contain this number of distinct non-null
+ values. When set to a negative value, which must be greater than or
+ equal to -1, the number of distinct non-null values is assumed to grow linearly with
+ the maximum possible number of tuples in the block range (about 290
+ rows per block). The default value is <literal>-0.1</literal>, and
+ the minimum number of distinct non-null values is <literal>16</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>false_positive_rate</literal></term>
+ <listitem>
+ <para>
+ Defines the desired false positive rate used by <acronym>BRIN</acronym>
+ bloom indexes for sizing of the Bloom filter. The values must be
+ between 0.0001 and 0.25. The default value is 0.01, which is 1% false
+ positive rate.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+
+ <para>
+ minmax-multi operator classes accept these parameters:
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>values_per_range</literal></term>
+ <listitem>
+ <para>
+ Defines the maximum number of values stored by <acronym>BRIN</acronym>
+ minmax indexes to summarize a block range. Each value may represent
+ either a point, or a boundary of an interval. Values must be between
+ 8 and 256, and the default value is 32.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </sect3>
+
+ </sect2>
+
+ <sect2 id="brin-extensibility">
+ <title>Extensibility</title>
+
+ <para>
+ The <acronym>BRIN</acronym> interface has a high level of abstraction,
+ requiring the access method implementer only to implement the semantics
+ of the data type being accessed. The <acronym>BRIN</acronym> layer
+ itself takes care of concurrency, logging and searching the index structure.
+ </para>
+
+ <para>
+ All it takes to get a <acronym>BRIN</acronym> access method working is to
+ implement a few user-defined methods, which define the behavior of
+ summary values stored in the index and the way they interact with
+ scan keys.
+ In short, <acronym>BRIN</acronym> combines
+ extensibility with generality, code reuse, and a clean interface.
+ </para>
+
+ <para>
+ There are four methods that an operator class for <acronym>BRIN</acronym>
+ must provide:
+
+ <variablelist>
+ <varlistentry>
+ <term><function>BrinOpcInfo *opcInfo(Oid type_oid)</function></term>
+ <listitem>
+ <para>
+ Returns internal information about the indexed columns' summary data.
+ The return value must point to a palloc'd <structname>BrinOpcInfo</structname>,
+ which has this definition:
+<programlisting>
+typedef struct BrinOpcInfo
+{
+ /* Number of columns stored in an index column of this opclass */
+ uint16 oi_nstored;
+
+ /* Opaque pointer for the opclass' private use */
+ void *oi_opaque;
+
+ /* Type cache entries of the stored columns */
+ TypeCacheEntry *oi_typcache[FLEXIBLE_ARRAY_MEMBER];
+} BrinOpcInfo;
+</programlisting>
+ <structname>BrinOpcInfo</structname>.<structfield>oi_opaque</structfield> can be used by the
+ operator class routines to pass information between support functions
+ during an index scan.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>bool consistent(BrinDesc *bdesc, BrinValues *column,
+ ScanKey *keys, int nkeys)</function></term>
+ <listitem>
+ <para>
+ Returns whether all the ScanKey entries are consistent with the given
+ indexed values for a range.
+ The attribute number to use is passed as part of the scan key.
+ Multiple scan keys for the same attribute may be passed at once; the
+ number of entries is determined by the <literal>nkeys</literal> parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>bool consistent(BrinDesc *bdesc, BrinValues *column,
+ ScanKey key)</function></term>
+ <listitem>
+ <para>
+ Returns whether the ScanKey is consistent with the given indexed
+ values for a range.
+ The attribute number to use is passed as part of the scan key.
+ This is an older backward-compatible variant of the consistent function.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>bool addValue(BrinDesc *bdesc, BrinValues *column,
+ Datum newval, bool isnull)</function></term>
+ <listitem>
+ <para>
+ Given an index tuple and an indexed value, modifies the indicated
+ attribute of the tuple so that it additionally represents the new value.
+ If any modification was done to the tuple, <literal>true</literal> is
+ returned.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>bool unionTuples(BrinDesc *bdesc, BrinValues *a,
+ BrinValues *b)</function></term>
+ <listitem>
+ <para>
+ Consolidates two index tuples. Given two index tuples, modifies the
+ indicated attribute of the first of them so that it represents both tuples.
+ The second tuple is not modified.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ An operator class for <acronym>BRIN</acronym> can optionally specify the
+ following method:
+
+ <variablelist>
+ <varlistentry>
+ <term><function>void options(local_relopts *relopts)</function></term>
+ <listitem>
+ <para>
+ Defines a set of user-visible parameters that control operator class
+ behavior.
+ </para>
+
+ <para>
+ The <function>options</function> function is passed a pointer to a
+ <structname>local_relopts</structname> struct, which needs to be
+ filled with a set of operator class specific options. The options
+ can be accessed from other support functions using the
+ <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
+ <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
+ </para>
+
+ <para>
+ Since both key extraction of indexed values and representation of the
+ key in <acronym>BRIN</acronym> are flexible, they may depend on
+ user-specified parameters.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ The core distribution includes support for four types of operator classes:
+ minmax, minmax-multi, inclusion and bloom. Operator class definitions
+ using them are shipped for in-core data types as appropriate. Additional
+ operator classes can be defined by the user for other data types using
+ equivalent definitions, without having to write any source code;
+ appropriate catalog entries being declared is enough. Note that
+ assumptions about the semantics of operator strategies are embedded in the
+ support functions' source code.
+ </para>
+
+ <para>
+ Operator classes that implement completely different semantics are also
+ possible, provided implementations of the four main support functions
+ described above are written. Note that backwards compatibility across major
+ releases is not guaranteed: for example, additional support functions might
+ be required in later releases.
+ </para>
+
+ <para>
+ To write an operator class for a data type that implements a totally
+ ordered set, it is possible to use the minmax support functions
+ alongside the corresponding operators, as shown in
+ <xref linkend="brin-extensibility-minmax-table"/>.
+ All operator class members (functions and operators) are mandatory.
+ </para>
+
+ <table id="brin-extensibility-minmax-table">
+ <title>Function and Support Numbers for Minmax Operator Classes</title>
+ <tgroup cols="2">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Operator class member</entry>
+ <entry>Object</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>Support Function 1</entry>
+ <entry>internal function <function>brin_minmax_opcinfo()</function></entry>
+ </row>
+ <row>
+ <entry>Support Function 2</entry>
+ <entry>internal function <function>brin_minmax_add_value()</function></entry>
+ </row>
+ <row>
+ <entry>Support Function 3</entry>
+ <entry>internal function <function>brin_minmax_consistent()</function></entry>
+ </row>
+ <row>
+ <entry>Support Function 4</entry>
+ <entry>internal function <function>brin_minmax_union()</function></entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 1</entry>
+ <entry>operator less-than</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 2</entry>
+ <entry>operator less-than-or-equal-to</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 3</entry>
+ <entry>operator equal-to</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 4</entry>
+ <entry>operator greater-than-or-equal-to</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 5</entry>
+ <entry>operator greater-than</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ To write an operator class for a complex data type which has values
+ included within another type, it's possible to use the inclusion support
+ functions alongside the corresponding operators, as shown
+ in <xref linkend="brin-extensibility-inclusion-table"/>. It requires
+ only a single additional function, which can be written in any language.
+ More functions can be defined for additional functionality. All operators
+ are optional. Some operators require other operators, as shown as
+ dependencies on the table.
+ </para>
+
+ <table id="brin-extensibility-inclusion-table">
+ <title>Function and Support Numbers for Inclusion Operator Classes</title>
+ <tgroup cols="3">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="2*"/>
+ <colspec colname="col3" colwidth="1*"/>
+ <thead>
+ <row>
+ <entry>Operator class member</entry>
+ <entry>Object</entry>
+ <entry>Dependency</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>Support Function 1</entry>
+ <entry>internal function <function>brin_inclusion_opcinfo()</function></entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Support Function 2</entry>
+ <entry>internal function <function>brin_inclusion_add_value()</function></entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Support Function 3</entry>
+ <entry>internal function <function>brin_inclusion_consistent()</function></entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Support Function 4</entry>
+ <entry>internal function <function>brin_inclusion_union()</function></entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Support Function 11</entry>
+ <entry>function to merge two elements</entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Support Function 12</entry>
+ <entry>optional function to check whether two elements are mergeable</entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Support Function 13</entry>
+ <entry>optional function to check if an element is contained within another</entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Support Function 14</entry>
+ <entry>optional function to check whether an element is empty</entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 1</entry>
+ <entry>operator left-of</entry>
+ <entry>Operator Strategy 4</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 2</entry>
+ <entry>operator does-not-extend-to-the-right-of</entry>
+ <entry>Operator Strategy 5</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 3</entry>
+ <entry>operator overlaps</entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 4</entry>
+ <entry>operator does-not-extend-to-the-left-of</entry>
+ <entry>Operator Strategy 1</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 5</entry>
+ <entry>operator right-of</entry>
+ <entry>Operator Strategy 2</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 6, 18</entry>
+ <entry>operator same-as-or-equal-to</entry>
+ <entry>Operator Strategy 7</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 7, 16, 24, 25</entry>
+ <entry>operator contains-or-equal-to</entry>
+ <entry></entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 8, 26, 27</entry>
+ <entry>operator is-contained-by-or-equal-to</entry>
+ <entry>Operator Strategy 3</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 9</entry>
+ <entry>operator does-not-extend-above</entry>
+ <entry>Operator Strategy 11</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 10</entry>
+ <entry>operator is-below</entry>
+ <entry>Operator Strategy 12</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 11</entry>
+ <entry>operator is-above</entry>
+ <entry>Operator Strategy 9</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 12</entry>
+ <entry>operator does-not-extend-below</entry>
+ <entry>Operator Strategy 10</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 20</entry>
+ <entry>operator less-than</entry>
+ <entry>Operator Strategy 5</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 21</entry>
+ <entry>operator less-than-or-equal-to</entry>
+ <entry>Operator Strategy 5</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 22</entry>
+ <entry>operator greater-than</entry>
+ <entry>Operator Strategy 1</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 23</entry>
+ <entry>operator greater-than-or-equal-to</entry>
+ <entry>Operator Strategy 1</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ Support function numbers 1 through 10 are reserved for the BRIN internal
+ functions, so the SQL level functions start with number 11. Support
+ function number 11 is the main function required to build the index.
+ It should accept two arguments with the same data type as the operator class,
+ and return the union of them. The inclusion operator class can store union
+ values with different data types if it is defined with the
+ <literal>STORAGE</literal> parameter. The return value of the union
+ function should match the <literal>STORAGE</literal> data type.
+ </para>
+
+ <para>
+ Support function numbers 12 and 14 are provided to support
+ irregularities of built-in data types. Function number 12
+ is used to support network addresses from different families which
+ are not mergeable. Function number 14 is used to support
+ empty ranges. Function number 13 is an optional but
+ recommended one, which allows the new value to be checked before
+ it is passed to the union function. As the BRIN framework can shortcut
+ some operations when the union is not changed, using this
+ function can improve index performance.
+ </para>
+
+ <para>
+ To write an operator class for a data type that implements only an equality
+ operator and supports hashing, it is possible to use the bloom support procedures
+ alongside the corresponding operators, as shown in
+ <xref linkend="brin-extensibility-bloom-table"/>.
+ All operator class members (procedures and operators) are mandatory.
+ </para>
+
+ <table id="brin-extensibility-bloom-table">
+ <title>Procedure and Support Numbers for Bloom Operator Classes</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Operator class member</entry>
+ <entry>Object</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>Support Procedure 1</entry>
+ <entry>internal function <function>brin_bloom_opcinfo()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 2</entry>
+ <entry>internal function <function>brin_bloom_add_value()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 3</entry>
+ <entry>internal function <function>brin_bloom_consistent()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 4</entry>
+ <entry>internal function <function>brin_bloom_union()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 5</entry>
+ <entry>internal function <function>brin_bloom_options()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 11</entry>
+ <entry>function to compute hash of an element</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 1</entry>
+ <entry>operator equal-to</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ Support procedure numbers 1-10 are reserved for the BRIN internal
+ functions, so the SQL level functions start with number 11. Support
+ function number 11 is the main function required to build the index.
+ It should accept one argument with the same data type as the operator class,
+ and return a hash of the value.
+ </para>
+
+ <para>
+ The minmax-multi operator class is also intended for data types implementing
+ a totally ordered set, and may be seen as a simple extension of the minmax
+ operator class. While minmax operator class summarizes values from each block
+ range into a single contiguous interval, minmax-multi allows summarization
+ into multiple smaller intervals to improve handling of outlier values.
+ It is possible to use the minmax-multi support procedures alongside the
+ corresponding operators, as shown in
+ <xref linkend="brin-extensibility-minmax-multi-table"/>.
+ All operator class members (procedures and operators) are mandatory.
+ </para>
+
+ <table id="brin-extensibility-minmax-multi-table">
+ <title>Procedure and Support Numbers for minmax-multi Operator Classes</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Operator class member</entry>
+ <entry>Object</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>Support Procedure 1</entry>
+ <entry>internal function <function>brin_minmax_multi_opcinfo()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 2</entry>
+ <entry>internal function <function>brin_minmax_multi_add_value()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 3</entry>
+ <entry>internal function <function>brin_minmax_multi_consistent()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 4</entry>
+ <entry>internal function <function>brin_minmax_multi_union()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 5</entry>
+ <entry>internal function <function>brin_minmax_multi_options()</function></entry>
+ </row>
+ <row>
+ <entry>Support Procedure 11</entry>
+ <entry>function to compute distance between two values (length of a range)</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 1</entry>
+ <entry>operator less-than</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 2</entry>
+ <entry>operator less-than-or-equal-to</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 3</entry>
+ <entry>operator equal-to</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 4</entry>
+ <entry>operator greater-than-or-equal-to</entry>
+ </row>
+ <row>
+ <entry>Operator Strategy 5</entry>
+ <entry>operator greater-than</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ Both minmax and inclusion operator classes support cross-data-type
+ operators, though with these the dependencies become more complicated.
+ The minmax operator class requires a full set of operators to be
+ defined with both arguments having the same data type. It allows
+ additional data types to be supported by defining extra sets
+ of operators. Inclusion operator class operator strategies are dependent
+ on another operator strategy as shown in
+ <xref linkend="brin-extensibility-inclusion-table"/>, or the same
+ operator strategy as themselves. They require the dependency
+ operator to be defined with the <literal>STORAGE</literal> data type as the
+ left-hand-side argument and the other supported data type to be the
+ right-hand-side argument of the supported operator. See
+ <literal>float4_minmax_ops</literal> as an example of minmax, and
+ <literal>box_inclusion_ops</literal> as an example of inclusion.
+ </para>
+ </sect2>
+ </sect1>
+
+ <sect1 id="hash-index">
+ <title>Hash Indexes</title>
+
+ <indexterm>
+ <primary>index</primary>
+ <secondary>Hash</secondary>
+ </indexterm>
+
+ <sect2 id="hash-intro">
+ <title>Overview</title>
+
+ <para>
+ <productname>PostgreSQL</productname>
+ includes an implementation of persistent on-disk hash indexes,
+ which are fully crash recoverable. Any data type can be indexed by a
+ hash index, including data types that do not have a well-defined linear
+ ordering. Hash indexes store only the hash value of the data being
+ indexed, thus there are no restrictions on the size of the data column
+ being indexed.
+ </para>
+
+ <para>
+ Hash indexes support only single-column indexes and do not allow
+ uniqueness checking.
+ </para>
+
+ <para>
+ Hash indexes support only the <literal>=</literal> operator,
+ so WHERE clauses that specify range operations will not be able to take
+ advantage of hash indexes.
+ </para>
+
+ <para>
+ Each hash index tuple stores just the 4-byte hash value, not the actual
+ column value. As a result, hash indexes may be much smaller than B-trees
+ when indexing longer data items such as UUIDs, URLs, etc. The absence of
+ the column value also makes all hash index scans lossy. Hash indexes may
+ take part in bitmap index scans and backward scans.
+ </para>
+
+ <para>
+ Hash indexes are best optimized for SELECT and UPDATE-heavy workloads
+ that use equality scans on larger tables. In a B-tree index, searches must
+ descend through the tree until the leaf page is found. In tables with
+ millions of rows, this descent can increase access time to data. The
+ equivalent of a leaf page in a hash index is referred to as a bucket page. In
+ contrast, a hash index allows accessing the bucket pages directly,
+ thereby potentially reducing index access time in larger tables. This
+ reduction in "logical I/O" becomes even more pronounced on indexes/data
+ larger than shared_buffers/RAM.
+ </para>
+
+ <para>
+ Hash indexes have been designed to cope with uneven distributions of
+ hash values. Direct access to the bucket pages works well if the hash
+ values are evenly distributed. When inserts mean that the bucket page
+ becomes full, additional overflow pages are chained to that specific
+ bucket page, locally expanding the storage for index tuples that match
+ that hash value. When scanning a hash bucket during queries, we need to
+ scan through all of the overflow pages. Thus an unbalanced hash index
+ might actually be worse than a B-tree in terms of number of block
+ accesses required, for some data.
+ </para>
+
+ <para>
+ As a result of the overflow cases, we can say that hash indexes are
+ most suitable for unique, nearly unique data or data with a low number
+ of rows per hash bucket.
+ One possible way to avoid problems is to exclude highly non-unique
+ values from the index using a partial index condition, but this may
+ not be suitable in many cases.
+ </para>
+
+ <para>
+ Like B-Trees, hash indexes perform simple index tuple deletion. This
+ is a deferred maintenance operation that deletes index tuples that are
+ known to be safe to delete (those whose item identifier's LP_DEAD bit
+ is already set). If an insert finds no space is available on a page we
+ try to avoid creating a new overflow page by attempting to remove dead
+ index tuples. Removal cannot occur if the page is pinned at that time.
+ Deletion of dead index pointers also occurs during VACUUM.
+ </para>
+
+ <para>
+ If it can, VACUUM will also try to squeeze the index tuples onto as
+ few overflow pages as possible, minimizing the overflow chain. If an
+ overflow page becomes empty, overflow pages can be recycled for reuse
+ in other buckets, though we never return them to the operating system.
+ There is currently no provision to shrink a hash index, other than by
+ rebuilding it with REINDEX.
+ There is no provision for reducing the number of buckets, either.
+ </para>
+
+ <para>
+ Hash indexes may expand the number of bucket pages as the number of
+ rows indexed grows. The hash key-to-bucket-number mapping is chosen so that
+ the index can be incrementally expanded. When a new bucket is to be added to
+ the index, exactly one existing bucket will need to be "split", with some of
+ its tuples being transferred to the new bucket according to the updated
+ key-to-bucket-number mapping.
+ </para>
+
+ <para>
+ The expansion occurs in the foreground, which could increase execution
+ time for user inserts. Thus, hash indexes may not be suitable for tables
+ with rapidly increasing number of rows.
+ </para>
+
+ </sect2>
+
+ <sect2 id="hash-implementation">
+ <title>Implementation</title>
+
+ <para>
+ There are four kinds of pages in a hash index: the meta page (page zero),
+ which contains statically allocated control information; primary bucket
+ pages; overflow pages; and bitmap pages, which keep track of overflow
+ pages that have been freed and are available for re-use. For addressing
+ purposes, bitmap pages are regarded as a subset of the overflow pages.
+ </para>
+
+ <para>
+ Both scanning the index and inserting tuples require locating the bucket
+ where a given tuple ought to be located. To do this, we need the bucket
+ count, highmask, and lowmask from the metapage; however, it's undesirable
+ for performance reasons to have to have to lock and pin the metapage for
+ every such operation. Instead, we retain a cached copy of the metapage
+ in each backend's relcache entry. This will produce the correct bucket
+ mapping as long as the target bucket hasn't been split since the last
+ cache refresh.
+ </para>
+
+ <para>
+ Primary bucket pages and overflow pages are allocated independently since
+ any given index might need more or fewer overflow pages relative to its
+ number of buckets. The hash code uses an interesting set of addressing
+ rules to support a variable number of overflow pages while not having to
+ move primary bucket pages around after they are created.
+ </para>
+
+ <para>
+ Each row in the table indexed is represented by a single index tuple in
+ the hash index. Hash index tuples are stored in bucket pages, and if
+ they exist, overflow pages. We speed up searches by keeping the index entries
+ in any one index page sorted by hash code, thus allowing binary search to be
+ used within an index page. Note however that there is *no* assumption about
+ the relative ordering of hash codes across different index pages of a bucket.
+ </para>
+
+ <para>
+ The bucket splitting algorithms to expand the hash index are too complex to
+ be worthy of mention here, though are described in more detail in
+ <filename>src/backend/access/hash/README</filename>.
+ The split algorithm is crash safe and can be restarted if not completed
+ successfully.
+ </para>
+
+ </sect2>
+
+ </sect1>
+
+</chapter>
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 73b497bcf8..0235c0e352 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -257,12 +257,7 @@ break is not needed in a wider output rendering.
&indexam;
&generic-wal;
&custom-rmgr;
- &btree;
- &gist;
- &spgist;
- &gin;
- &brin;
- &hash;
+ &indextypes;
&storage;
&transaction;
&bki;
diff --git a/doc/src/sgml/spgist.sgml b/doc/src/sgml/spgist.sgml
deleted file mode 100644
index 102f8627bd..0000000000
--- a/doc/src/sgml/spgist.sgml
+++ /dev/null
@@ -1,1076 +0,0 @@
-<!-- doc/src/sgml/spgist.sgml -->
-
-<chapter id="spgist">
-<title>SP-GiST Indexes</title>
-
- <indexterm>
- <primary>index</primary>
- <secondary>SP-GiST</secondary>
- </indexterm>
-
-<sect1 id="spgist-intro">
- <title>Introduction</title>
-
- <para>
- <acronym>SP-GiST</acronym> is an abbreviation for space-partitioned
- <acronym>GiST</acronym>. <acronym>SP-GiST</acronym> supports partitioned
- search trees, which facilitate development of a wide range of different
- non-balanced data structures, such as quad-trees, k-d trees, and radix
- trees (tries). The common feature of these structures is that they
- repeatedly divide the search space into partitions that need not be
- of equal size. Searches that are well matched to the partitioning rule
- can be very fast.
- </para>
-
- <para>
- These popular data structures were originally developed for in-memory
- usage. In main memory, they are usually designed as a set of dynamically
- allocated nodes linked by pointers. This is not suitable for direct
- storing on disk, since these chains of pointers can be rather long which
- would require too many disk accesses. In contrast, disk-based data
- structures should have a high fanout to minimize I/O. The challenge
- addressed by <acronym>SP-GiST</acronym> is to map search tree nodes to
- disk pages in such a way that a search need access only a few disk pages,
- even if it traverses many nodes.
- </para>
-
- <para>
- Like <acronym>GiST</acronym>, <acronym>SP-GiST</acronym> is meant to allow
- the development of custom data types with the appropriate access methods,
- by an expert in the domain of the data type, rather than a database expert.
- </para>
-
- <para>
- Some of the information here is derived from Purdue University's
- SP-GiST Indexing Project
- <ulink url="https://www.cs.purdue.edu/spgist/">web site</ulink>.
- The <acronym>SP-GiST</acronym> implementation in
- <productname>PostgreSQL</productname> is primarily maintained by Teodor
- Sigaev and Oleg Bartunov, and there is more information on their
- <!-- URL will be changed -->
- <ulink url="http://www.sai.msu.su/~megera/wiki/spgist_dev">web site</ulink>.
- </para>
-
-</sect1>
-
-<sect1 id="spgist-builtin-opclasses">
- <title>Built-in Operator Classes</title>
-
- <para>
- The core <productname>PostgreSQL</productname> distribution
- includes the <acronym>SP-GiST</acronym> operator classes shown in
- <xref linkend="spgist-builtin-opclasses-table"/>.
- </para>
-
- <table id="spgist-builtin-opclasses-table">
- <title>Built-in <acronym>SP-GiST</acronym> Operator Classes</title>
- <tgroup cols="3">
- <thead>
- <row>
- <entry>Name</entry>
- <entry>Indexable Operators</entry>
- <entry>Ordering Operators</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry valign="middle" morerows="11"><literal>box_ops</literal></entry>
- <entry><literal><< (box,box)</literal></entry>
- <entry valign="middle" morerows="11"><literal><-> (box,point)</literal></entry>
- </row>
- <row><entry><literal>&< (box,box)</literal></entry></row>
- <row><entry><literal>&> (box,box)</literal></entry></row>
- <row><entry><literal>>> (box,box)</literal></entry></row>
- <row><entry><literal><@ (box,box)</literal></entry></row>
- <row><entry><literal>@> (box,box)</literal></entry></row>
- <row><entry><literal>~= (box,box)</literal></entry></row>
- <row><entry><literal>&& (box,box)</literal></entry></row>
- <row><entry><literal><<| (box,box)</literal></entry></row>
- <row><entry><literal>&<| (box,box)</literal></entry></row>
- <row><entry><literal>|&> (box,box)</literal></entry></row>
- <row><entry><literal>|>> (box,box)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="10"><literal>inet_ops</literal></entry>
- <entry><literal><< (inet,inet)</literal></entry>
- <entry valign="middle" morerows="10"></entry>
- </row>
- <row><entry><literal><<= (inet,inet)</literal></entry></row>
- <row><entry><literal>>> (inet,inet)</literal></entry></row>
- <row><entry><literal>>>= (inet,inet)</literal></entry></row>
- <row><entry><literal>= (inet,inet)</literal></entry></row>
- <row><entry><literal><> (inet,inet)</literal></entry></row>
- <row><entry><literal>< (inet,inet)</literal></entry></row>
- <row><entry><literal><= (inet,inet)</literal></entry></row>
- <row><entry><literal>> (inet,inet)</literal></entry></row>
- <row><entry><literal>>= (inet,inet)</literal></entry></row>
- <row><entry><literal>&& (inet,inet)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="5"><literal>kd_point_ops</literal></entry>
- <entry><literal>|>> (point,point)</literal></entry>
- <entry valign="middle" morerows="5"><literal><-> (point,point)</literal></entry>
- </row>
- <row><entry><literal><< (point,point)</literal></entry></row>
- <row><entry><literal>>> (point,point)</literal></entry></row>
- <row><entry><literal><<| (point,point)</literal></entry></row>
- <row><entry><literal>~= (point,point)</literal></entry></row>
- <row><entry><literal><@ (point,box)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="11"><literal>poly_ops</literal></entry>
- <entry><literal><< (polygon,polygon)</literal></entry>
- <entry valign="middle" morerows="11"><literal><-> (polygon,point)</literal></entry>
- </row>
- <row><entry><literal>&< (polygon,polygon)</literal></entry></row>
- <row><entry><literal>&> (polygon,polygon)</literal></entry></row>
- <row><entry><literal>>> (polygon,polygon)</literal></entry></row>
- <row><entry><literal><@ (polygon,polygon)</literal></entry></row>
- <row><entry><literal>@> (polygon,polygon)</literal></entry></row>
- <row><entry><literal>~= (polygon,polygon)</literal></entry></row>
- <row><entry><literal>&& (polygon,polygon)</literal></entry></row>
- <row><entry><literal><<| (polygon,polygon)</literal></entry></row>
- <row><entry><literal>&<| (polygon,polygon)</literal></entry></row>
- <row><entry><literal>|>> (polygon,polygon)</literal></entry></row>
- <row><entry><literal>|&> (polygon,polygon)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="5"><literal>quad_point_ops</literal></entry>
- <entry><literal>|>> (point,point)</literal></entry>
- <entry valign="middle" morerows="5"><literal><-> (point,point)</literal></entry>
- </row>
- <row><entry><literal><< (point,point)</literal></entry></row>
- <row><entry><literal>>> (point,point)</literal></entry></row>
- <row><entry><literal><<| (point,point)</literal></entry></row>
- <row><entry><literal>~= (point,point)</literal></entry></row>
- <row><entry><literal><@ (point,box)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="9"><literal>range_ops</literal></entry>
- <entry><literal>= (anyrange,anyrange)</literal></entry>
- <entry valign="middle" morerows="9"></entry>
- </row>
- <row><entry><literal>&& (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>@> (anyrange,anyelement)</literal></entry></row>
- <row><entry><literal>@> (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal><@ (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal><< (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>>> (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>&< (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>&> (anyrange,anyrange)</literal></entry></row>
- <row><entry><literal>-|- (anyrange,anyrange)</literal></entry></row>
-
- <row>
- <entry valign="middle" morerows="9"><literal>text_ops</literal></entry>
- <entry><literal>= (text,text)</literal></entry>
- <entry valign="middle" morerows="9"></entry>
- </row>
- <row><entry><literal>< (text,text)</literal></entry></row>
- <row><entry><literal><= (text,text)</literal></entry></row>
- <row><entry><literal>> (text,text)</literal></entry></row>
- <row><entry><literal>>= (text,text)</literal></entry></row>
- <row><entry><literal>~<~ (text,text)</literal></entry></row>
- <row><entry><literal>~<=~ (text,text)</literal></entry></row>
- <row><entry><literal>~>=~ (text,text)</literal></entry></row>
- <row><entry><literal>~>~ (text,text)</literal></entry></row>
- <row><entry><literal>^@ (text,text)</literal></entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- Of the two operator classes for type <type>point</type>,
- <literal>quad_point_ops</literal> is the default. <literal>kd_point_ops</literal>
- supports the same operators but uses a different index data structure that
- may offer better performance in some applications.
- </para>
- <para>
- The <literal>quad_point_ops</literal>, <literal>kd_point_ops</literal> and
- <literal>poly_ops</literal> operator classes support the <literal><-></literal>
- ordering operator, which enables the k-nearest neighbor (<literal>k-NN</literal>)
- search over indexed point or polygon data sets.
- </para>
-
-</sect1>
-
-<sect1 id="spgist-extensibility">
- <title>Extensibility</title>
-
- <para>
- <acronym>SP-GiST</acronym> offers an interface with a high level of
- abstraction, requiring the access method developer to implement only
- methods specific to a given data type. The <acronym>SP-GiST</acronym> core
- is responsible for efficient disk mapping and searching the tree structure.
- It also takes care of concurrency and logging considerations.
- </para>
-
- <para>
- Leaf tuples of an <acronym>SP-GiST</acronym> tree usually contain values
- of the same data type as the indexed column, although it is also possible
- for them to contain lossy representations of the indexed column.
- Leaf tuples stored at the root level will directly represent
- the original indexed data value, but leaf tuples at lower
- levels might contain only a partial value, such as a suffix.
- In that case the operator class support functions must be able to
- reconstruct the original value using information accumulated from the
- inner tuples that are passed through to reach the leaf level.
- </para>
-
- <para>
- When an <acronym>SP-GiST</acronym> index is created with
- <literal>INCLUDE</literal> columns, the values of those columns are also
- stored in leaf tuples. The <literal>INCLUDE</literal> columns are of no
- concern to the <acronym>SP-GiST</acronym> operator class, so they are
- not discussed further here.
- </para>
-
- <para>
- Inner tuples are more complex, since they are branching points in the
- search tree. Each inner tuple contains a set of one or more
- <firstterm>nodes</firstterm>, which represent groups of similar leaf values.
- A node contains a downlink that leads either to another, lower-level inner
- tuple, or to a short list of leaf tuples that all lie on the same index page.
- Each node normally has a <firstterm>label</firstterm> that describes it; for example,
- in a radix tree the node label could be the next character of the string
- value. (Alternatively, an operator class can omit the node labels, if it
- works with a fixed set of nodes for all inner tuples;
- see <xref linkend="spgist-null-labels"/>.)
- Optionally, an inner tuple can have a <firstterm>prefix</firstterm> value
- that describes all its members. In a radix tree this could be the common
- prefix of the represented strings. The prefix value is not necessarily
- really a prefix, but can be any data needed by the operator class;
- for example, in a quad-tree it can store the central point that the four
- quadrants are measured with respect to. A quad-tree inner tuple would
- then also contain four nodes corresponding to the quadrants around this
- central point.
- </para>
-
- <para>
- Some tree algorithms require knowledge of level (or depth) of the current
- tuple, so the <acronym>SP-GiST</acronym> core provides the possibility for
- operator classes to manage level counting while descending the tree.
- There is also support for incrementally reconstructing the represented
- value when that is needed, and for passing down additional data (called
- <firstterm>traverse values</firstterm>) during a tree descent.
- </para>
-
- <note>
- <para>
- The <acronym>SP-GiST</acronym> core code takes care of null entries.
- Although <acronym>SP-GiST</acronym> indexes do store entries for nulls
- in indexed columns, this is hidden from the index operator class code:
- no null index entries or search conditions will ever be passed to the
- operator class methods. (It is assumed that <acronym>SP-GiST</acronym>
- operators are strict and so cannot succeed for null values.) Null values
- are therefore not discussed further here.
- </para>
- </note>
-
- <para>
- There are five user-defined methods that an index operator class for
- <acronym>SP-GiST</acronym> must provide, and two are optional. All five
- mandatory methods follow the convention of accepting two <type>internal</type>
- arguments, the first of which is a pointer to a C struct containing input
- values for the support method, while the second argument is a pointer to a
- C struct where output values must be placed. Four of the mandatory methods just
- return <type>void</type>, since all their results appear in the output struct; but
- <function>leaf_consistent</function> returns a <type>boolean</type> result.
- The methods must not modify any fields of their input structs. In all
- cases, the output struct is initialized to zeroes before calling the
- user-defined method. The optional sixth method <function>compress</function>
- accepts a <type>datum</type> to be indexed as the only argument and returns a value suitable
- for physical storage in a leaf tuple. The optional seventh method
- <function>options</function> accepts an <type>internal</type> pointer to a C struct, where
- opclass-specific parameters should be placed, and returns <type>void</type>.
- </para>
-
- <para>
- The five mandatory user-defined methods are:
- </para>
-
- <variablelist>
- <varlistentry>
- <term><function>config</function></term>
- <listitem>
- <para>
- Returns static information about the index implementation, including
- the data type OIDs of the prefix and node label data types.
- </para>
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-<programlisting>
-CREATE FUNCTION my_config(internal, internal) RETURNS void ...
-</programlisting>
- The first argument is a pointer to a <structname>spgConfigIn</structname>
- C struct, containing input data for the function.
- The second argument is a pointer to a <structname>spgConfigOut</structname>
- C struct, which the function must fill with result data.
-<programlisting>
-typedef struct spgConfigIn
-{
- Oid attType; /* Data type to be indexed */
-} spgConfigIn;
-
-typedef struct spgConfigOut
-{
- Oid prefixType; /* Data type of inner-tuple prefixes */
- Oid labelType; /* Data type of inner-tuple node labels */
- Oid leafType; /* Data type of leaf-tuple values */
- bool canReturnData; /* Opclass can reconstruct original data */
- bool longValuesOK; /* Opclass can cope with values > 1 page */
-} spgConfigOut;
-</programlisting>
-
- <structfield>attType</structfield> is passed in order to support polymorphic
- index operator classes; for ordinary fixed-data-type operator classes, it
- will always have the same value and so can be ignored.
- </para>
-
- <para>
- For operator classes that do not use prefixes,
- <structfield>prefixType</structfield> can be set to <literal>VOIDOID</literal>.
- Likewise, for operator classes that do not use node labels,
- <structfield>labelType</structfield> can be set to <literal>VOIDOID</literal>.
- <structfield>canReturnData</structfield> should be set true if the operator class
- is capable of reconstructing the originally-supplied index value.
- <structfield>longValuesOK</structfield> should be set true only when the
- <structfield>attType</structfield> is of variable length and the operator
- class is capable of segmenting long values by repeated suffixing
- (see <xref linkend="spgist-limits"/>).
- </para>
-
- <para>
- <structfield>leafType</structfield> should match the index storage type
- defined by the operator class's <structfield>opckeytype</structfield>
- catalog entry.
- (Note that <structfield>opckeytype</structfield> can be zero,
- implying the storage type is the same as the operator class's input
- type, which is the most common situation.)
- For reasons of backward compatibility, the <function>config</function>
- method can set <structfield>leafType</structfield> to some other value,
- and that value will be used; but this is deprecated since the index
- contents are then incorrectly identified in the catalogs.
- Also, it's permissible to
- leave <structfield>leafType</structfield> uninitialized (zero);
- that is interpreted as meaning the index storage type derived from
- <structfield>opckeytype</structfield>.
- </para>
-
- <para>
- When <structfield>attType</structfield>
- and <structfield>leafType</structfield> are different, the optional
- method <function>compress</function> must be provided.
- Method <function>compress</function> is responsible
- for transformation of datums to be indexed from <structfield>attType</structfield>
- to <structfield>leafType</structfield>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>choose</function></term>
- <listitem>
- <para>
- Chooses a method for inserting a new value into an inner tuple.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-<programlisting>
-CREATE FUNCTION my_choose(internal, internal) RETURNS void ...
-</programlisting>
- The first argument is a pointer to a <structname>spgChooseIn</structname>
- C struct, containing input data for the function.
- The second argument is a pointer to a <structname>spgChooseOut</structname>
- C struct, which the function must fill with result data.
-<programlisting>
-typedef struct spgChooseIn
-{
- Datum datum; /* original datum to be indexed */
- Datum leafDatum; /* current datum to be stored at leaf */
- int level; /* current level (counting from zero) */
-
- /* Data from current inner tuple */
- bool allTheSame; /* tuple is marked all-the-same? */
- bool hasPrefix; /* tuple has a prefix? */
- Datum prefixDatum; /* if so, the prefix value */
- int nNodes; /* number of nodes in the inner tuple */
- Datum *nodeLabels; /* node label values (NULL if none) */
-} spgChooseIn;
-
-typedef enum spgChooseResultType
-{
- spgMatchNode = 1, /* descend into existing node */
- spgAddNode, /* add a node to the inner tuple */
- spgSplitTuple /* split inner tuple (change its prefix) */
-} spgChooseResultType;
-
-typedef struct spgChooseOut
-{
- spgChooseResultType resultType; /* action code, see above */
- union
- {
- struct /* results for spgMatchNode */
- {
- int nodeN; /* descend to this node (index from 0) */
- int levelAdd; /* increment level by this much */
- Datum restDatum; /* new leaf datum */
- } matchNode;
- struct /* results for spgAddNode */
- {
- Datum nodeLabel; /* new node's label */
- int nodeN; /* where to insert it (index from 0) */
- } addNode;
- struct /* results for spgSplitTuple */
- {
- /* Info to form new upper-level inner tuple with one child tuple */
- bool prefixHasPrefix; /* tuple should have a prefix? */
- Datum prefixPrefixDatum; /* if so, its value */
- int prefixNNodes; /* number of nodes */
- Datum *prefixNodeLabels; /* their labels (or NULL for
- * no labels) */
- int childNodeN; /* which node gets child tuple */
-
- /* Info to form new lower-level inner tuple with all old nodes */
- bool postfixHasPrefix; /* tuple should have a prefix? */
- Datum postfixPrefixDatum; /* if so, its value */
- } splitTuple;
- } result;
-} spgChooseOut;
-</programlisting>
-
- <structfield>datum</structfield> is the original datum of
- <structname>spgConfigIn</structname>.<structfield>attType</structfield>
- type that was to be inserted into the index.
- <structfield>leafDatum</structfield> is a value of
- <structname>spgConfigOut</structname>.<structfield>leafType</structfield>
- type, which is initially a result of method
- <function>compress</function> applied to <structfield>datum</structfield>
- when method <function>compress</function> is provided, or the same value as
- <structfield>datum</structfield> otherwise.
- <structfield>leafDatum</structfield> can change at lower levels of the tree
- if the <function>choose</function> or <function>picksplit</function>
- methods change it. When the insertion search reaches a leaf page,
- the current value of <structfield>leafDatum</structfield> is what will be stored
- in the newly created leaf tuple.
- <structfield>level</structfield> is the current inner tuple's level, starting at
- zero for the root level.
- <structfield>allTheSame</structfield> is true if the current inner tuple is
- marked as containing multiple equivalent nodes
- (see <xref linkend="spgist-all-the-same"/>).
- <structfield>hasPrefix</structfield> is true if the current inner tuple contains
- a prefix; if so,
- <structfield>prefixDatum</structfield> is its value.
- <structfield>nNodes</structfield> is the number of child nodes contained in the
- inner tuple, and
- <structfield>nodeLabels</structfield> is an array of their label values, or
- NULL if there are no labels.
- </para>
-
- <para>
- The <function>choose</function> function can determine either that
- the new value matches one of the existing child nodes, or that a new
- child node must be added, or that the new value is inconsistent with
- the tuple prefix and so the inner tuple must be split to create a
- less restrictive prefix.
- </para>
-
- <para>
- If the new value matches one of the existing child nodes,
- set <structfield>resultType</structfield> to <literal>spgMatchNode</literal>.
- Set <structfield>nodeN</structfield> to the index (from zero) of that node in
- the node array.
- Set <structfield>levelAdd</structfield> to the increment in
- <structfield>level</structfield> caused by descending through that node,
- or leave it as zero if the operator class does not use levels.
- Set <structfield>restDatum</structfield> to equal <structfield>leafDatum</structfield>
- if the operator class does not modify datums from one level to the
- next, or otherwise set it to the modified value to be used as
- <structfield>leafDatum</structfield> at the next level.
- </para>
-
- <para>
- If a new child node must be added,
- set <structfield>resultType</structfield> to <literal>spgAddNode</literal>.
- Set <structfield>nodeLabel</structfield> to the label to be used for the new
- node, and set <structfield>nodeN</structfield> to the index (from zero) at which
- to insert the node in the node array.
- After the node has been added, the <function>choose</function>
- function will be called again with the modified inner tuple;
- that call should result in an <literal>spgMatchNode</literal> result.
- </para>
-
- <para>
- If the new value is inconsistent with the tuple prefix,
- set <structfield>resultType</structfield> to <literal>spgSplitTuple</literal>.
- This action moves all the existing nodes into a new lower-level
- inner tuple, and replaces the existing inner tuple with a tuple
- having a single downlink pointing to the new lower-level inner tuple.
- Set <structfield>prefixHasPrefix</structfield> to indicate whether the new
- upper tuple should have a prefix, and if so set
- <structfield>prefixPrefixDatum</structfield> to the prefix value. This new
- prefix value must be sufficiently less restrictive than the original
- to accept the new value to be indexed.
- Set <structfield>prefixNNodes</structfield> to the number of nodes needed in the
- new tuple, and set <structfield>prefixNodeLabels</structfield> to a palloc'd array
- holding their labels, or to NULL if node labels are not required.
- Note that the total size of the new upper tuple must be no more
- than the total size of the tuple it is replacing; this constrains
- the lengths of the new prefix and new labels.
- Set <structfield>childNodeN</structfield> to the index (from zero) of the node
- that will downlink to the new lower-level inner tuple.
- Set <structfield>postfixHasPrefix</structfield> to indicate whether the new
- lower-level inner tuple should have a prefix, and if so set
- <structfield>postfixPrefixDatum</structfield> to the prefix value. The
- combination of these two prefixes and the downlink node's label
- (if any) must have the same meaning as the original prefix, because
- there is no opportunity to alter the node labels that are moved to
- the new lower-level tuple, nor to change any child index entries.
- After the node has been split, the <function>choose</function>
- function will be called again with the replacement inner tuple.
- That call may return an <literal>spgAddNode</literal> result, if no suitable
- node was created by the <literal>spgSplitTuple</literal> action. Eventually
- <function>choose</function> must return <literal>spgMatchNode</literal> to
- allow the insertion to descend to the next level.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>picksplit</function></term>
- <listitem>
- <para>
- Decides how to create a new inner tuple over a set of leaf tuples.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-<programlisting>
-CREATE FUNCTION my_picksplit(internal, internal) RETURNS void ...
-</programlisting>
- The first argument is a pointer to a <structname>spgPickSplitIn</structname>
- C struct, containing input data for the function.
- The second argument is a pointer to a <structname>spgPickSplitOut</structname>
- C struct, which the function must fill with result data.
-<programlisting>
-typedef struct spgPickSplitIn
-{
- int nTuples; /* number of leaf tuples */
- Datum *datums; /* their datums (array of length nTuples) */
- int level; /* current level (counting from zero) */
-} spgPickSplitIn;
-
-typedef struct spgPickSplitOut
-{
- bool hasPrefix; /* new inner tuple should have a prefix? */
- Datum prefixDatum; /* if so, its value */
-
- int nNodes; /* number of nodes for new inner tuple */
- Datum *nodeLabels; /* their labels (or NULL for no labels) */
-
- int *mapTuplesToNodes; /* node index for each leaf tuple */
- Datum *leafTupleDatums; /* datum to store in each new leaf tuple */
-} spgPickSplitOut;
-</programlisting>
-
- <structfield>nTuples</structfield> is the number of leaf tuples provided.
- <structfield>datums</structfield> is an array of their datum values of
- <structname>spgConfigOut</structname>.<structfield>leafType</structfield>
- type.
- <structfield>level</structfield> is the current level that all the leaf tuples
- share, which will become the level of the new inner tuple.
- </para>
-
- <para>
- Set <structfield>hasPrefix</structfield> to indicate whether the new inner
- tuple should have a prefix, and if so set
- <structfield>prefixDatum</structfield> to the prefix value.
- Set <structfield>nNodes</structfield> to indicate the number of nodes that
- the new inner tuple will contain, and
- set <structfield>nodeLabels</structfield> to an array of their label values,
- or to NULL if node labels are not required.
- Set <structfield>mapTuplesToNodes</structfield> to an array that gives the index
- (from zero) of the node that each leaf tuple should be assigned to.
- Set <structfield>leafTupleDatums</structfield> to an array of the values to
- be stored in the new leaf tuples (these will be the same as the
- input <structfield>datums</structfield> if the operator class does not modify
- datums from one level to the next).
- Note that the <function>picksplit</function> function is
- responsible for palloc'ing the
- <structfield>nodeLabels</structfield>, <structfield>mapTuplesToNodes</structfield> and
- <structfield>leafTupleDatums</structfield> arrays.
- </para>
-
- <para>
- If more than one leaf tuple is supplied, it is expected that the
- <function>picksplit</function> function will classify them into more than
- one node; otherwise it is not possible to split the leaf tuples
- across multiple pages, which is the ultimate purpose of this
- operation. Therefore, if the <function>picksplit</function> function
- ends up placing all the leaf tuples in the same node, the core
- SP-GiST code will override that decision and generate an inner
- tuple in which the leaf tuples are assigned at random to several
- identically-labeled nodes. Such a tuple is marked
- <literal>allTheSame</literal> to signify that this has happened. The
- <function>choose</function> and <function>inner_consistent</function> functions
- must take suitable care with such inner tuples.
- See <xref linkend="spgist-all-the-same"/> for more information.
- </para>
-
- <para>
- <function>picksplit</function> can be applied to a single leaf tuple only
- in the case that the <function>config</function> function set
- <structfield>longValuesOK</structfield> to true and a larger-than-a-page input
- value has been supplied. In this case the point of the operation is
- to strip off a prefix and produce a new, shorter leaf datum value.
- The call will be repeated until a leaf datum short enough to fit on
- a page has been produced. See <xref linkend="spgist-limits"/> for
- more information.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>inner_consistent</function></term>
- <listitem>
- <para>
- Returns set of nodes (branches) to follow during tree search.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-<programlisting>
-CREATE FUNCTION my_inner_consistent(internal, internal) RETURNS void ...
-</programlisting>
- The first argument is a pointer to a <structname>spgInnerConsistentIn</structname>
- C struct, containing input data for the function.
- The second argument is a pointer to a <structname>spgInnerConsistentOut</structname>
- C struct, which the function must fill with result data.
-
-<programlisting>
-typedef struct spgInnerConsistentIn
-{
- ScanKey scankeys; /* array of operators and comparison values */
- ScanKey orderbys; /* array of ordering operators and comparison
- * values */
- int nkeys; /* length of scankeys array */
- int norderbys; /* length of orderbys array */
-
- Datum reconstructedValue; /* value reconstructed at parent */
- void *traversalValue; /* opclass-specific traverse value */
- MemoryContext traversalMemoryContext; /* put new traverse values here */
- int level; /* current level (counting from zero) */
- bool returnData; /* original data must be returned? */
-
- /* Data from current inner tuple */
- bool allTheSame; /* tuple is marked all-the-same? */
- bool hasPrefix; /* tuple has a prefix? */
- Datum prefixDatum; /* if so, the prefix value */
- int nNodes; /* number of nodes in the inner tuple */
- Datum *nodeLabels; /* node label values (NULL if none) */
-} spgInnerConsistentIn;
-
-typedef struct spgInnerConsistentOut
-{
- int nNodes; /* number of child nodes to be visited */
- int *nodeNumbers; /* their indexes in the node array */
- int *levelAdds; /* increment level by this much for each */
- Datum *reconstructedValues; /* associated reconstructed values */
- void **traversalValues; /* opclass-specific traverse values */
- double **distances; /* associated distances */
-} spgInnerConsistentOut;
-</programlisting>
-
- The array <structfield>scankeys</structfield>, of length <structfield>nkeys</structfield>,
- describes the index search condition(s). These conditions are
- combined with AND — only index entries that satisfy all of
- them are interesting. (Note that <structfield>nkeys</structfield> = 0 implies
- that all index entries satisfy the query.) Usually the consistent
- function only cares about the <structfield>sk_strategy</structfield> and
- <structfield>sk_argument</structfield> fields of each array entry, which
- respectively give the indexable operator and comparison value.
- In particular it is not necessary to check <structfield>sk_flags</structfield> to
- see if the comparison value is NULL, because the SP-GiST core code
- will filter out such conditions.
- The array <structfield>orderbys</structfield>, of length <structfield>norderbys</structfield>,
- describes ordering operators (if any) in the same manner.
- <structfield>reconstructedValue</structfield> is the value reconstructed for the
- parent tuple; it is <literal>(Datum) 0</literal> at the root level or if the
- <function>inner_consistent</function> function did not provide a value at the
- parent level.
- <structfield>traversalValue</structfield> is a pointer to any traverse data
- passed down from the previous call of <function>inner_consistent</function>
- on the parent index tuple, or NULL at the root level.
- <structfield>traversalMemoryContext</structfield> is the memory context in which
- to store output traverse values (see below).
- <structfield>level</structfield> is the current inner tuple's level, starting at
- zero for the root level.
- <structfield>returnData</structfield> is <literal>true</literal> if reconstructed data is
- required for this query; this will only be so if the
- <function>config</function> function asserted <structfield>canReturnData</structfield>.
- <structfield>allTheSame</structfield> is true if the current inner tuple is
- marked <quote>all-the-same</quote>; in this case all the nodes have the
- same label (if any) and so either all or none of them match the query
- (see <xref linkend="spgist-all-the-same"/>).
- <structfield>hasPrefix</structfield> is true if the current inner tuple contains
- a prefix; if so,
- <structfield>prefixDatum</structfield> is its value.
- <structfield>nNodes</structfield> is the number of child nodes contained in the
- inner tuple, and
- <structfield>nodeLabels</structfield> is an array of their label values, or
- NULL if the nodes do not have labels.
- </para>
-
- <para>
- <structfield>nNodes</structfield> must be set to the number of child nodes that
- need to be visited by the search, and
- <structfield>nodeNumbers</structfield> must be set to an array of their indexes.
- If the operator class keeps track of levels, set
- <structfield>levelAdds</structfield> to an array of the level increments
- required when descending to each node to be visited. (Often these
- increments will be the same for all the nodes, but that's not
- necessarily so, so an array is used.)
- If value reconstruction is needed, set
- <structfield>reconstructedValues</structfield> to an array of the values
- reconstructed for each child node to be visited; otherwise, leave
- <structfield>reconstructedValues</structfield> as NULL.
- The reconstructed values are assumed to be of type
- <structname>spgConfigOut</structname>.<structfield>leafType</structfield>.
- (However, since the core system will do nothing with them except
- possibly copy them, it is sufficient for them to have the
- same <literal>typlen</literal> and <literal>typbyval</literal>
- properties as <structfield>leafType</structfield>.)
- If ordered search is performed, set <structfield>distances</structfield>
- to an array of distance values according to <structfield>orderbys</structfield>
- array (nodes with lowest distances will be processed first). Leave it
- NULL otherwise.
- If it is desired to pass down additional out-of-band information
- (<quote>traverse values</quote>) to lower levels of the tree search,
- set <structfield>traversalValues</structfield> to an array of the appropriate
- traverse values, one for each child node to be visited; otherwise,
- leave <structfield>traversalValues</structfield> as NULL.
- Note that the <function>inner_consistent</function> function is
- responsible for palloc'ing the
- <structfield>nodeNumbers</structfield>, <structfield>levelAdds</structfield>,
- <structfield>distances</structfield>,
- <structfield>reconstructedValues</structfield>, and
- <structfield>traversalValues</structfield> arrays in the current memory context.
- However, any output traverse values pointed to by
- the <structfield>traversalValues</structfield> array should be allocated
- in <structfield>traversalMemoryContext</structfield>.
- Each traverse value must be a single palloc'd chunk.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>leaf_consistent</function></term>
- <listitem>
- <para>
- Returns true if a leaf tuple satisfies a query.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-<programlisting>
-CREATE FUNCTION my_leaf_consistent(internal, internal) RETURNS bool ...
-</programlisting>
- The first argument is a pointer to a <structname>spgLeafConsistentIn</structname>
- C struct, containing input data for the function.
- The second argument is a pointer to a <structname>spgLeafConsistentOut</structname>
- C struct, which the function must fill with result data.
-<programlisting>
-typedef struct spgLeafConsistentIn
-{
- ScanKey scankeys; /* array of operators and comparison values */
- ScanKey orderbys; /* array of ordering operators and comparison
- * values */
- int nkeys; /* length of scankeys array */
- int norderbys; /* length of orderbys array */
-
- Datum reconstructedValue; /* value reconstructed at parent */
- void *traversalValue; /* opclass-specific traverse value */
- int level; /* current level (counting from zero) */
- bool returnData; /* original data must be returned? */
-
- Datum leafDatum; /* datum in leaf tuple */
-} spgLeafConsistentIn;
-
-typedef struct spgLeafConsistentOut
-{
- Datum leafValue; /* reconstructed original data, if any */
- bool recheck; /* set true if operator must be rechecked */
- bool recheckDistances; /* set true if distances must be rechecked */
- double *distances; /* associated distances */
-} spgLeafConsistentOut;
-</programlisting>
-
- The array <structfield>scankeys</structfield>, of length <structfield>nkeys</structfield>,
- describes the index search condition(s). These conditions are
- combined with AND — only index entries that satisfy all of
- them satisfy the query. (Note that <structfield>nkeys</structfield> = 0 implies
- that all index entries satisfy the query.) Usually the consistent
- function only cares about the <structfield>sk_strategy</structfield> and
- <structfield>sk_argument</structfield> fields of each array entry, which
- respectively give the indexable operator and comparison value.
- In particular it is not necessary to check <structfield>sk_flags</structfield> to
- see if the comparison value is NULL, because the SP-GiST core code
- will filter out such conditions.
- The array <structfield>orderbys</structfield>, of length <structfield>norderbys</structfield>,
- describes the ordering operators in the same manner.
- <structfield>reconstructedValue</structfield> is the value reconstructed for the
- parent tuple; it is <literal>(Datum) 0</literal> at the root level or if the
- <function>inner_consistent</function> function did not provide a value at the
- parent level.
- <structfield>traversalValue</structfield> is a pointer to any traverse data
- passed down from the previous call of <function>inner_consistent</function>
- on the parent index tuple, or NULL at the root level.
- <structfield>level</structfield> is the current leaf tuple's level, starting at
- zero for the root level.
- <structfield>returnData</structfield> is <literal>true</literal> if reconstructed data is
- required for this query; this will only be so if the
- <function>config</function> function asserted <structfield>canReturnData</structfield>.
- <structfield>leafDatum</structfield> is the key value of
- <structname>spgConfigOut</structname>.<structfield>leafType</structfield>
- stored in the current leaf tuple.
- </para>
-
- <para>
- The function must return <literal>true</literal> if the leaf tuple matches the
- query, or <literal>false</literal> if not. In the <literal>true</literal> case,
- if <structfield>returnData</structfield> is <literal>true</literal> then
- <structfield>leafValue</structfield> must be set to the value (of type
- <structname>spgConfigIn</structname>.<structfield>attType</structfield>)
- originally supplied to be indexed for this leaf tuple. Also,
- <structfield>recheck</structfield> may be set to <literal>true</literal> if the match
- is uncertain and so the operator(s) must be re-applied to the actual
- heap tuple to verify the match.
- If ordered search is performed, set <structfield>distances</structfield>
- to an array of distance values according to <structfield>orderbys</structfield>
- array. Leave it NULL otherwise. If at least one of returned distances
- is not exact, set <structfield>recheckDistances</structfield> to true.
- In this case, the executor will calculate the exact distances after
- fetching the tuple from the heap, and will reorder the tuples if needed.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <para>
- The optional user-defined methods are:
- </para>
-
- <variablelist>
- <varlistentry>
- <term><function>Datum compress(Datum in)</function></term>
- <listitem>
- <para>
- Converts a data item into a format suitable for physical storage in
- a leaf tuple of the index. It accepts a value of type
- <structname>spgConfigIn</structname>.<structfield>attType</structfield>
- and returns a value of type
- <structname>spgConfigOut</structname>.<structfield>leafType</structfield>.
- The output value must not contain an out-of-line TOAST pointer.
- </para>
-
- <para>
- Note: the <function>compress</function> method is only applied to
- values to be stored. The consistent methods receive query
- <structfield>scankeys</structfield> unchanged, without transformation
- using <function>compress</function>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><function>options</function></term>
- <listitem>
- <para>
- Defines a set of user-visible parameters that control operator class
- behavior.
- </para>
-
- <para>
- The <acronym>SQL</acronym> declaration of the function must look like this:
-
-<programlisting>
-CREATE OR REPLACE FUNCTION my_options(internal)
-RETURNS void
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-</programlisting>
- </para>
-
- <para>
- The function is passed a pointer to a <structname>local_relopts</structname>
- struct, which needs to be filled with a set of operator class
- specific options. The options can be accessed from other support
- functions using the <literal>PG_HAS_OPCLASS_OPTIONS()</literal> and
- <literal>PG_GET_OPCLASS_OPTIONS()</literal> macros.
- </para>
-
- <para>
- Since the representation of the key in <acronym>SP-GiST</acronym> is
- flexible, it may depend on user-specified parameters.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- <para>
- All the SP-GiST support methods are normally called in a short-lived
- memory context; that is, <varname>CurrentMemoryContext</varname> will be reset
- after processing of each tuple. It is therefore not very important to
- worry about pfree'ing everything you palloc. (The <function>config</function>
- method is an exception: it should try to avoid leaking memory. But
- usually the <function>config</function> method need do nothing but assign
- constants into the passed parameter struct.)
- </para>
-
- <para>
- If the indexed column is of a collatable data type, the index collation
- will be passed to all the support methods, using the standard
- <function>PG_GET_COLLATION()</function> mechanism.
- </para>
-
-</sect1>
-
-<sect1 id="spgist-implementation">
- <title>Implementation</title>
-
- <para>
- This section covers implementation details and other tricks that are
- useful for implementers of <acronym>SP-GiST</acronym> operator classes to
- know.
- </para>
-
- <sect2 id="spgist-limits">
- <title>SP-GiST Limits</title>
-
- <para>
- Individual leaf tuples and inner tuples must fit on a single index page
- (8kB by default). Therefore, when indexing values of variable-length
- data types, long values can only be supported by methods such as radix
- trees, in which each level of the tree includes a prefix that is short
- enough to fit on a page, and the final leaf level includes a suffix also
- short enough to fit on a page. The operator class should set
- <structfield>longValuesOK</structfield> to true only if it is prepared to arrange for
- this to happen. Otherwise, the <acronym>SP-GiST</acronym> core will
- reject any request to index a value that is too large to fit
- on an index page.
- </para>
-
- <para>
- Likewise, it is the operator class's responsibility that inner tuples
- do not grow too large to fit on an index page; this limits the number
- of child nodes that can be used in one inner tuple, as well as the
- maximum size of a prefix value.
- </para>
-
- <para>
- Another limitation is that when an inner tuple's node points to a set
- of leaf tuples, those tuples must all be in the same index page.
- (This is a design decision to reduce seeking and save space in the
- links that chain such tuples together.) If the set of leaf tuples
- grows too large for a page, a split is performed and an intermediate
- inner tuple is inserted. For this to fix the problem, the new inner
- tuple <emphasis>must</emphasis> divide the set of leaf values into more than one
- node group. If the operator class's <function>picksplit</function> function
- fails to do that, the <acronym>SP-GiST</acronym> core resorts to
- extraordinary measures described in <xref linkend="spgist-all-the-same"/>.
- </para>
-
- <para>
- When <structfield>longValuesOK</structfield> is true, it is expected
- that successive levels of the <acronym>SP-GiST</acronym> tree will
- absorb more and more information into the prefixes and node labels of
- the inner tuples, making the required leaf datum smaller and smaller,
- so that eventually it will fit on a page.
- To prevent bugs in operator classes from causing infinite insertion
- loops, the <acronym>SP-GiST</acronym> core will raise an error if the
- leaf datum does not become any smaller within ten cycles
- of <function>choose</function> method calls.
- </para>
- </sect2>
-
- <sect2 id="spgist-null-labels">
- <title>SP-GiST Without Node Labels</title>
-
- <para>
- Some tree algorithms use a fixed set of nodes for each inner tuple;
- for example, in a quad-tree there are always exactly four nodes
- corresponding to the four quadrants around the inner tuple's centroid
- point. In such a case the code typically works with the nodes by
- number, and there is no need for explicit node labels. To suppress
- node labels (and thereby save some space), the <function>picksplit</function>
- function can return NULL for the <structfield>nodeLabels</structfield> array,
- and likewise the <function>choose</function> function can return NULL for
- the <structfield>prefixNodeLabels</structfield> array during
- a <literal>spgSplitTuple</literal> action.
- This will in turn result in <structfield>nodeLabels</structfield> being NULL during
- subsequent calls to <function>choose</function> and <function>inner_consistent</function>.
- In principle, node labels could be used for some inner tuples and omitted
- for others in the same index.
- </para>
-
- <para>
- When working with an inner tuple having unlabeled nodes, it is an error
- for <function>choose</function> to return <literal>spgAddNode</literal>, since the set
- of nodes is supposed to be fixed in such cases.
- </para>
- </sect2>
-
- <sect2 id="spgist-all-the-same">
- <title><quote>All-the-Same</quote> Inner Tuples</title>
-
- <para>
- The <acronym>SP-GiST</acronym> core can override the results of the
- operator class's <function>picksplit</function> function when
- <function>picksplit</function> fails to divide the supplied leaf values into
- at least two node categories. When this happens, the new inner tuple
- is created with multiple nodes that each have the same label (if any)
- that <function>picksplit</function> gave to the one node it did use, and the
- leaf values are divided at random among these equivalent nodes.
- The <literal>allTheSame</literal> flag is set on the inner tuple to warn the
- <function>choose</function> and <function>inner_consistent</function> functions that the
- tuple does not have the node set that they might otherwise expect.
- </para>
-
- <para>
- When dealing with an <literal>allTheSame</literal> tuple, a <function>choose</function>
- result of <literal>spgMatchNode</literal> is interpreted to mean that the new
- value can be assigned to any of the equivalent nodes; the core code will
- ignore the supplied <structfield>nodeN</structfield> value and descend into one
- of the nodes at random (so as to keep the tree balanced). It is an
- error for <function>choose</function> to return <literal>spgAddNode</literal>, since
- that would make the nodes not all equivalent; the
- <literal>spgSplitTuple</literal> action must be used if the value to be inserted
- doesn't match the existing nodes.
- </para>
-
- <para>
- When dealing with an <literal>allTheSame</literal> tuple, the
- <function>inner_consistent</function> function should return either all or none
- of the nodes as targets for continuing the index search, since they are
- all equivalent. This may or may not require any special-case code,
- depending on how much the <function>inner_consistent</function> function normally
- assumes about the meaning of the nodes.
- </para>
- </sect2>
-
-</sect1>
-
-<sect1 id="spgist-examples">
- <title>Examples</title>
-
- <para>
- The <productname>PostgreSQL</productname> source distribution includes
- several examples of index operator classes for <acronym>SP-GiST</acronym>,
- as described in <xref linkend="spgist-builtin-opclasses-table"/>. Look
- into <filename>src/backend/access/spgist/</filename>
- and <filename>src/backend/utils/adt/</filename> to see the code.
- </para>
-
-</sect1>
-
-</chapter>
--
2.39.3 (Apple Git-145)
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-20 17:35 Bruce Momjian <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 1 reply; 38+ messages in thread
From: Bruce Momjian @ 2024-03-20 17:35 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On Wed, Mar 20, 2024 at 12:43:08PM -0400, Robert Haas wrote:
> Overall, I think this achieves a minor but pleasant level of
> de-cluttering of the index. It's going to take a lot more than one
> morning's work to produce a major improvement, but at least this is
> something.
I think this kind of doc structure review is long overdue.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Only you can decide what is important to you.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-20 18:16 Robert Haas <[email protected]>
parent: Bruce Momjian <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-20 18:16 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On Wed, Mar 20, 2024 at 1:35 PM Bruce Momjian <[email protected]> wrote:
> On Wed, Mar 20, 2024 at 12:43:08PM -0400, Robert Haas wrote:
> > Overall, I think this achieves a minor but pleasant level of
> > de-cluttering of the index. It's going to take a lot more than one
> > morning's work to produce a major improvement, but at least this is
> > something.
>
> I think this kind of doc structure review is long overdue.
Thanks, Bruce!
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-20 21:05 Alvaro Herrera <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 1 reply; 38+ messages in thread
From: Alvaro Herrera @ 2024-03-20 21:05 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On 2024-Mar-20, Robert Haas wrote:
> 0003 merges all of the "Internals" chapters whose names are the names
> of built-in index access methods (Btree, Gin, etc.) into a single
> chapter called "Built-In Index Access Methods". All of these chapters
> have a very similar structure and none of them are very long, so it
> makes a lot of sense, at least in my mind, to consolidate them into
> one.
I think you can achieve this with a much smaller patch that just changes
the outer tag in each file so that each file is a <sect1>, then create a
single file that includes all of these plus an additional outer tag for
the <chapter> (or maybe just add the <chapter> in postgres.sgml). This
has the advantage that each AM continues to be a separate single file,
and you still have your desired structure.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-20 21:21 Robert Haas <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-20 21:21 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On Wed, Mar 20, 2024 at 5:05 PM Alvaro Herrera <[email protected]> wrote:
> I think you can achieve this with a much smaller patch that just changes
> the outer tag in each file so that each file is a <sect1>, then create a
> single file that includes all of these plus an additional outer tag for
> the <chapter> (or maybe just add the <chapter> in postgres.sgml). This
> has the advantage that each AM continues to be a separate single file,
> and you still have your desired structure.
Right, that could also be done, and not just for 0003. I just wasn't
sure that was the right approach. It would mean that the division of
the SGML into files continues to reflect the original chapter
divisions rather than the current ones forever. In the short run
that's less churn, less back-patching pain, etc.; but in the long term
it means you've got relics of a structure that doesn't exist any more
sticking around forever.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-21 23:33 Peter Eisentraut <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Peter Eisentraut @ 2024-03-21 23:33 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
On 19.03.24 14:50, Tom Lane wrote:
> Daniel Gustafsson <[email protected]> writes:
>> It's actually not very odd, the reference section is using <reference> elements
>> and we had missed the arabic numerals setting on those. The attached fixes
>> that for me. That being said, we've had roman numerals for the reference
>> section since forever (all the way down to the 7.2 docs online has it) so maybe
>> it was intentional?
>
> I'm quite sure it *was* intentional. Maybe it was a bad idea, but
> it's not that way simply because nobody thought about it.
Looks to me it was just that way because it's the default setting of the
stylesheets.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-21 23:37 Peter Eisentraut <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 1 reply; 38+ messages in thread
From: Peter Eisentraut @ 2024-03-21 23:37 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Andrew Dunstan <[email protected]>; +Cc: pgsql-hackers
On 20.03.24 17:43, Robert Haas wrote:
> 0001 removes the "Installation from Binaries" chapter. The whole thing
> is four sentences. I moved the most important information into the
> "Installation from Source Code" chapter and retitled it
> "Installation".
But this separation was explicitly added a few years ago, because most
people just want to read about the binaries.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-22 12:50 Robert Haas <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 38+ messages in thread
From: Robert Haas @ 2024-03-22 12:50 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On Thu, Mar 21, 2024 at 7:37 PM Peter Eisentraut <[email protected]> wrote:
> On 20.03.24 17:43, Robert Haas wrote:
> > 0001 removes the "Installation from Binaries" chapter. The whole thing
> > is four sentences. I moved the most important information into the
> > "Installation from Source Code" chapter and retitled it
> > "Installation".
>
> But this separation was explicitly added a few years ago, because most
> people just want to read about the binaries.
I really doubt that this is true. I've been installing software on
UNIX-like operating systems for more than 30 years now, and I don't
think there's been a single time when I have ever consulted the
documentation for a software package to find the download location for
that package. When I first started out, everything was ftp rather than
www, so you went to ftp.whatever.{com,org,net,gov,edu} and tried to
download the distribution bundle, and then you untarred it and ran
configure and make. Then you read the README or the documentation or
whatever afterward. These days, I think what people do is either (a)
use their package manager to install PostgreSQL and then come to the
documentation afterward to find out how to use it or (b) do a search
for "PostgreSQL download" and click on whatever comes up. I'm not
saying there's never been a user who made use of this section of the
documentation to find the download location, but surely the normal
thing to do if you come to www.postgresql.org and you want to download
the software is to click "Download" on the nav bar, not
"Documentation," then a specific version, then chapter 16, then the
exact same download link that's already there on the nav bar.
I do agree that it is very questionable whether "Installation from
Source Code" is of sufficient interest to ordinary users to justify
including it in "III. Server Administration." Most people, probably
including many extension developers, are only going to install the
binary packages. But the solution to that isn't to have a
four-sentence chapter telling me about a download location that I
likely found long before I looked at the documentation, and that I can
certainly find very easily without needing the documentation. Rather,
what we should do if we think that installing from source code is of
marginal interest is move it to an appendix. As I said to Alvaro
yesterday, I think that a "Developer Guide" appendix could be a good
place to house a number of things that currently have toplevel
chapters but don't really need them because they're only of interest
to a small minority of users. This might be another thing that could
go there.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-22 13:35 Peter Eisentraut <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 38+ messages in thread
From: Peter Eisentraut @ 2024-03-22 13:35 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On 22.03.24 13:50, Robert Haas wrote:
> On Thu, Mar 21, 2024 at 7:37 PM Peter Eisentraut <[email protected]> wrote:
>> On 20.03.24 17:43, Robert Haas wrote:
>>> 0001 removes the "Installation from Binaries" chapter. The whole thing
>>> is four sentences. I moved the most important information into the
>>> "Installation from Source Code" chapter and retitled it
>>> "Installation".
>>
>> But this separation was explicitly added a few years ago, because most
>> people just want to read about the binaries.
>
> I really doubt that this is true.
Here is the thread:
https://www.postgresql.org/message-id/flat/CABUevExRCf8waYOsrCO-QxQL50XGapMf5dnWScOXj7X%3DMXW--g%40m...
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-22 14:10 Robert Haas <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 38+ messages in thread
From: Robert Haas @ 2024-03-22 14:10 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On Fri, Mar 22, 2024 at 9:35 AM Peter Eisentraut <[email protected]> wrote:
> >> But this separation was explicitly added a few years ago, because most
> >> people just want to read about the binaries.
> >
> > I really doubt that this is true.
>
> Here is the thread:
> https://www.postgresql.org/message-id/flat/CABUevExRCf8waYOsrCO-QxQL50XGapMf5dnWScOXj7X%3DMXW--g%40m...
Sorry. I didn't mean to dispute the point that the section was added a
few years ago, nor the point that most people just want to read about
the binaries. I am confident that both of those things are true. What
I do want to dispute is that having a four-sentence chapter in the
documentation index that tells people something they can find much
more easily without using the documentation at all is a good plan. I
agree with the concern that Magnus expressed on the thread, i.e:
> It's kind of strange that if you start your PostgreSQL journey by reading our instructions, you get nothing useful about installing PostgreSQL from binary packages other than "go ask somebody else about it".
But I don't agree that this was the right way to address that problem.
I think it would have been better to just add the download link to the
existing installation chapter. That's actually what we had in chapter
18, "Installation from Source Code on Windows", since removed. But for
some reason we decided that on non-Windows platforms, it needed a
whole new chapter rather than an extra sentence in the existing one. I
think that's massively overkill.
Alternately, I think it would be reasonable to address the concern by
just moving all the stuff about building from source code to an
appendix, and assume people can figure out how to download the
software without us needing to say anything in the documentation at
all. What was weird about the state before that patch, IMHO, was that
we both talked about building from source code and didn't talk about
binary packages. That can be addressed either by adding a mention of
binary packages, or by deemphasizing the idea of installing from
source code.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-25 15:40 Peter Eisentraut <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 2 replies; 38+ messages in thread
From: Peter Eisentraut @ 2024-03-25 15:40 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On 22.03.24 15:10, Robert Haas wrote:
> Sorry. I didn't mean to dispute the point that the section was added a
> few years ago, nor the point that most people just want to read about
> the binaries. I am confident that both of those things are true. What
> I do want to dispute is that having a four-sentence chapter in the
> documentation index that tells people something they can find much
> more easily without using the documentation at all is a good plan.
I think a possible problem we need to consider with these proposals to
combine chapters is that they could make the chapters themselves too
deep and harder to navigate. For example, if we combined the
installation from source and binaries chapters, the structure of the new
chapter would presumably be
<chapter> Installation
<sect1> Installation from Binaries
<sect1> Installation from Source
<sect2> Requirements
<sect2> Getting the Source
<sect2> Building and Installation with Autoconf and Make
<sect2> Building and Installation with Meson
etc.
This would mean that the entire "Installation from Source" part would be
rendered on a single HTML page.
The rendering can be adjusted to some degree, but then we also need to
make sure any new chunking makes sense in other chapters. (And it might
also change a bunch of externally known HTML links.)
I think maybe more could also be done at the top-level structure, too.
Right now, we have <book> -> <part> -> <chapter>. We could add <set> on
top of that.
We could also play with CSS or JavaScript to make the top-level table of
contents more navigable, with collapsing subsections or whatever.
We could also render additional tables of contents or indexes, so there
is more than one way to navigate into the content from the top.
We could also build better search.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-25 16:01 Robert Haas <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 1 reply; 38+ messages in thread
From: Robert Haas @ 2024-03-25 16:01 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On Mon, Mar 25, 2024 at 11:40 AM Peter Eisentraut <[email protected]> wrote:
> I think a possible problem we need to consider with these proposals to
> combine chapters is that they could make the chapters themselves too
> deep and harder to navigate. For example, if we combined the
> installation from source and binaries chapters, the structure of the new
> chapter would presumably be
I agree with this in theory, but in practice I think the patches that
I posted don't have this issue to a degree that is problematic, and I
posted some specific proposals on adjustments that we could make to
ameliorate the problem if other people feel differently.
> I think maybe more could also be done at the top-level structure, too.
> Right now, we have <book> -> <part> -> <chapter>. We could add <set> on
> top of that.
>
> We could also play with CSS or JavaScript to make the top-level table of
> contents more navigable, with collapsing subsections or whatever.
>
> We could also render additional tables of contents or indexes, so there
> is more than one way to navigate into the content from the top.
>
> We could also build better search.
These are all reasonable ideas. I think some better CSS and JavaScript
could definitely help, and I also wondered whether the entrypoint to
the documentation has to be the index page, or whether it could maybe
be a page we've crafted specifically for that purpose, that might
include some text as well as a bunch of links.
But that having been said, I don't believe that any of those ideas (or
anything else we do) will obviate the need for some curation of the
toplevel index. If you're going to add another level, as you propose
in the first point, you still need to make decisions about which
things properly go at which levels. If you're going to allow for
collapsing subsections, you still want the overall tree in which
subsections are be expanded and collapsed to make logical sense. If
you have multiple ways to navigate to the content, one of them will
probably be still the index, and it should be good. And good search is
good, but it shouldn't be the only convenient way to find the content.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-03-29 13:40 Robert Haas <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Robert Haas @ 2024-03-29 13:40 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
OK, so I'm coming back to this thread after giving it a few days to
cool off. My last series of patches proposed to do five things:
1. Merge the four-sentence "Installation from Binaries" chapter back
into "Installation from Source". I thought this was a slam-dunk, but
Peter pointed out that exactly the opposite of this was done a few
years ago to create the "Installation from Binaries" chapter in the
first place. Based on subsequent discussion, what I'm now inclined to
do is come up with a new proposal that involves moving the information
about compiling from source to an appendix. So never mind about this
one for now.
2. Demote "Monitoring Disk Usage" from a chapter on its own to a
section of the "Monitoring Database Activity" chapter. I haven't seen
any objections to this, and I'd like to move ahead with it.
3. Merge the separate chapters on various built-in index AMs into one.
Peter didn't think this was a good idea, but Tom and Alvaro's comments
focused on how to do it mechanically, and on whether the chapters
needed to be reordered afterwards, which I took to mean that they were
OK with the basic concept. David Johnston was also clearly in favor of
it. So I'd like to move ahead with this one, too.
4. Consolidate the "Generic WAL Records" and "Custom WAL Resource
Managers" chapters, which cover related topics, into a single one. I
didn't see anyone object to this, but David Johnston pointed out that
the patch I posted was a few bricks short of a load, because it really
needed to put some introductory text into the new chapter. I'll study
this a bit more and propose a new patch that does the same thing a bit
more carefully than my previous version did.
5. Consolidate all of the procedural language chapters into one. This
was clearly the most controversial part of the proposal. I'm going to
lay this one aside for now and possibly come back to it at a later
time.
I hope that this way of proceeding makes sense to people.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-05 16:01 Robert Haas <[email protected]>
parent: Peter Eisentraut <[email protected]>
1 sibling, 0 replies; 38+ messages in thread
From: Robert Haas @ 2024-04-05 16:01 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; pgsql-hackers
On Mon, Mar 25, 2024 at 11:40 AM Peter Eisentraut <[email protected]> wrote:
> I think a possible problem we need to consider with these proposals to
> combine chapters is that they could make the chapters themselves too
> deep and harder to navigate.
I looked into various options for further combining chapters and/or
appendixes and found that this is indeed a huge problem. For example,
I had thought of creating a Developer Information chapter in the
appendix and moving various existing chapters and appendixes inside of
it, but that means that the <sect1> elements in those chapters get
demoted to <sect2>, and what used to be a whole chapter or appendix
becomes a <sect1>. And since you get one HTML page per <sect1>, that
means that instead of a bunch of individual HTML pages of very
pleasant length, you suddenly get one very long HTML page that is,
exactly as you say, hard to navigate.
> The rendering can be adjusted to some degree, but then we also need to
> make sure any new chunking makes sense in other chapters. (And it might
> also change a bunch of externally known HTML links.)
I looked into this and I'm unclear how much customization is possible.
I gather that the current scheme comes from having chunk.section.depth
of 1, and I guess you can change that to 2 to get an HTML page per
<sect2>, but it seems like it would take a LOT of restructuring to
make that work. It would be much easier if you could vary this across
different parts of the documentation; for instance, if you could say,
well, in this particular chapter or appendix, I want
chunk.section.depth of 2, but elsewhere 1, that would be quite handy,
but after several hours reading various things about DocBook on the
Internet, I was still unable to determine conclusively whether this
was possible. There's an interesting comment in
stylesheet-speedup-xhtml.xsl that says "Since we set a fixed
$chunk.section.depth, we can do away with a bunch of complicated XPath
searches for the previous and next sections at various levels." That
sounds like it's suggesting that it is in fact possible for this
setting to vary, but I don't know if that's true, or how to do it, and
it sounds like there might be performance consequences, too.
> I think maybe more could also be done at the top-level structure, too.
> Right now, we have <book> -> <part> -> <chapter>. We could add <set> on
> top of that.
Does this let you create structures of non-uniform depth? i.e. is
there a way that we can group some chapters into sets while leaving
others as standalone chapters, or somesuch?
I'm not 100% confident that non-uniform depth (either via <set> or via
chunk.section.depth or via some other mechanism) is a good idea.
There's a sort of uniformity to our navigation right now that does
have some real appeal. The downside, though, is that if you want
something to be a single HTML page, it's got to either be a chapter
(or appendix) by itself with no sections inside of it, or it's got to
be a <sect1> inside of a chapter, and so anything that's long enough
that it should be an HTML page by itself can never be more than one
level below the index. And that seems to make it quite difficult to
keep the index small.
Without some kind of variable-depth structure, the only other ways
that I can see to improve things are:
1. Make chunk.section.depth 2 and restructure the entire documentation
until the results look reasonable. This might be possible but I bet
it's difficult. We have, at present, chapters of *wildly* varying
length, from a few sentences to many, many pages. That is perhaps a
bad thing; you most likely wouldn't do that in a printed book. But
fixing it is a huge project. We don't necessarily have the same amount
of content about each topic, and there isn't necessarily a way of
grouping related topics together that produces units of relatively
uniform length. I think it's sensible to try to make improvements
where we can, by pushing stuff down that's short and not that
important, but finding our way to a chunk.section.depth=2 world that
feels good to most people compared to what we have today seems like
it's going to be challening.
2. Replace the current index with a custom index or landing page of
some kind. Or keep the current index and add a new landing page
alongside it. Something that isn't derived automatically from the
documentation structure but is created by hand.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-15 05:00 jian he <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 0 replies; 38+ messages in thread
From: jian he @ 2024-04-15 05:00 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Robert Haas <[email protected]>; pgsql-hackers
On Wed, Mar 20, 2024 at 5:40 AM Andrew Dunstan <[email protected]> wrote:
>
>
> +many for improving the index.
>
> My own pet docs peeve is a purely editorial one: func.sgml is a 30k line beast, and I think there's a good case for splitting out at least the larger chunks of it.
>
I think I successfully reduced func.sgml from 311322 lines to 13167 lines.
(base-commit: 93582974315174d544592185d797a2b44696d1e5)
writing a patch would be unreviewable.
key gotcha is put the contents between opening `<sect1>` and closing
`</sect1>` (both inclusive)
into a new file.
in func.sgml, using `&entity` to refernce the new file.
also update filelist.sgml
here is how I do it:
I found out these build html files are the biggest one:
doc/src/sgml/html/functions-string.html
doc/src/sgml/html/functions-matching.html
doc/src/sgml/html/functions-datetime.html
doc/src/sgml/html/functions-json.html
doc/src/sgml/html/functions-aggregate.html
doc/src/sgml/html/functions-info.html
doc/src/sgml/html/functions-admin.html
so create these new sgml files hold corrspedoning content:
func-string.sgml
func-matching.sgml
func-datetime.sgml
func-json.sgml
func-aggregate.sgml
func-info.sgml
func-admin.sgml
based on funs.sgml structure pattern:
<sect1 id="functions-string">
next section1 line number:
<sect1 id="functions-binarystring">
<sect1 id="functions-matching">
next section1 line number:
<sect1 id="functions-formatting">
<sect1 id="functions-datetime">
next section1 line number:
<sect1 id="functions-enum">
<sect1 id="functions-json">
next section1 line number:
<sect1 id="functions-sequence">
<sect1 id="functions-aggregate">
next section1 line number:
<sect1 id="functions-window">
<sect1 id="functions-info">
next section1 line number:
<sect1 id="functions-admin">
<sect1 id="functions-admin">
next section1 line number:
<sect1 id="functions-trigger">
------------------------------------
step1: pipe the relative line range contents to new sgml files.
(example: line 2407 to line 4177 include all the content correspond to
functions-string.html)
sed -n '2407,4177 p' func.sgml > func-string.sgml
sed -n '5328,7756 p' func.sgml > func-matching.sgml
sed -n '8939,11122 p' func.sgml > func-datetime.sgml
sed -n '15498,19348 p' func.sgml > func-json.sgml
sed -n '21479,22896 p' func.sgml > func-aggregate.sgml
sed -n '24257,27896 p' func.sgml > func-info.sgml
sed -n '27898,30579 p' func.sgml > func-admin.sgml
step2:
in place delete these line ranges in func.sgml
sed --in-place "2407,4177d ; 5328,7756d ; 8939,11122d ; 15498,19348d
; 21479,22896d ; 24257,27896d ; 27898,30579d" \
func.sgml
reference: https://unix.stackexchange.com/questions/676210/matching-multiple-ranges-with-sed-range-expressions
https://www.gnu.org/software/sed/manual/sed.html#Command_002dLine-Options
step3:
put following lines into relative position in func.sgml:
(based on above structure pattern, quickly location line position)
`
&func-string
&func-matching
&func-datetime
&func-json
&func-aggregate
&func-info
&func-admin
`
step4: update filelist.sgml:
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 3fb0709f..0b78a361 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -18,6 +18,13 @@
<!ENTITY ddl SYSTEM "ddl.sgml">
<!ENTITY dml SYSTEM "dml.sgml">
<!ENTITY func SYSTEM "func.sgml">
+<!ENTITY func-string SYSTEM "func-string.sgml">
+<!ENTITY func-matching SYSTEM "func-matching.sgml">
+<!ENTITY func-datetime SYSTEM "func-datetime.sgml">
+<!ENTITY func-json SYSTEM "func-json.sgml">
+<!ENTITY func-aggregate SYSTEM "func-aggregate.sgml">
+<!ENTITY func-info SYSTEM "func-info.sgml">
+<!ENTITY func-admin SYSTEM "func-admin.sgml">
<!ENTITY indices SYSTEM "indices.sgml">
<!ENTITY json SYSTEM "json.sgml">
<!ENTITY mvcc SYSTEM "mvcc.sgml">
doc/src/sgml/filelist.sgml | 7 +
doc/src/sgml/func-admin.sgml | 2682 +++++
doc/src/sgml/func-aggregate.sgml | 1418 +++
doc/src/sgml/func-datetime.sgml | 2184 ++++
doc/src/sgml/func-info.sgml | 3640 ++++++
doc/src/sgml/func-json.sgml | 3851 ++++++
doc/src/sgml/func-matching.sgml | 2429 ++++
doc/src/sgml/func-string.sgml | 1771 +++
doc/src/sgml/func.sgml | 17979 +----------------------------
we can do it one by one, but it's still worth it.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-16 18:23 Andres Freund <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 1 reply; 38+ messages in thread
From: Andres Freund @ 2024-04-16 18:23 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Robert Haas <[email protected]>; pgsql-hackers
Hi,
On 2024-03-19 17:39:39 -0400, Andrew Dunstan wrote:
> My own pet docs peeve is a purely editorial one: func.sgml is a 30k line
> beast, and I think there's a good case for splitting out at least the
> larger chunks of it.
I think we should work on generating a lot of func.sgml. Particularly the
signature etc should just come from pg_proc.dat, it's pointlessly painful to
generate that by hand. And for a lot of the functions we should probably move
the existing func.sgml comments to the description in pg_proc.dat.
I suspect that we can't just generate all the documentation from pg_proc,
because of xrefs etc. Although perhaps we could just strip those out for
pg_proc.
We'd need to add some more metadata to pg_proc, for grouping kinds of
functions together. But that seems doable.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-16 19:05 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 2 replies; 38+ messages in thread
From: Tom Lane @ 2024-04-16 19:05 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
Andres Freund <[email protected]> writes:
> I think we should work on generating a lot of func.sgml. Particularly the
> signature etc should just come from pg_proc.dat, it's pointlessly painful to
> generate that by hand. And for a lot of the functions we should probably move
> the existing func.sgml comments to the description in pg_proc.dat.
Where are you going to get the examples and text descriptions from?
(And no, I don't agree that the pg_description string should match
what's in the docs. The description string has to be a short
one-liner in just about every case.)
This sounds to me like it would be a painful exercise with not a
lot of benefit in the end.
I do agree with Andrew that splitting func.sgml into multiple files
would be beneficial.
regards, tom lane
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-16 19:29 Bruce Momjian <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 38+ messages in thread
From: Bruce Momjian @ 2024-04-16 19:29 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
On Tue, Apr 16, 2024 at 03:05:32PM -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > I think we should work on generating a lot of func.sgml. Particularly the
> > signature etc should just come from pg_proc.dat, it's pointlessly painful to
> > generate that by hand. And for a lot of the functions we should probably move
> > the existing func.sgml comments to the description in pg_proc.dat.
>
> Where are you going to get the examples and text descriptions from?
> (And no, I don't agree that the pg_description string should match
> what's in the docs. The description string has to be a short
> one-liner in just about every case.)
>
> This sounds to me like it would be a painful exercise with not a
> lot of benefit in the end.
Maybe we could _verify_ the contents of func.sgml against pg_proc.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Only you can decide what is important to you.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-16 20:17 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 38+ messages in thread
From: Andres Freund @ 2024-04-16 20:17 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
Hi,
On 2024-04-16 15:05:32 -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > I think we should work on generating a lot of func.sgml. Particularly the
> > signature etc should just come from pg_proc.dat, it's pointlessly painful to
> > generate that by hand. And for a lot of the functions we should probably move
> > the existing func.sgml comments to the description in pg_proc.dat.
>
> Where are you going to get the examples and text descriptions from?
I think there's a few different way to do that. E.g. having long_desc, example
fields in pg_proc.dat. Or having examples and description in a separate file
and "enriching" that with auto-generated function signatures.
> (And no, I don't agree that the pg_description string should match
> what's in the docs. The description string has to be a short
> one-liner in just about every case.)
Definitely shouldn't be the same in all cases, but I think there's a decent
number of cases where they can be the same. The differences between the two is
often minimal today.
Entirely randomly chosen example:
{ oid => '2825',
descr => 'slope of the least-squares-fit linear equation determined by the (X, Y) pairs',
proname => 'regr_slope', prokind => 'a', proisstrict => 'f',
prorettype => 'float8', proargtypes => 'float8 float8',
prosrc => 'aggregate_dummy' },
and
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
<primary>regression slope</primary>
</indexterm>
<indexterm>
<primary>regr_slope</primary>
</indexterm>
<function>regr_slope</function> ( <parameter>Y</parameter> <type>double precision</type>, <parameter>X</parameter> <type>double precision</type> )
<returnvalue>double precision</returnvalue>
</para>
<para>
Computes the slope of the least-squares-fit linear equation determined
by the (<parameter>X</parameter>, <parameter>Y</parameter>)
pairs.
</para></entry>
<entry>Yes</entry>
</row>
The description is quite similar, the pg_proc entry lacks argument names.
> This sounds to me like it would be a painful exercise with not a
> lot of benefit in the end.
I think the manual work for writing signatures in sgml is not insignificant,
nor is the volume of sgml for them. Manually maintaining the signatures makes
it impractical to significantly improve the presentation - which I don't think
is all that great today.
And the lack of argument names in the pg_proc entries is occasionally fairly
annoying, because a \df+ doesn't provide enough information to use functions.
It'd also be quite useful if clients could render more of the documentation
for functions. People are used to language servers providing full
documentation for functions etc...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-17 06:46 Corey Huinker <[email protected]>
parent: Bruce Momjian <[email protected]>
0 siblings, 1 reply; 38+ messages in thread
From: Corey Huinker @ 2024-04-17 06:46 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
>
> > This sounds to me like it would be a painful exercise with not a
> > lot of benefit in the end.
>
> Maybe we could _verify_ the contents of func.sgml against pg_proc.
>
All of the functions redefined in catalog/system_functions.sql complicate
using pg_proc.dat as a doc generator or source of validation. We'd probably
do better to validate against a live instance, and even then the benefit
wouldn't be great.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-17 11:07 Dagfinn Ilmari Mannsåker <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 2 replies; 38+ messages in thread
From: Dagfinn Ilmari Mannsåker @ 2024-04-17 11:07 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
Andres Freund <[email protected]> writes:
> Definitely shouldn't be the same in all cases, but I think there's a decent
> number of cases where they can be the same. The differences between the two is
> often minimal today.
>
> Entirely randomly chosen example:
>
> { oid => '2825',
> descr => 'slope of the least-squares-fit linear equation determined by the (X, Y) pairs',
> proname => 'regr_slope', prokind => 'a', proisstrict => 'f',
> prorettype => 'float8', proargtypes => 'float8 float8',
> prosrc => 'aggregate_dummy' },
>
> and
>
> <row>
> <entry role="func_table_entry"><para role="func_signature">
> <indexterm>
> <primary>regression slope</primary>
> </indexterm>
> <indexterm>
> <primary>regr_slope</primary>
> </indexterm>
> <function>regr_slope</function> ( <parameter>Y</parameter> <type>double precision</type>, <parameter>X</parameter> <type>double precision</type> )
> <returnvalue>double precision</returnvalue>
> </para>
> <para>
> Computes the slope of the least-squares-fit linear equation determined
> by the (<parameter>X</parameter>, <parameter>Y</parameter>)
> pairs.
> </para></entry>
> <entry>Yes</entry>
> </row>
>
>
> The description is quite similar, the pg_proc entry lacks argument names.
>
>
>> This sounds to me like it would be a painful exercise with not a
>> lot of benefit in the end.
>
> I think the manual work for writing signatures in sgml is not insignificant,
> nor is the volume of sgml for them. Manually maintaining the signatures makes
> it impractical to significantly improve the presentation - which I don't think
> is all that great today.
And it's very inconsistent. For example, some functions use <optional>
tags for optional parameters, others use square brackets, and some use
<literal>VARIADIC</literal> to indicate variadic parameters, others use
ellipses (sometimes in <optional> tags or brackets).
> And the lack of argument names in the pg_proc entries is occasionally fairly
> annoying, because a \df+ doesn't provide enough information to use functions.
I was also annoyed by this the other day (specifically wrt. the boolean
arguments to pg_ls_dir), and started whipping up a Perl script to parse
func.sgml and generate missing proargnames values for pg_proc.dat, which
is how I discovered the above. The script currently has a pile of hacky
regexes to cope with that, so I'd be happy to submit a doc patch to turn
it into actual markup to get rid of that, if people think that's a
worhtwhile use of time and won't clash with any other plans for the
documentation.
> It'd also be quite useful if clients could render more of the documentation
> for functions. People are used to language servers providing full
> documentation for functions etc...
A more user-friendly version of \df+ (maybe spelled \hf, for symmetry
with \h for commands?) would certainly be nice.
> Greetings,
>
> Andres Freund
- ilmari
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-17 17:11 Corey Huinker <[email protected]>
parent: Dagfinn Ilmari Mannsåker <[email protected]>
1 sibling, 0 replies; 38+ messages in thread
From: Corey Huinker @ 2024-04-17 17:11 UTC (permalink / raw)
To: Dagfinn Ilmari Mannsåker <[email protected]>; +Cc: Andres Freund <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
>
> And it's very inconsistent. For example, some functions use <optional>
> tags for optional parameters, others use square brackets, and some use
> <literal>VARIADIC</literal> to indicate variadic parameters, others use
> ellipses (sometimes in <optional> tags or brackets).
Having just written a couple of those functions, I wasn't able to find any
guidance on how to document them with regards to <optional> vs [], etc.
Having such a thing would be helpful.
While we're throwing out ideas, does it make sense to have function
parameters and return values be things that can accept COMMENTs? Like so:
COMMENT ON FUNCTION function_name [ ( [ [ argmode ] [ argname ] argtype [,
...] ] ) ] ARGUMENT argname IS '....';
COMMENT ON FUNCTION function_name [ ( [ [ argmode ] [ argname ] argtype [,
...] ] ) ] RETURN VALUE IS '....';
I don't think this is a great idea, but if we're going to auto-generate
documentation then we've got to store the metadata somewhere, and
pg_proc.dat is already lacking relevant details.
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-17 17:21 Andres Freund <[email protected]>
parent: Corey Huinker <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: Andres Freund @ 2024-04-17 17:21 UTC (permalink / raw)
To: Corey Huinker <[email protected]>; +Cc: Bruce Momjian <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
Hi,
On 2024-04-17 02:46:53 -0400, Corey Huinker wrote:
> > > This sounds to me like it would be a painful exercise with not a
> > > lot of benefit in the end.
> >
> > Maybe we could _verify_ the contents of func.sgml against pg_proc.
> >
>
> All of the functions redefined in catalog/system_functions.sql complicate
> using pg_proc.dat as a doc generator or source of validation. We'd probably
> do better to validate against a live instance, and even then the benefit
> wouldn't be great.
There are 80 'CREATE OR REPLACE's in system_functions.sql, 1016 occurrences of
func_table_entry in funcs.sgml and 3.3k functions in pg_proc. I'm not saying
that differences due to system_functions.sql wouldn't be annoying to deal
with, but it'd also be far from the end of the world.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-17 17:28 Andres Freund <[email protected]>
parent: Dagfinn Ilmari Mannsåker <[email protected]>
1 sibling, 1 reply; 38+ messages in thread
From: Andres Freund @ 2024-04-17 17:28 UTC (permalink / raw)
To: Dagfinn Ilmari Mannsåker <[email protected]>; +Cc: Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
Hi,
On 2024-04-17 12:07:24 +0100, Dagfinn Ilmari Mannsåker wrote:
> Andres Freund <[email protected]> writes:
> > I think the manual work for writing signatures in sgml is not insignificant,
> > nor is the volume of sgml for them. Manually maintaining the signatures makes
> > it impractical to significantly improve the presentation - which I don't think
> > is all that great today.
>
> And it's very inconsistent. For example, some functions use <optional>
> tags for optional parameters, others use square brackets, and some use
> <literal>VARIADIC</literal> to indicate variadic parameters, others use
> ellipses (sometimes in <optional> tags or brackets).
That seems almost inevitably the outcome of many people having to manually
infer the recommended semantics, for writing something boring but nontrivial,
from a 30k line file.
> > And the lack of argument names in the pg_proc entries is occasionally fairly
> > annoying, because a \df+ doesn't provide enough information to use functions.
>
> I was also annoyed by this the other day (specifically wrt. the boolean
> arguments to pg_ls_dir),
My bane is regexp_match et al, I have given up on remembering the argument
order.
> and started whipping up a Perl script to parse func.sgml and generate
> missing proargnames values for pg_proc.dat, which is how I discovered the
> above.
Nice.
> The script currently has a pile of hacky regexes to cope with that,
> so I'd be happy to submit a doc patch to turn it into actual markup to get
> rid of that, if people think that's a worhtwhile use of time and won't clash
> with any other plans for the documentation.
I guess it's a bit hard to say without knowing how voluminious the changes
would be. If we end up rewriting the whole file the tradeoff is less clear
than if it's a dozen inconsistent entries.
> > It'd also be quite useful if clients could render more of the documentation
> > for functions. People are used to language servers providing full
> > documentation for functions etc...
>
> A more user-friendly version of \df+ (maybe spelled \hf, for symmetry
> with \h for commands?) would certainly be nice.
Indeed.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-17 18:37 Dagfinn Ilmari Mannsåker <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 38+ messages in thread
From: Dagfinn Ilmari Mannsåker @ 2024-04-17 18:37 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
Andres Freund <[email protected]> writes:
> Hi,
>
> On 2024-04-17 12:07:24 +0100, Dagfinn Ilmari Mannsåker wrote:
>> Andres Freund <[email protected]> writes:
>> > I think the manual work for writing signatures in sgml is not insignificant,
>> > nor is the volume of sgml for them. Manually maintaining the signatures makes
>> > it impractical to significantly improve the presentation - which I don't think
>> > is all that great today.
>>
>> And it's very inconsistent. For example, some functions use <optional>
>> tags for optional parameters, others use square brackets, and some use
>> <literal>VARIADIC</literal> to indicate variadic parameters, others use
>> ellipses (sometimes in <optional> tags or brackets).
>
> That seems almost inevitably the outcome of many people having to manually
> infer the recommended semantics, for writing something boring but nontrivial,
> from a 30k line file.
As Corey mentioned elsethread, having a markup style guide (maybe a
comment at the top of the file?) would be nice.
>> > And the lack of argument names in the pg_proc entries is occasionally fairly
>> > annoying, because a \df+ doesn't provide enough information to use functions.
>>
>> I was also annoyed by this the other day (specifically wrt. the boolean
>> arguments to pg_ls_dir),
>
> My bane is regexp_match et al, I have given up on remembering the argument
> order.
There's a thread elsewhere about those specifically, but I can't be
bothered to find the link right now.
>> and started whipping up a Perl script to parse func.sgml and generate
>> missing proargnames values for pg_proc.dat, which is how I discovered the
>> above.
>
> Nice.
>
>> The script currently has a pile of hacky regexes to cope with that,
>> so I'd be happy to submit a doc patch to turn it into actual markup to get
>> rid of that, if people think that's a worhtwhile use of time and won't clash
>> with any other plans for the documentation.
>
> I guess it's a bit hard to say without knowing how voluminious the changes
> would be. If we end up rewriting the whole file the tradeoff is less clear
> than if it's a dozen inconsistent entries.
It turned out to not be that many that used [] for optional parameters,
see the attached patch.
I havent dealt with variadic yet, since the two styles are visually
different, not just markup (<optional>...</optional> renders as [...]).
The two styles for variadic are the what I call caller-style:
concat ( val1 "any" [, val2 "any" [, ...] ] )
format(formatstr text [, formatarg "any" [, ...] ])
which shows more clearly how you'd call it, versus definition-style:
num_nonnulls ( VARIADIC "any" )
jsonb_extract_path ( from_json jsonb, VARIADIC path_elems text[] )
which matches the CREATE FUNCTION statement. I don't have a strong
opinion on which we should use, but we should be consistent.
> Greetings,
>
> Andres Freund
- ilmari
Attachments:
[text/x-diff] 0001-func.sgml-Consistently-use-optional-to-indicate-opti.patch (11.4K, ../../[email protected]/2-0001-func.sgml-Consistently-use-optional-to-indicate-opti.patch)
download | inline diff:
From f71e0669eb25b205bd5065f15657ba6d749261f3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <[email protected]>
Date: Wed, 17 Apr 2024 16:00:52 +0100
Subject: [PATCH] func.sgml: Consistently use <optional> to indicate optional
parameters
Some functions were using square brackets instead.
---
doc/src/sgml/func.sgml | 54 +++++++++++++++++++++---------------------
1 file changed, 27 insertions(+), 27 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8dfb42ad4d..afaaf61d69 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -3036,7 +3036,7 @@
<primary>concat</primary>
</indexterm>
<function>concat</function> ( <parameter>val1</parameter> <type>"any"</type>
- [, <parameter>val2</parameter> <type>"any"</type> [, ...] ] )
+ <optional>, <parameter>val2</parameter> <type>"any"</type> [, ...] </optional> )
<returnvalue>text</returnvalue>
</para>
<para>
@@ -3056,7 +3056,7 @@
</indexterm>
<function>concat_ws</function> ( <parameter>sep</parameter> <type>text</type>,
<parameter>val1</parameter> <type>"any"</type>
- [, <parameter>val2</parameter> <type>"any"</type> [, ...] ] )
+ <optional>, <parameter>val2</parameter> <type>"any"</type> [, ...] </optional> )
<returnvalue>text</returnvalue>
</para>
<para>
@@ -3076,7 +3076,7 @@
<primary>format</primary>
</indexterm>
<function>format</function> ( <parameter>formatstr</parameter> <type>text</type>
- [, <parameter>formatarg</parameter> <type>"any"</type> [, ...] ] )
+ <optional>, <parameter>formatarg</parameter> <type>"any"</type> [, ...] </optional> )
<returnvalue>text</returnvalue>
</para>
<para>
@@ -3170,7 +3170,7 @@
<primary>parse_ident</primary>
</indexterm>
<function>parse_ident</function> ( <parameter>qualified_identifier</parameter> <type>text</type>
- [, <parameter>strict_mode</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>true</literal> ] )
+ <optional>, <parameter>strict_mode</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>true</literal> </optional> )
<returnvalue>text[]</returnvalue>
</para>
<para>
@@ -3309,8 +3309,8 @@
<primary>regexp_count</primary>
</indexterm>
<function>regexp_count</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type>
- [, <parameter>start</parameter> <type>integer</type>
- [, <parameter>flags</parameter> <type>text</type> ] ] )
+ <optional>, <parameter>start</parameter> <type>integer</type>
+ <optional>, <parameter>flags</parameter> <type>text</type> </optional> </optional> )
<returnvalue>integer</returnvalue>
</para>
<para>
@@ -3331,11 +3331,11 @@
<primary>regexp_instr</primary>
</indexterm>
<function>regexp_instr</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type>
- [, <parameter>start</parameter> <type>integer</type>
- [, <parameter>N</parameter> <type>integer</type>
- [, <parameter>endoption</parameter> <type>integer</type>
- [, <parameter>flags</parameter> <type>text</type>
- [, <parameter>subexpr</parameter> <type>integer</type> ] ] ] ] ] )
+ <optional>, <parameter>start</parameter> <type>integer</type>
+ <optional>, <parameter>N</parameter> <type>integer</type>
+ <optional>, <parameter>endoption</parameter> <type>integer</type>
+ <optional>, <parameter>flags</parameter> <type>text</type>
+ <optional>, <parameter>subexpr</parameter> <type>integer</type> </optional> </optional> </optional> </optional> </optional> )
<returnvalue>integer</returnvalue>
</para>
<para>
@@ -3360,7 +3360,7 @@
<primary>regexp_like</primary>
</indexterm>
<function>regexp_like</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type>
- [, <parameter>flags</parameter> <type>text</type> ] )
+ <optional>, <parameter>flags</parameter> <type>text</type> </optional> )
<returnvalue>boolean</returnvalue>
</para>
<para>
@@ -3380,7 +3380,7 @@
<indexterm>
<primary>regexp_match</primary>
</indexterm>
- <function>regexp_match</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> [, <parameter>flags</parameter> <type>text</type> ] )
+ <function>regexp_match</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> <optional>, <parameter>flags</parameter> <type>text</type> </optional> )
<returnvalue>text[]</returnvalue>
</para>
<para>
@@ -3400,7 +3400,7 @@
<indexterm>
<primary>regexp_matches</primary>
</indexterm>
- <function>regexp_matches</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> [, <parameter>flags</parameter> <type>text</type> ] )
+ <function>regexp_matches</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> <optional>, <parameter>flags</parameter> <type>text</type> </optional> )
<returnvalue>setof text[]</returnvalue>
</para>
<para>
@@ -3426,8 +3426,8 @@
<primary>regexp_replace</primary>
</indexterm>
<function>regexp_replace</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type>, <parameter>replacement</parameter> <type>text</type>
- [, <parameter>start</parameter> <type>integer</type> ]
- [, <parameter>flags</parameter> <type>text</type> ] )
+ <optional>, <parameter>start</parameter> <type>integer</type> </optional>
+ <optional>, <parameter>flags</parameter> <type>text</type> </optional> )
<returnvalue>text</returnvalue>
</para>
<para>
@@ -3447,7 +3447,7 @@
<function>regexp_replace</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type>, <parameter>replacement</parameter> <type>text</type>,
<parameter>start</parameter> <type>integer</type>,
<parameter>N</parameter> <type>integer</type>
- [, <parameter>flags</parameter> <type>text</type> ] )
+ <optional>, <parameter>flags</parameter> <type>text</type> </optional> )
<returnvalue>text</returnvalue>
</para>
<para>
@@ -3467,7 +3467,7 @@
<indexterm>
<primary>regexp_split_to_array</primary>
</indexterm>
- <function>regexp_split_to_array</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> [, <parameter>flags</parameter> <type>text</type> ] )
+ <function>regexp_split_to_array</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> <optional>, <parameter>flags</parameter> <type>text</type> </optional> )
<returnvalue>text[]</returnvalue>
</para>
<para>
@@ -3486,7 +3486,7 @@
<indexterm>
<primary>regexp_split_to_table</primary>
</indexterm>
- <function>regexp_split_to_table</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> [, <parameter>flags</parameter> <type>text</type> ] )
+ <function>regexp_split_to_table</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type> <optional>, <parameter>flags</parameter> <type>text</type> </optional> )
<returnvalue>setof text</returnvalue>
</para>
<para>
@@ -3510,10 +3510,10 @@
<primary>regexp_substr</primary>
</indexterm>
<function>regexp_substr</function> ( <parameter>string</parameter> <type>text</type>, <parameter>pattern</parameter> <type>text</type>
- [, <parameter>start</parameter> <type>integer</type>
- [, <parameter>N</parameter> <type>integer</type>
- [, <parameter>flags</parameter> <type>text</type>
- [, <parameter>subexpr</parameter> <type>integer</type> ] ] ] ] )
+ <optional>, <parameter>start</parameter> <type>integer</type>
+ <optional>, <parameter>N</parameter> <type>integer</type>
+ <optional>, <parameter>flags</parameter> <type>text</type>
+ <optional>, <parameter>subexpr</parameter> <type>integer</type> </optional> </optional> </optional> </optional> )
<returnvalue>text</returnvalue>
</para>
<para>
@@ -3980,7 +3980,7 @@
<para>
<synopsis>
-<function>format</function>(<parameter>formatstr</parameter> <type>text</type> [, <parameter>formatarg</parameter> <type>"any"</type> [, ...] ])
+<function>format</function>(<parameter>formatstr</parameter> <type>text</type> <optional>, <parameter>formatarg</parameter> <type>"any"</type> [, ...] </optional>)
</synopsis>
<parameter>formatstr</parameter> is a format string that specifies how the
result should be formatted. Text in the format string is copied
@@ -10568,7 +10568,7 @@
<para>
<synopsis>
-date_trunc(<replaceable>field</replaceable>, <replaceable>source</replaceable> [, <replaceable>time_zone</replaceable> ])
+date_trunc(<replaceable>field</replaceable>, <replaceable>source</replaceable> <optional>, <replaceable>time_zone</replaceable> </optional>)
</synopsis>
<replaceable>source</replaceable> is a value expression of type
<type>timestamp</type>, <type>timestamp with time zone</type>,
@@ -29308,11 +29308,11 @@
<indexterm>
<primary>pg_logical_emit_message</primary>
</indexterm>
- <function>pg_logical_emit_message</function> ( <parameter>transactional</parameter> <type>boolean</type>, <parameter>prefix</parameter> <type>text</type>, <parameter>content</parameter> <type>text</type> [, <parameter>flush</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>false</literal>] )
+ <function>pg_logical_emit_message</function> ( <parameter>transactional</parameter> <type>boolean</type>, <parameter>prefix</parameter> <type>text</type>, <parameter>content</parameter> <type>text</type> <optional>, <parameter>flush</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>false</literal></optional> )
<returnvalue>pg_lsn</returnvalue>
</para>
<para role="func_signature">
- <function>pg_logical_emit_message</function> ( <parameter>transactional</parameter> <type>boolean</type>, <parameter>prefix</parameter> <type>text</type>, <parameter>content</parameter> <type>bytea</type> [, <parameter>flush</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>false</literal>] )
+ <function>pg_logical_emit_message</function> ( <parameter>transactional</parameter> <type>boolean</type>, <parameter>prefix</parameter> <type>text</type>, <parameter>content</parameter> <type>bytea</type> <optional>, <parameter>flush</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>false</literal></optional> )
<returnvalue>pg_lsn</returnvalue>
</para>
<para>
--
2.39.2
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: documentation structure
@ 2024-04-18 13:28 jian he <[email protected]>
parent: Dagfinn Ilmari Mannsåker <[email protected]>
0 siblings, 0 replies; 38+ messages in thread
From: jian he @ 2024-04-18 13:28 UTC (permalink / raw)
To: Dagfinn Ilmari Mannsåker <[email protected]>; +Cc: Andres Freund <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; pgsql-hackers
On Thu, Apr 18, 2024 at 2:37 AM Dagfinn Ilmari Mannsåker
<[email protected]> wrote:
>
> Andres Freund <[email protected]> writes:
>
> > Hi,
> >
> > On 2024-04-17 12:07:24 +0100, Dagfinn Ilmari Mannsåker wrote:
> >> Andres Freund <[email protected]> writes:
> >> > I think the manual work for writing signatures in sgml is not insignificant,
> >> > nor is the volume of sgml for them. Manually maintaining the signatures makes
> >> > it impractical to significantly improve the presentation - which I don't think
> >> > is all that great today.
> >>
> >> And it's very inconsistent. For example, some functions use <optional>
> >> tags for optional parameters, others use square brackets, and some use
> >> <literal>VARIADIC</literal> to indicate variadic parameters, others use
> >> ellipses (sometimes in <optional> tags or brackets).
> >
> > That seems almost inevitably the outcome of many people having to manually
> > infer the recommended semantics, for writing something boring but nontrivial,
> > from a 30k line file.
>
> As Corey mentioned elsethread, having a markup style guide (maybe a
> comment at the top of the file?) would be nice.
>
> >> > And the lack of argument names in the pg_proc entries is occasionally fairly
> >> > annoying, because a \df+ doesn't provide enough information to use functions.
> >>
> >> I was also annoyed by this the other day (specifically wrt. the boolean
> >> arguments to pg_ls_dir),
> >
> > My bane is regexp_match et al, I have given up on remembering the argument
> > order.
>
> There's a thread elsewhere about those specifically, but I can't be
> bothered to find the link right now.
>
> >> and started whipping up a Perl script to parse func.sgml and generate
> >> missing proargnames values for pg_proc.dat, which is how I discovered the
> >> above.
> >
> > Nice.
> >
> >> The script currently has a pile of hacky regexes to cope with that,
> >> so I'd be happy to submit a doc patch to turn it into actual markup to get
> >> rid of that, if people think that's a worhtwhile use of time and won't clash
> >> with any other plans for the documentation.
> >
> > I guess it's a bit hard to say without knowing how voluminious the changes
> > would be. If we end up rewriting the whole file the tradeoff is less clear
> > than if it's a dozen inconsistent entries.
>
> It turned out to not be that many that used [] for optional parameters,
> see the attached patch.
>
hi.
I manually checked the html output. It looks good to me.
^ permalink raw reply [nested|flat] 38+ messages in thread
end of thread, other threads:[~2024-04-18 13:28 UTC | newest]
Thread overview: 38+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-12 13:07 [PATCH 6/6] Ignore correlation for new BRIN opclasses Tomas Vondra <[email protected]>
2024-03-18 14:11 documentation structure Robert Haas <[email protected]>
2024-03-18 14:54 ` Re: documentation structure Matthias van de Meent <[email protected]>
2024-03-18 17:21 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-18 19:06 ` Re: documentation structure Roberto Mello <[email protected]>
2024-03-18 21:40 ` Re: documentation structure Laurenz Albe <[email protected]>
2024-03-18 22:51 ` Re: documentation structure Tom Lane <[email protected]>
2024-03-18 23:34 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-19 09:05 ` Re: documentation structure Daniel Gustafsson <[email protected]>
2024-03-19 13:50 ` Re: documentation structure Tom Lane <[email protected]>
2024-03-21 23:33 ` Re: documentation structure Peter Eisentraut <[email protected]>
2024-03-20 14:32 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-19 21:39 ` Re: documentation structure Andrew Dunstan <[email protected]>
2024-03-20 16:43 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-20 17:35 ` Re: documentation structure Bruce Momjian <[email protected]>
2024-03-20 18:16 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-20 21:05 ` Re: documentation structure Alvaro Herrera <[email protected]>
2024-03-20 21:21 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-21 23:37 ` Re: documentation structure Peter Eisentraut <[email protected]>
2024-03-22 12:50 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-22 13:35 ` Re: documentation structure Peter Eisentraut <[email protected]>
2024-03-22 14:10 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-25 15:40 ` Re: documentation structure Peter Eisentraut <[email protected]>
2024-03-25 16:01 ` Re: documentation structure Robert Haas <[email protected]>
2024-03-29 13:40 ` Re: documentation structure Robert Haas <[email protected]>
2024-04-05 16:01 ` Re: documentation structure Robert Haas <[email protected]>
2024-04-15 05:00 ` Re: documentation structure jian he <[email protected]>
2024-04-16 18:23 ` Re: documentation structure Andres Freund <[email protected]>
2024-04-16 19:05 ` Re: documentation structure Tom Lane <[email protected]>
2024-04-16 19:29 ` Re: documentation structure Bruce Momjian <[email protected]>
2024-04-17 06:46 ` Re: documentation structure Corey Huinker <[email protected]>
2024-04-17 17:21 ` Re: documentation structure Andres Freund <[email protected]>
2024-04-16 20:17 ` Re: documentation structure Andres Freund <[email protected]>
2024-04-17 11:07 ` Re: documentation structure Dagfinn Ilmari Mannsåker <[email protected]>
2024-04-17 17:11 ` Re: documentation structure Corey Huinker <[email protected]>
2024-04-17 17:28 ` Re: documentation structure Andres Freund <[email protected]>
2024-04-17 18:37 ` Re: documentation structure Dagfinn Ilmari Mannsåker <[email protected]>
2024-04-18 13:28 ` Re: documentation structure jian he <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox