public inbox for [email protected]
help / color / mirror / Atom feedFrom: Bruce Momjian <[email protected]>
To: PostgreSQL-documentation <[email protected]>
Subject: Update docs for GIN index improvements
Date: Sat, 16 May 2015 08:37:59 -0400
Message-ID: <[email protected]> (raw)
List-Unsubscribe: <mailto:[email protected]?body=unsub%20pgsql-docs>
We have made major improvements in GIN in the past few releases,
particularly 9.4, but the full text search docs recommending when to use
GIN and GiST indexes have not been updated. This blog post empirically
shows many of the improvements:
http://blog.pgaddict.com/posts/performance-since-postgresql-7-4-to-9-4-fulltext
I would like to apply the attached patch to 9.5 to recommend GIN indexes
for full text search, and remove mention of many old GIN limitations.
--
Bruce Momjian <[email protected]> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-docs mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
Attachments:
[text/x-diff] gin.diff (7.4K, 2-gin.diff)
download | inline diff:
diff --git a/doc/src/sgml/textsearch.sgml b/doc/src/sgml/textsearch.sgml
new file mode 100644
index b1c669f..538e6ad
*** a/doc/src/sgml/textsearch.sgml
--- b/doc/src/sgml/textsearch.sgml
*************** SELECT plainto_tsquery('supernovae stars
*** 3192,3198 ****
</sect1>
<sect1 id="textsearch-indexes">
! <title>GiST and GIN Index Types</title>
<indexterm zone="textsearch-indexes">
<primary>text search</primary>
--- 3192,3198 ----
</sect1>
<sect1 id="textsearch-indexes">
! <title>GIN and GiST Index Types</title>
<indexterm zone="textsearch-indexes">
<primary>text search</primary>
*************** SELECT plainto_tsquery('supernovae stars
*** 3213,3230 ****
<term>
<indexterm zone="textsearch-indexes">
<primary>index</primary>
! <secondary>GiST</secondary>
<tertiary>text search</tertiary>
</indexterm>
! <literal>CREATE INDEX <replaceable>name</replaceable> ON <replaceable>table</replaceable> USING GIST (<replaceable>column</replaceable>);</literal>
</term>
<listitem>
<para>
! Creates a GiST (Generalized Search Tree)-based index.
! The <replaceable>column</replaceable> can be of <type>tsvector</> or
! <type>tsquery</> type.
</para>
</listitem>
</varlistentry>
--- 3213,3229 ----
<term>
<indexterm zone="textsearch-indexes">
<primary>index</primary>
! <secondary>GIN</secondary>
<tertiary>text search</tertiary>
</indexterm>
! <literal>CREATE INDEX <replaceable>name</replaceable> ON <replaceable>table</replaceable> USING GIN (<replaceable>column</replaceable>);</literal>
</term>
<listitem>
<para>
! Creates a GIN (Generalized Inverted Index)-based index.
! The <replaceable>column</replaceable> must be of <type>tsvector</> type.
</para>
</listitem>
</varlistentry>
*************** SELECT plainto_tsquery('supernovae stars
*** 3234,3250 ****
<term>
<indexterm zone="textsearch-indexes">
<primary>index</primary>
! <secondary>GIN</secondary>
<tertiary>text search</tertiary>
</indexterm>
! <literal>CREATE INDEX <replaceable>name</replaceable> ON <replaceable>table</replaceable> USING GIN (<replaceable>column</replaceable>);</literal>
</term>
<listitem>
<para>
! Creates a GIN (Generalized Inverted Index)-based index.
! The <replaceable>column</replaceable> must be of <type>tsvector</> type.
</para>
</listitem>
</varlistentry>
--- 3233,3250 ----
<term>
<indexterm zone="textsearch-indexes">
<primary>index</primary>
! <secondary>GiST</secondary>
<tertiary>text search</tertiary>
</indexterm>
! <literal>CREATE INDEX <replaceable>name</replaceable> ON <replaceable>table</replaceable> USING GIST (<replaceable>column</replaceable>);</literal>
</term>
<listitem>
<para>
! Creates a GiST (Generalized Search Tree)-based index.
! The <replaceable>column</replaceable> can be of <type>tsvector</> or
! <type>tsquery</> type.
</para>
</listitem>
</varlistentry>
*************** SELECT plainto_tsquery('supernovae stars
*** 3253,3265 ****
</para>
<para>
! There are substantial performance differences between the two index types,
! so it is important to understand their characteristics.
</para>
<para>
A GiST index is <firstterm>lossy</firstterm>, meaning that the index
! may produce false matches, and it is necessary
to check the actual table row to eliminate such false matches.
(<productname>PostgreSQL</productname> does this automatically when needed.)
GiST indexes are lossy because each document is represented in the
--- 3253,3270 ----
</para>
<para>
! GIN indexes are the preferred text search index type. As inverted
! indexes, they contain an index entry for each word (lexeme), with a
! compressed list of matching locations. Multi-word searches can find
! the first match, then use the index to remove rows that are lacking
! additional words. GIN indexes store only the words (lexemes) of
! <type>tsvector</> values, and not their weight labels. Thus a table
! row recheck is needed when using a query that involves weights.
</para>
<para>
A GiST index is <firstterm>lossy</firstterm>, meaning that the index
! might produce false matches, and it is necessary
to check the actual table row to eliminate such false matches.
(<productname>PostgreSQL</productname> does this automatically when needed.)
GiST indexes are lossy because each document is represented in the
*************** SELECT plainto_tsquery('supernovae stars
*** 3281,3333 ****
</para>
<para>
- GIN indexes are not lossy for standard queries, but their performance
- depends logarithmically on the number of unique words.
- (However, GIN indexes store only the words (lexemes) of <type>tsvector</>
- values, and not their weight labels. Thus a table row recheck is needed
- when using a query that involves weights.)
- </para>
-
- <para>
- In choosing which index type to use, GiST or GIN, consider these
- performance differences:
-
- <itemizedlist spacing="compact" mark="bullet">
- <listitem>
- <para>
- GIN index lookups are about three times faster than GiST
- </para>
- </listitem>
- <listitem>
- <para>
- GIN indexes take about three times longer to build than GiST
- </para>
- </listitem>
- <listitem>
- <para>
- GIN indexes are moderately slower to update than GiST indexes, but
- about 10 times slower if fast-update support was disabled
- (see <xref linkend="gin-fast-update"> for details)
- </para>
- </listitem>
- <listitem>
- <para>
- GIN indexes are two-to-three times larger than GiST indexes
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>
- As a rule of thumb, <acronym>GIN</acronym> indexes are best for static data
- because lookups are faster. For dynamic data, GiST indexes are
- faster to update. Specifically, <acronym>GiST</acronym> indexes are very
- good for dynamic data and fast if the number of unique words (lexemes) is
- under 100,000, while <acronym>GIN</acronym> indexes will handle 100,000+
- lexemes better but are slower to update.
- </para>
-
- <para>
Note that <acronym>GIN</acronym> index build time can often be improved
by increasing <xref linkend="guc-maintenance-work-mem">, while
<acronym>GiST</acronym> index build time is not sensitive to that
--- 3286,3291 ----
*************** SELECT plainto_tsquery('supernovae stars
*** 3335,3341 ****
</para>
<para>
! Partitioning of big collections and the proper use of GiST and GIN indexes
allows the implementation of very fast searches with online update.
Partitioning can be done at the database level using table inheritance,
or by distributing documents over
--- 3293,3299 ----
</para>
<para>
! Partitioning of big collections and the proper use of GIN and GiST indexes
allows the implementation of very fast searches with online update.
Partitioning can be done at the database level using table inheritance,
or by distributing documents over
view thread (2+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: Update docs for GIN index improvements
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox