public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v30 4/6] Subscripting documentation
29+ messages / 3 participants
[nested] [flat]
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v30 4/6] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 34bc0d0526..328a1da6fe 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -7911,6 +7911,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry><structfield>typdefaultbin</structfield></entry>
<entry><type>pg_node_tree</type></entry>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 9ec1af780b..057010157e 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -314,6 +319,7 @@
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 3da2365ea9..650e21b7e1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index 1b6aaf0a55..189f03b41e 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -600,6 +600,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular element. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ always can create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 175315f3d7..bd622f5ef3 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename> corresponding.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..d701631223
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedures to
+ handle subscripting expressions. They must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbsdata
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedures and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--g5st7yssq7xavhfb
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v30-0005-Polymorphic-subscripting.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v34 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 7e99928d0c..104cea3584 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8990,6 +8990,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index e486006224..23c1764571 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 828396d4a9..3bebb34c1a 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--j77abfbqdsj2lq6d
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v34-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v32 4/6] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--xuxpg5p32zizz4zy
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v32-0005-Polymorphic-subscripting.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v29 4/6] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 55694c4368..b8fde6af12 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8012,6 +8012,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry><structfield>typdefaultbin</structfield></entry>
<entry><type>pg_node_tree</type></entry>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index a3046f22d0..9a4ce542ba 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -314,6 +319,7 @@
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 3da2365ea9..650e21b7e1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index 6ff8751870..d6f2f1d55f 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -600,6 +600,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular element. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ always can create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 175315f3d7..bd622f5ef3 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename> corresponding.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..d701631223
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedures to
+ handle subscripting expressions. They must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbsdata
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedures and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--adquyx6kg5n26q6j
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v29-0005-Polymorphic-subscripting.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v31 4/6] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 9d8fa0bec3..439bee0678 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8957,6 +8957,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index c1ffb14571..5d3cf479b4 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 68179f71cd..34249f8c60 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..3bffe8049b 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular element. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ always can create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..ec67761c66 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename> corresponding.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..d701631223
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedures to
+ handle subscripting expressions. They must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbsdata
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedures and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--bvn7grlx53cj2mie
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v31-0005-Polymorphic-subscripting.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v35 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 113 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 204 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 475 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 7e99928d0c..104cea3584 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8990,6 +8990,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index e486006224..23c1764571 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 828396d4a9..3bebb34c1a 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..5282df0361
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,113 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ // You can also store any necessary information into sbsref->refopaque
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..aaac7927c6
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,204 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+
+ /* You can also store any necessary information into sbsref->refopaque */
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--evj76j7ekwacmcsz
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v35-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v32 4/6] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--ques2rolqe435p6o
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v32-0005-Polymorphic-subscripting.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* [PATCH v33 4/5] Subscripting documentation
@ 2019-02-01 10:47 erthalion <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: erthalion @ 2019-02-01 10:47 UTC (permalink / raw)
Supporting documentation for generalized subscripting. It includes the
description of a new field in pg_type, the new section for jsonb
documentation about subscripting feature on this data type, and also the
tutorial about how to write subscripting operator for a custom data type.
Reviewed-by: Tom Lane, Arthur Zakirov, Oleksandr Shulgin
---
doc/src/sgml/catalogs.sgml | 8 ++
doc/src/sgml/extend.sgml | 6 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/json.sgml | 39 ++++++
doc/src/sgml/ref/create_type.sgml | 33 ++++-
doc/src/sgml/xsubscripting.sgml | 111 +++++++++++++++++
src/tutorial/Makefile | 4 +-
src/tutorial/subscripting.c | 201 ++++++++++++++++++++++++++++++
src/tutorial/subscripting.source | 71 +++++++++++
9 files changed, 470 insertions(+), 4 deletions(-)
create mode 100644 doc/src/sgml/xsubscripting.sgml
create mode 100644 src/tutorial/subscripting.c
create mode 100644 src/tutorial/subscripting.source
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 048ff284f7..b17e9ee382 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8971,6 +8971,14 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry><structfield>typsubshandler</structfield></entry>
+ <entry><type>regproc</type></entry>
+ <entry><literal><link linkend="catalog-pg-proc"><structname>pg_proc</structname></link>.oid</literal></entry>
+ <entry>Custom subscripting function with type-specific logic for parsing
+ and validation, or 0 if this type doesn't support subscripting.</entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>typdefault</structfield> <type>text</type>
diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 890ff97b7a..65cf5706b6 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -33,6 +33,11 @@
operators (starting in <xref linkend="xoper"/>)
</para>
</listitem>
+ <listitem>
+ <para>
+ subscripting procedure (starting in <xref linkend="xsubscripting"/>)
+ </para>
+ </listitem>
<listitem>
<para>
operator classes for indexes (starting in <xref linkend="xindex"/>)
@@ -481,6 +486,7 @@ RETURNS anycompatible AS ...
&xaggr;
&xtypes;
&xoper;
+ &xsubscripting;
&xindex;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..4c3fd37fc1 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -69,6 +69,7 @@
<!ENTITY xplang SYSTEM "xplang.sgml">
<!ENTITY xoper SYSTEM "xoper.sgml">
<!ENTITY xtypes SYSTEM "xtypes.sgml">
+<!ENTITY xsubscripting SYSTEM "xsubscripting.sgml">
<!ENTITY plperl SYSTEM "plperl.sgml">
<!ENTITY plpython SYSTEM "plpython.sgml">
<!ENTITY plsql SYSTEM "plpgsql.sgml">
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index c0a6554d4d..5c538dca05 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -602,6 +602,45 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</para>
</sect2>
+ <sect2 id="jsonb-subscripting">
+ <title><type>jsonb</type> Subscripting</title>
+ <para>
+ <type>jsonb</type> data type supports array-style subscripting expressions
+ to extract or update particular elements. It's possible to use multiple
+ subscripting expressions to extract nested values. In this case, a chain of
+ subscripting expressions follows the same rules as the
+ <literal>path</literal> argument in <literal>jsonb_set</literal> function,
+ e.g. in case of arrays it is a 0-based operation or that negative integers
+ that appear in <literal>path</literal> count from the end of JSON arrays.
+ The result of subscripting expressions is always jsonb data type. An
+ example of subscripting syntax:
+<programlisting>
+-- Extract value by key
+SELECT ('{"a": 1}'::jsonb)['a'];
+
+-- Extract nested value by key path
+SELECT ('{"a": {"b": {"c": 1}}}'::jsonb)['a']['b']['c'];
+
+-- Extract element by index
+SELECT ('[1, "2", null]'::jsonb)[1];
+
+-- Update value by key
+UPDATE table_name SET jsonb_field['key'] = 1;
+
+-- Select records using where clause with subscripting. Since the result of
+-- subscripting is jsonb and we basically want to compare two jsonb objects, we
+-- need to put the value in double quotes to be able to convert it to jsonb.
+SELECT * FROM table_name WHERE jsonb_field['key'] = '"value"';
+</programlisting>
+
+ There is no special indexing support for such kind of expressions, but you
+ can always create a functional index that includes it
+<programlisting>
+CREATE INDEX idx ON table_name ((jsonb_field['key']));
+</programlisting>
+ </para>
+ </sect2>
+
<sect2>
<title>Transforms</title>
diff --git a/doc/src/sgml/ref/create_type.sgml b/doc/src/sgml/ref/create_type.sgml
index 111f8e65d2..a34df4d247 100644
--- a/doc/src/sgml/ref/create_type.sgml
+++ b/doc/src/sgml/ref/create_type.sgml
@@ -54,6 +54,7 @@ CREATE TYPE <replaceable class="parameter">name</replaceable> (
[ , ELEMENT = <replaceable class="parameter">element</replaceable> ]
[ , DELIMITER = <replaceable class="parameter">delimiter</replaceable> ]
[ , COLLATABLE = <replaceable class="parameter">collatable</replaceable> ]
+ [ , SUBSCRIPTING_HANDLER = <replaceable class="parameter">subscripting_handler_function</replaceable> ]
)
CREATE TYPE <replaceable class="parameter">name</replaceable>
@@ -196,8 +197,9 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
<replaceable class="parameter">receive_function</replaceable>,
<replaceable class="parameter">send_function</replaceable>,
<replaceable class="parameter">type_modifier_input_function</replaceable>,
- <replaceable class="parameter">type_modifier_output_function</replaceable> and
- <replaceable class="parameter">analyze_function</replaceable>
+ <replaceable class="parameter">type_modifier_output_function</replaceable>,
+ <replaceable class="parameter">analyze_function</replaceable>,
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
are optional. Generally these functions have to be coded in C
or another low-level language.
</para>
@@ -454,6 +456,22 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
make use of the collation information; this does not happen
automatically merely by marking the type collatable.
</para>
+
+ <para>
+ The optional
+ <replaceable class="parameter">subscripting_handler_function</replaceable>
+ contains type-specific logic for subscripting of the data type.
+ By default, there is no such function provided, which means that the data
+ type doesn't support subscripting. The subscripting function must be
+ declared to take a single argument of type <type>internal</type>, and return
+ a <type>internal</type> result. There are two examples of implementation for
+ subscripting functions in case of array
+ (<replaceable class="parameter">array_subscripting_handler</replaceable>)
+ and jsonb
+ (<replaceable class="parameter">jsonb_subscripting_handler</replaceable>)
+ types in <filename>src/backend/utils/adt/arrayfuncs.c</filename> and
+ <filename>src/backend/utils/adt/jsonfuncs.c</filename>, respectively.
+ </para>
</refsect2>
<refsect2>
@@ -769,6 +787,17 @@ CREATE TYPE <replaceable class="parameter">name</replaceable>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">subscripting_handler_function</replaceable></term>
+ <listitem>
+ <para>
+ The name of a function that returns list of type-specific callback functions to
+ support subscripting logic for the data type.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/doc/src/sgml/xsubscripting.sgml b/doc/src/sgml/xsubscripting.sgml
new file mode 100644
index 0000000000..7224e81fa2
--- /dev/null
+++ b/doc/src/sgml/xsubscripting.sgml
@@ -0,0 +1,111 @@
+<!-- doc/src/sgml/xsubscripting.sgml -->
+
+ <sect1 id="xsubscripting">
+ <title>User-defined subscripting procedure</title>
+
+ <indexterm zone="xsubscripting">
+ <primary>custom subscripting</primary>
+ </indexterm>
+ <para>
+ When you define a new base type, you can also specify a custom procedure to
+ handle subscripting expressions. It must contain logic for verification and
+ evaluation of this expression, i.e. fetching or updating some data in this
+ data type. For instance:
+</para>
+<programlisting><![CDATA[
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = someType;
+ sbsref->refassgntype = someType;
+
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ // some validation and coercion logic
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some assignment logic
+
+ return newContainer;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ // Some fetch logic based on sbstate
+}]]>
+</programlisting>
+
+<para>
+ Then you can define a subscripting procedure and a custom data type:
+</para>
+<programlisting>
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '<replaceable>filename</replaceable>'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 4,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler,
+);
+</programlisting>
+
+<para>
+ and use it as usual:
+</para>
+<programlisting>
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
+</programlisting>
+
+
+ <para>
+ The examples of custom subscripting implementation can be found in
+ <filename>subscripting.sql</filename> and <filename>subscripting.c</filename>
+ in the <filename>src/tutorial</filename> directory of the source distribution.
+ See the <filename>README</filename> file in that directory for instructions
+ about running the examples.
+ </para>
+
+</sect1>
diff --git a/src/tutorial/Makefile b/src/tutorial/Makefile
index 16dc390f71..0ead60c2d4 100644
--- a/src/tutorial/Makefile
+++ b/src/tutorial/Makefile
@@ -13,8 +13,8 @@
#
#-------------------------------------------------------------------------
-MODULES = complex funcs
-DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql
+MODULES = complex funcs subscripting
+DATA_built = advanced.sql basics.sql complex.sql funcs.sql syscat.sql subscripting.sql
ifdef NO_PGXS
subdir = src/tutorial
diff --git a/src/tutorial/subscripting.c b/src/tutorial/subscripting.c
new file mode 100644
index 0000000000..1eb8c45652
--- /dev/null
+++ b/src/tutorial/subscripting.c
@@ -0,0 +1,201 @@
+/*
+ * src/tutorial/subscripting.c
+ *
+ ******************************************************************************
+ This file contains routines that can be bound to a Postgres backend and
+ called by the backend in the process of processing queries. The calling
+ format for these routines is dictated by Postgres architecture.
+******************************************************************************/
+
+#include "postgres.h"
+
+#include "catalog/pg_type.h"
+#include "executor/executor.h"
+#include "executor/execExpr.h"
+#include "nodes/nodeFuncs.h"
+#include "parser/parse_coerce.h"
+#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct Custom
+{
+ int first;
+ int second;
+} Custom;
+
+SubscriptingRef * custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref);
+SubscriptingRef * custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate);
+Datum custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate);
+Datum custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate);
+
+PG_FUNCTION_INFO_V1(custom_in);
+PG_FUNCTION_INFO_V1(custom_out);
+PG_FUNCTION_INFO_V1(custom_subscripting_handler);
+
+/*****************************************************************************
+ * Input/Output functions
+ *****************************************************************************/
+
+Datum
+custom_in(PG_FUNCTION_ARGS)
+{
+ char *str = PG_GETARG_CSTRING(0);
+ int firstValue,
+ secondValue;
+ Custom *result;
+
+ if (sscanf(str, " ( %d , %d )", &firstValue, &secondValue) != 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid input syntax for complex: \"%s\"",
+ str)));
+
+
+ result = (Custom *) palloc(sizeof(Custom));
+ result->first = firstValue;
+ result->second = secondValue;
+ PG_RETURN_POINTER(result);
+}
+
+Datum
+custom_out(PG_FUNCTION_ARGS)
+{
+ Custom *custom = (Custom *) PG_GETARG_POINTER(0);
+ char *result;
+
+ result = psprintf("(%d, %d)", custom->first, custom->second);
+ PG_RETURN_CSTRING(result);
+}
+
+/*****************************************************************************
+ * Custom subscripting logic functions
+ *****************************************************************************/
+
+Datum
+custom_subscripting_handler(PG_FUNCTION_ARGS)
+{
+ SubscriptRoutines *sbsroutines = (SubscriptRoutines *)
+ palloc(sizeof(SubscriptRoutines));
+
+ sbsroutines->prepare = custom_subscript_prepare;
+ sbsroutines->validate = custom_subscript_validate;
+ sbsroutines->fetch = custom_subscript_fetch;
+ sbsroutines->assign = custom_subscript_assign;
+
+ PG_RETURN_POINTER(sbsroutines);
+}
+
+SubscriptingRef *
+custom_subscript_prepare(bool isAssignment, SubscriptingRef *sbsref)
+{
+ sbsref->refelemtype = INT4OID;
+ sbsref->refassgntype = INT4OID;
+ return sbsref;
+}
+
+SubscriptingRef *
+custom_subscript_validate(bool isAssignment, SubscriptingRef *sbsref,
+ ParseState *pstate)
+{
+ List *upperIndexpr = NIL;
+ ListCell *l;
+
+ if (sbsref->reflowerindexpr != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *)lfirst(sbsref->reflowerindexpr->head))))));
+
+ foreach(l, sbsref->refupperindexpr)
+ {
+ Node *subexpr = (Node *) lfirst(l);
+
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript does not support slices"),
+ parser_errposition(pstate, exprLocation(
+ ((Node *) lfirst(sbsref->refupperindexpr->head))))));
+
+ subexpr = coerce_to_target_type(pstate,
+ subexpr, exprType(subexpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (subexpr == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom subscript must have integer type"),
+ parser_errposition(pstate, exprLocation(subexpr))));
+
+ upperIndexpr = lappend(upperIndexpr, subexpr);
+
+ if (isAssignment)
+ {
+ Node *assignExpr = (Node *) sbsref->refassgnexpr;
+ Node *new_from;
+
+ new_from = coerce_to_target_type(pstate,
+ assignExpr, exprType(assignExpr),
+ INT4OID, -1,
+ COERCION_ASSIGNMENT,
+ COERCE_IMPLICIT_CAST,
+ -1);
+ if (new_from == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("custom assignment requires int type"),
+ errhint("You will need to rewrite or cast the expression."),
+ parser_errposition(pstate, exprLocation(assignExpr))));
+ sbsref->refassgnexpr = (Expr *)new_from;
+ }
+ }
+
+ sbsref->refupperindexpr = upperIndexpr;
+
+ return sbsref;
+}
+
+Datum
+custom_subscript_fetch(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ Custom *container= (Custom *) containerSource;
+ int index;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ return (Datum) container->first;
+ else
+ return (Datum) container->second;
+}
+
+Datum
+custom_subscript_assign(Datum containerSource, SubscriptingRefState *sbstate)
+{
+ int index;
+ Custom *container = (Custom *) containerSource;
+
+ if (sbstate->resnull)
+ return containerSource;
+
+ if (sbstate->numupper != 1)
+ ereport(ERROR, (errmsg("custom does not support nested subscripting")));
+
+ index = DatumGetInt32(sbstate->upperindex[0]);
+
+ if (index == 1)
+ container->first = DatumGetInt32(sbstate->replacevalue);
+ else
+ container->second = DatumGetInt32(sbstate->replacevalue);
+
+ return (Datum) container;
+}
diff --git a/src/tutorial/subscripting.source b/src/tutorial/subscripting.source
new file mode 100644
index 0000000000..837cf30612
--- /dev/null
+++ b/src/tutorial/subscripting.source
@@ -0,0 +1,71 @@
+---------------------------------------------------------------------------
+--
+-- subscripting.sql-
+-- This file shows how to create a new subscripting procedure for
+-- user-defined type.
+--
+--
+-- Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+-- Portions Copyright (c) 1994, Regents of the University of California
+--
+-- src/tutorial/subscripting.source
+--
+---------------------------------------------------------------------------
+
+-----------------------------
+-- Creating a new type:
+-- We are going to create a new type called 'complex' which represents
+-- complex numbers.
+-- A user-defined type must have an input and an output function, and
+-- optionally can have binary input and output functions. All of these
+-- are usually user-defined C functions.
+-----------------------------
+
+-- Assume the user defined functions are in /home/erthalion/programms/postgresql-master/src/tutorial/complex$DLSUFFIX
+-- (we do not want to assume this is in the dynamic loader search path).
+-- Look at $PWD/complex.c for the source. Note that we declare all of
+-- them as STRICT, so we do not need to cope with NULL inputs in the
+-- C code. We also mark them IMMUTABLE, since they always return the
+-- same outputs given the same inputs.
+
+-- the input function 'complex_in' takes a null-terminated string (the
+-- textual representation of the type) and turns it into the internal
+-- (in memory) representation. You will get a message telling you 'complex'
+-- does not exist yet but that's okay.
+
+CREATE FUNCTION custom_in(cstring)
+ RETURNS custom
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+-- the output function 'complex_out' takes the internal representation and
+-- converts it into the textual representation.
+
+CREATE FUNCTION custom_out(custom)
+ RETURNS cstring
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE FUNCTION custom_subscripting_handler(internal)
+ RETURNS internal
+ AS '_OBJWD_/subscripting'
+ LANGUAGE C IMMUTABLE STRICT;
+
+CREATE TYPE custom (
+ internallength = 8,
+ input = custom_in,
+ output = custom_out,
+ subscripting_handler = custom_subscripting_handler
+);
+
+-- we can use it in a table
+
+CREATE TABLE test_subscripting (
+ data custom
+);
+
+INSERT INTO test_subscripting VALUES ('(1, 2)');
+
+SELECT data[0] from test_subscripting;
+
+UPDATE test_subscripting SET data[1] = 3;
--
2.21.0
--u66dlpg6gaevsofr
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0005-Filling-gaps-in-jsonb-arrays.patch"
^ permalink raw reply [nested|flat] 29+ messages in thread
* further #include cleanup (IWYU)
@ 2024-10-20 07:50 Peter Eisentraut <[email protected]>
2024-10-20 09:37 ` Re: further #include cleanup (IWYU) Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 29+ messages in thread
From: Peter Eisentraut @ 2024-10-20 07:50 UTC (permalink / raw)
To: pgsql-hackers
This is a continuation of [0] ("backend *.c #include cleanup (IWYU)"),
which removed a bunch of unneeded #include's, found by the
include-what-you-use (IWYU) tool. I went through the rest of the code
and did similar cleanups. The patches are organized by code area, but
they are otherwise not separate. The first patch ("Remove unused
#include's from backend .c files") is mostly stuff that is new since
[0]. The remaining patches deal with code that wasn't touched by [0],
so a lot of the changes have similar patterns as [0].
[0]:
https://www.postgresql.org/message-id/flat/af837490-6b2f-46df-ba05-37ea6a6653fc%40eisentraut.org
From 365b6f236589b02ada99b55acdc4cfff33c71a05 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Thu, 8 Feb 2024 16:26:33 +0100
Subject: [PATCH 1/5] Remove unused #include's from backend .c files
as determined by IWYU
---
src/backend/access/brin/brin_bloom.c | 1 -
src/backend/access/gin/ginvacuum.c | 1 -
src/backend/access/heap/vacuumlazy.c | 2 --
src/backend/access/nbtree/nbtree.c | 3 ---
src/backend/access/nbtree/nbtsort.c | 1 -
src/backend/access/sequence/sequence.c | 2 +-
src/backend/access/transam/slru.c | 2 +-
src/backend/access/transam/xlog.c | 2 --
src/backend/backup/basebackup.c | 1 -
src/backend/backup/basebackup_incremental.c | 2 --
src/backend/catalog/pg_depend.c | 1 -
src/backend/commands/analyze.c | 2 --
src/backend/commands/cluster.c | 1 -
src/backend/commands/copyfromparse.c | 1 -
src/backend/commands/createas.c | 2 --
src/backend/commands/indexcmds.c | 1 -
src/backend/commands/vacuum.c | 1 -
src/backend/commands/waitlsn.c | 2 --
src/backend/executor/nodeBitmapHeapscan.c | 1 -
src/backend/executor/nodeHash.c | 1 -
src/backend/lib/integerset.c | 1 -
src/backend/libpq/be-secure-gssapi.c | 2 +-
src/backend/libpq/be-secure-openssl.c | 3 +--
src/backend/main/main.c | 1 -
src/backend/parser/gram.y | 4 ---
src/backend/parser/parse_clause.c | 2 --
src/backend/parser/parse_expr.c | 2 --
src/backend/parser/parse_jsontable.c | 4 +--
src/backend/postmaster/auxprocess.c | 5 ----
src/backend/postmaster/launch_backend.c | 26 +++++++------------
src/backend/postmaster/postmaster.c | 5 +---
src/backend/postmaster/walwriter.c | 2 --
src/backend/replication/logical/slotsync.c | 2 --
src/backend/replication/slotfuncs.c | 4 ---
src/backend/storage/aio/read_stream.c | 1 -
src/backend/storage/buffer/buf_init.c | 1 -
src/backend/storage/file/fd.c | 1 -
src/backend/storage/ipc/ipci.c | 2 --
src/backend/storage/ipc/sinvaladt.c | 1 -
src/backend/storage/lmgr/lmgr.c | 1 -
src/backend/storage/lmgr/lock.c | 1 -
src/backend/storage/lmgr/s_lock.c | 1 -
src/backend/storage/page/checksum.c | 2 +-
src/backend/storage/smgr/bulk_write.c | 1 -
src/backend/tcop/fastpath.c | 1 -
src/backend/tcop/postgres.c | 1 -
src/backend/utils/Gen_fmgrtab.pl | 1 -
src/backend/utils/activity/backend_progress.c | 1 -
src/backend/utils/activity/backend_status.c | 2 --
src/backend/utils/activity/pgstat.c | 1 -
src/backend/utils/activity/pgstat_relation.c | 1 -
src/backend/utils/activity/wait_event.c | 1 -
src/backend/utils/adt/float.c | 1 -
src/backend/utils/adt/jsonpath_exec.c | 2 --
src/backend/utils/adt/pg_upgrade_support.c | 1 -
src/backend/utils/adt/waitfuncs.c | 2 +-
src/backend/utils/hash/pg_crc.c | 2 +-
src/backend/utils/misc/injection_point.c | 7 ++---
src/backend/utils/misc/pg_config.c | 1 -
59 files changed, 22 insertions(+), 108 deletions(-)
diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c
index f94e2b0bfbd..2e210d3e227 100644
--- a/src/backend/access/brin/brin_bloom.c
+++ b/src/backend/access/brin/brin_bloom.c
@@ -124,7 +124,6 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/pg_am.h"
-#include "catalog/pg_amop.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
#include "utils/fmgrprotos.h"
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index b3f415e2849..c6e8b39d4c9 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -19,7 +19,6 @@
#include "access/xloginsert.h"
#include "commands/vacuum.h"
#include "miscadmin.h"
-#include "postmaster/autovacuum.h"
#include "storage/indexfsm.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..793bd33cb4d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -35,7 +35,6 @@
#include "access/genam.h"
#include "access/heapam.h"
-#include "access/heapam_xlog.h"
#include "access/htup_details.h"
#include "access/multixact.h"
#include "access/tidstore.h"
@@ -56,7 +55,6 @@
#include "storage/freespace.h"
#include "storage/lmgr.h"
#include "utils/lsyscache.h"
-#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/timestamp.h"
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index f4f79f27062..484ede8c2e1 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -20,10 +20,8 @@
#include "access/nbtree.h"
#include "access/relscan.h"
-#include "access/xloginsert.h"
#include "commands/progress.h"
#include "commands/vacuum.h"
-#include "miscadmin.h"
#include "nodes/execnodes.h"
#include "pgstat.h"
#include "storage/bulk_write.h"
@@ -31,7 +29,6 @@
#include "storage/indexfsm.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
-#include "storage/smgr.h"
#include "utils/fmgrprotos.h"
#include "utils/index_selfuncs.h"
#include "utils/memutils.h"
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 5cca0d4f520..fb9a05f7af1 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -45,7 +45,6 @@
#include "access/relscan.h"
#include "access/table.h"
#include "access/xact.h"
-#include "access/xloginsert.h"
#include "catalog/index.h"
#include "commands/progress.h"
#include "executor/instrument.h"
diff --git a/src/backend/access/sequence/sequence.c b/src/backend/access/sequence/sequence.c
index 8d6b7bb5dc2..d2cf95aadc2 100644
--- a/src/backend/access/sequence/sequence.c
+++ b/src/backend/access/sequence/sequence.c
@@ -22,7 +22,7 @@
#include "access/relation.h"
#include "access/sequence.h"
-#include "storage/lmgr.h"
+#include "utils/rel.h"
static inline void validate_relation_kind(Relation r);
diff --git a/src/backend/access/transam/slru.c b/src/backend/access/transam/slru.c
index 889eff1815b..f249c3cd05e 100644
--- a/src/backend/access/transam/slru.c
+++ b/src/backend/access/transam/slru.c
@@ -70,7 +70,7 @@
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/shmem.h"
-#include "utils/guc_hooks.h"
+#include "utils/guc.h"
/*
* Converts segment number to the filename of the segment.
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 9102c8d772e..11f623cabd1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -74,7 +74,6 @@
#include "pg_trace.h"
#include "pgstat.h"
#include "port/atomics.h"
-#include "port/pg_iovec.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
#include "postmaster/walsummarizer.h"
@@ -98,7 +97,6 @@
#include "utils/guc_hooks.h"
#include "utils/guc_tables.h"
#include "utils/injection_point.h"
-#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
diff --git a/src/backend/backup/basebackup.c b/src/backend/backup/basebackup.c
index 0f8cddcbeeb..e2ed9081d1c 100644
--- a/src/backend/backup/basebackup.c
+++ b/src/backend/backup/basebackup.c
@@ -33,7 +33,6 @@
#include "nodes/pg_list.h"
#include "pgstat.h"
#include "pgtar.h"
-#include "port.h"
#include "postmaster/syslogger.h"
#include "postmaster/walsummarizer.h"
#include "replication/slot.h"
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index f581a5f9e68..275615877eb 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -27,9 +27,7 @@
#include "common/hashfn.h"
#include "common/int.h"
#include "common/parse_manifest.h"
-#include "datatype/timestamp.h"
#include "postmaster/walsummarizer.h"
-#include "utils/timestamp.h"
#define BLOCKS_PER_READ 512
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index cfd7ef51dfa..2b4514e8a35 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -28,7 +28,6 @@
#include "miscadmin.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
-#include "utils/syscache.h"
#include "utils/rel.h"
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 38fb4c3ef23..9a56de2282f 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -41,7 +41,6 @@
#include "parser/parse_oper.h"
#include "parser/parse_relation.h"
#include "pgstat.h"
-#include "postmaster/autovacuum.h"
#include "statistics/extended_stats_internal.h"
#include "statistics/statistics.h"
#include "storage/bufmgr.h"
@@ -54,7 +53,6 @@
#include "utils/pg_rusage.h"
#include "utils/sampling.h"
#include "utils/sortsupport.h"
-#include "utils/spccache.h"
#include "utils/syscache.h"
#include "utils/timestamp.h"
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 78f96789b0e..ae0863d9a25 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -32,7 +32,6 @@
#include "catalog/namespace.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_am.h"
-#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
#include "catalog/toasting.h"
#include "commands/cluster.h"
diff --git a/src/backend/commands/copyfromparse.c b/src/backend/commands/copyfromparse.c
index 654fecb1b14..d1d43b53d83 100644
--- a/src/backend/commands/copyfromparse.c
+++ b/src/backend/commands/copyfromparse.c
@@ -70,7 +70,6 @@
#include "libpq/pqformat.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/miscnodes.h"
#include "pgstat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 68ec122dbf9..aaeaaffe405 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -35,14 +35,12 @@
#include "commands/prepare.h"
#include "commands/tablecmds.h"
#include "commands/view.h"
-#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "rewrite/rewriteHandler.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index e33ad815295..2f652463e3c 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -58,7 +58,6 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ac8f5d9c259..86f36b36954 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -34,7 +34,6 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
-#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
diff --git a/src/backend/commands/waitlsn.c b/src/backend/commands/waitlsn.c
index 501938f4330..ce1537136c8 100644
--- a/src/backend/commands/waitlsn.c
+++ b/src/backend/commands/waitlsn.c
@@ -21,7 +21,6 @@
#include "access/xlog.h"
#include "access/xlogrecovery.h"
#include "commands/waitlsn.h"
-#include "funcapi.h"
#include "miscadmin.h"
#include "storage/latch.h"
#include "storage/proc.h"
@@ -29,7 +28,6 @@
#include "utils/fmgrprotos.h"
#include "utils/pg_lsn.h"
#include "utils/snapmgr.h"
-#include "utils/wait_event_types.h"
static int waitlsn_cmp(const pairingheap_node *a, const pairingheap_node *b,
void *arg);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3c63bdd93df..01d825c47bd 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -46,7 +46,6 @@
#include "pgstat.h"
#include "storage/bufmgr.h"
#include "utils/rel.h"
-#include "utils/snapmgr.h"
#include "utils/spccache.h"
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 0456a017dc6..3e22d50e3a4 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -35,7 +35,6 @@
#include "executor/nodeHash.h"
#include "executor/nodeHashjoin.h"
#include "miscadmin.h"
-#include "port/atomics.h"
#include "port/pg_bitutils.h"
#include "utils/dynahash.h"
#include "utils/lsyscache.h"
diff --git a/src/backend/lib/integerset.c b/src/backend/lib/integerset.c
index 9f1d4db52b2..bd9b28e19bd 100644
--- a/src/backend/lib/integerset.c
+++ b/src/backend/lib/integerset.c
@@ -72,7 +72,6 @@
#include "postgres.h"
#include "lib/integerset.h"
-#include "port/pg_bitutils.h"
#include "utils/memutils.h"
diff --git a/src/backend/libpq/be-secure-gssapi.c b/src/backend/libpq/be-secure-gssapi.c
index 2d36c76324a..5a009776d12 100644
--- a/src/backend/libpq/be-secure-gssapi.c
+++ b/src/backend/libpq/be-secure-gssapi.c
@@ -18,9 +18,9 @@
#include "libpq/auth.h"
#include "libpq/be-gssapi-common.h"
#include "libpq/libpq.h"
-#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "port/pg_bswap.h"
#include "utils/injection_point.h"
#include "utils/memutils.h"
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index 9d503104be3..b3d3f83a157 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -33,8 +33,7 @@
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/latch.h"
-#include "tcop/tcopprot.h"
-#include "utils/builtins.h"
+#include "utils/guc.h"
#include "utils/memutils.h"
/*
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index 4672aab8378..aea93a02298 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -32,7 +32,6 @@
#include "bootstrap/bootstrap.h"
#include "common/username.h"
-#include "port/atomics.h"
#include "postmaster/postmaster.h"
#include "tcop/tcopprot.h"
#include "utils/help_config.h"
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4aa8646af7b..f63d1538df7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -48,7 +48,6 @@
#include <ctype.h>
#include <limits.h>
-#include "access/tableam.h"
#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -59,10 +58,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parser.h"
-#include "storage/lmgr.h"
-#include "utils/date.h"
#include "utils/datetime.h"
-#include "utils/numeric.h"
#include "utils/xml.h"
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 4c976909088..979926b6052 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -40,11 +40,9 @@
#include "parser/parse_target.h"
#include "parser/parse_type.h"
#include "parser/parser.h"
-#include "parser/parsetree.h"
#include "rewrite/rewriteManip.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
-#include "utils/guc.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/syscache.h"
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index ef0b560f5e3..c2806297aa4 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -16,7 +16,6 @@
#include "postgres.h"
#include "catalog/pg_aggregate.h"
-#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "commands/dbcommands.h"
#include "miscadmin.h"
@@ -37,7 +36,6 @@
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/fmgroids.h"
-#include "utils/jsonb.h"
#include "utils/lsyscache.h"
#include "utils/timestamp.h"
#include "utils/xml.h"
diff --git a/src/backend/parser/parse_jsontable.c b/src/backend/parser/parse_jsontable.c
index 6d4e4968202..ecb140e6e71 100644
--- a/src/backend/parser/parse_jsontable.c
+++ b/src/backend/parser/parse_jsontable.c
@@ -15,9 +15,7 @@
#include "postgres.h"
-#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
-#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -26,7 +24,7 @@
#include "parser/parse_expr.h"
#include "parser/parse_relation.h"
#include "parser/parse_type.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/json.h"
#include "utils/lsyscache.h"
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 74b8a00c94b..d19174bda39 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -18,11 +18,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/auxprocess.h"
-#include "postmaster/bgwriter.h"
-#include "postmaster/startup.h"
-#include "postmaster/walsummarizer.h"
-#include "postmaster/walwriter.h"
-#include "replication/walreceiver.h"
#include "storage/condition_variable.h"
#include "storage/ipc.h"
#include "storage/proc.h"
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 0ae23fdf55e..25020cedad0 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,15 +33,9 @@
#include <unistd.h>
-#include "access/xlog.h"
-#include "common/file_utils.h"
#include "libpq/libpq-be.h"
-#include "libpq/pqsignal.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
-#include "port.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
#include "postmaster/fork_process.h"
@@ -54,20 +48,9 @@
#include "replication/slotsync.h"
#include "replication/walreceiver.h"
#include "storage/dsm.h"
-#include "storage/fd.h"
-#include "storage/ipc.h"
#include "storage/pg_shmem.h"
-#include "storage/pmsignal.h"
-#include "storage/proc.h"
-#include "storage/procsignal.h"
#include "tcop/backend_startup.h"
-#include "tcop/tcopprot.h"
-#include "utils/builtins.h"
-#include "utils/datetime.h"
-#include "utils/guc.h"
-#include "utils/injection_point.h"
#include "utils/memutils.h"
-#include "utils/timestamp.h"
#ifdef EXEC_BACKEND
#include "nodes/queryjumble.h"
@@ -78,6 +61,15 @@
#ifdef EXEC_BACKEND
+#include "common/file_utils.h"
+#include "storage/fd.h"
+#include "storage/lwlock.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "storage/procsignal.h"
+#include "tcop/tcopprot.h"
+#include "utils/injection_point.h"
+
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
typedef struct
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 85fd24e8287..407f55996ba 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -92,8 +92,6 @@
#include "access/xlog.h"
#include "access/xlogrecovery.h"
#include "common/file_perm.h"
-#include "common/file_utils.h"
-#include "common/ip.h"
#include "common/pg_prng.h"
#include "lib/ilist.h"
#include "libpq/libpq.h"
@@ -102,7 +100,6 @@
#include "pgstat.h"
#include "port/pg_bswap.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -114,7 +111,6 @@
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pmsignal.h"
-#include "storage/proc.h"
#include "tcop/backend_startup.h"
#include "tcop/tcopprot.h"
#include "utils/datetime.h"
@@ -124,6 +120,7 @@
#include "utils/varlena.h"
#ifdef EXEC_BACKEND
+#include "common/file_utils.h"
#include "storage/pg_shmem.h"
#endif
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 6e7918a78d4..91013f6e936 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -54,12 +54,10 @@
#include "storage/bufmgr.h"
#include "storage/condition_variable.h"
#include "storage/fd.h"
-#include "storage/ipc.h"
#include "storage/lwlock.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/smgr.h"
-#include "utils/guc.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
#include "utils/resowner.h"
diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c
index f9649eec1a5..d62186a5107 100644
--- a/src/backend/replication/logical/slotsync.c
+++ b/src/backend/replication/logical/slotsync.c
@@ -55,9 +55,7 @@
#include "commands/dbcommands.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
-#include "postmaster/postmaster.h"
#include "replication/logical.h"
#include "replication/slotsync.h"
#include "replication/snapbuild.h"
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index c7bfbb15e0a..488a161b3e7 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -17,16 +17,12 @@
#include "access/xlogrecovery.h"
#include "access/xlogutils.h"
#include "funcapi.h"
-#include "miscadmin.h"
-#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/slot.h"
#include "replication/slotsync.h"
#include "utils/builtins.h"
#include "utils/guc.h"
-#include "utils/inval.h"
#include "utils/pg_lsn.h"
-#include "utils/resowner.h"
/*
* Helper function for creating a new physical replication slot with
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index b8ea1180e25..3d30e6224f7 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -89,7 +89,6 @@
*/
#include "postgres.h"
-#include "catalog/pg_tablespace.h"
#include "miscadmin.h"
#include "storage/fd.h"
#include "storage/smgr.h"
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 09bec6449b6..56761a8eedc 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -16,7 +16,6 @@
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-#include "storage/proc.h"
BufferDescPadded *BufferDescriptors;
char *BufferBlocks;
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index ec1505802b9..42bf857e873 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -93,7 +93,6 @@
#include "common/pg_prng.h"
#include "miscadmin.h"
#include "pgstat.h"
-#include "portability/mem.h"
#include "postmaster/startup.h"
#include "storage/fd.h"
#include "storage/ipc.h"
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 10fc18f2529..b0852564388 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -31,7 +31,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
-#include "postmaster/postmaster.h"
#include "postmaster/walsummarizer.h"
#include "replication/logicallauncher.h"
#include "replication/origin.h"
@@ -50,7 +49,6 @@
#include "storage/procarray.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
-#include "storage/spin.h"
#include "utils/guc.h"
#include "utils/injection_point.h"
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index db59637c438..ff817449a44 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -17,7 +17,6 @@
#include <signal.h>
#include <unistd.h>
-#include "access/transam.h"
#include "miscadmin.h"
#include "storage/ipc.h"
#include "storage/proc.h"
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index 094522acb41..9f43620e472 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -24,7 +24,6 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "utils/inval.h"
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 09a8ac15784..97c0fd469dc 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -41,7 +41,6 @@
#include "pg_trace.h"
#include "storage/proc.h"
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "storage/spin.h"
#include "storage/standby.h"
#include "utils/memutils.h"
diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c
index 69549a65dba..f37d338566f 100644
--- a/src/backend/storage/lmgr/s_lock.c
+++ b/src/backend/storage/lmgr/s_lock.c
@@ -51,7 +51,6 @@
#include <unistd.h>
#include "common/pg_prng.h"
-#include "port/atomics.h"
#include "storage/s_lock.h"
#include "utils/wait_event.h"
diff --git a/src/backend/storage/page/checksum.c b/src/backend/storage/page/checksum.c
index 243878fe9e0..ffc3d245887 100644
--- a/src/backend/storage/page/checksum.c
+++ b/src/backend/storage/page/checksum.c
@@ -19,4 +19,4 @@
* external programs can incorporate the checksum code by #include'ing
* that file from the exported Postgres headers. (Compare our CRC code.)
*/
-#include "storage/checksum_impl.h"
+#include "storage/checksum_impl.h" /* IWYU pragma: keep */
diff --git a/src/backend/storage/smgr/bulk_write.c b/src/backend/storage/smgr/bulk_write.c
index 1a5f3ce96e1..f0a65bfe242 100644
--- a/src/backend/storage/smgr/bulk_write.c
+++ b/src/backend/storage/smgr/bulk_write.c
@@ -36,7 +36,6 @@
#include "access/xloginsert.h"
#include "access/xlogrecord.h"
-#include "storage/bufmgr.h"
#include "storage/bufpage.h"
#include "storage/bulk_write.h"
#include "storage/proc.h"
diff --git a/src/backend/tcop/fastpath.c b/src/backend/tcop/fastpath.c
index deac71335c2..c6281cf86d3 100644
--- a/src/backend/tcop/fastpath.c
+++ b/src/backend/tcop/fastpath.c
@@ -26,7 +26,6 @@
#include "libpq/protocol.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "port/pg_bswap.h"
#include "tcop/fastpath.h"
#include "tcop/tcopprot.h"
#include "utils/acl.h"
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7f5eada9d45..8cc23a9cef8 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -53,7 +53,6 @@
#include "pg_getopt.h"
#include "pg_trace.h"
#include "pgstat.h"
-#include "postmaster/autovacuum.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/logicallauncher.h"
diff --git a/src/backend/utils/Gen_fmgrtab.pl b/src/backend/utils/Gen_fmgrtab.pl
index 37628a85c73..738828b95c6 100644
--- a/src/backend/utils/Gen_fmgrtab.pl
+++ b/src/backend/utils/Gen_fmgrtab.pl
@@ -182,7 +182,6 @@
#include "postgres.h"
-#include "access/transam.h"
#include "utils/fmgrtab.h"
#include "utils/fmgrprotos.h"
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index c78c5eb5076..b80249a79e7 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -12,7 +12,6 @@
#include "access/parallel.h"
#include "libpq/pqformat.h"
-#include "port/atomics.h" /* for memory barriers */
#include "utils/backend_progress.h"
#include "utils/backend_status.h"
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 34a55e2177d..bdb3a296ca6 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -16,11 +16,9 @@
#include "miscadmin.h"
#include "pg_trace.h"
#include "pgstat.h"
-#include "port/atomics.h" /* for memory barriers */
#include "storage/ipc.h"
#include "storage/proc.h" /* for MyProc */
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h" /* for application_name */
#include "utils/memutils.h"
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index d1768a89f6e..be48432cc38 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -105,7 +105,6 @@
#include "access/xlog.h"
#include "lib/dshash.h"
#include "pgstat.h"
-#include "port/atomics.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/lwlock.h"
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..36d3adf7310 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -20,7 +20,6 @@
#include "access/twophase_rmgr.h"
#include "access/xact.h"
#include "catalog/catalog.h"
-#include "postmaster/autovacuum.h"
#include "utils/memutils.h"
#include "utils/pgstat_internal.h"
#include "utils/rel.h"
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index bbf59482be1..d930277140c 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -22,7 +22,6 @@
*/
#include "postgres.h"
-#include "port/pg_bitutils.h"
#include "storage/lmgr.h" /* for GetLockNameFromTagType */
#include "storage/lwlock.h" /* for GetLWLockIdentifier */
#include "storage/spin.h"
diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c
index 6fa6ffb51f5..f5927bcc564 100644
--- a/src/backend/utils/adt/float.c
+++ b/src/backend/utils/adt/float.c
@@ -27,7 +27,6 @@
#include "utils/float.h"
#include "utils/fmgrprotos.h"
#include "utils/sortsupport.h"
-#include "utils/timestamp.h"
/*
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 1184cba983a..ac214598037 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -61,7 +61,6 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
-#include "executor/execExpr.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "nodes/miscnodes.h"
@@ -74,7 +73,6 @@
#include "utils/formatting.h"
#include "utils/json.h"
#include "utils/jsonpath.h"
-#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/timestamp.h"
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index c54b08fe180..8a45b5827e3 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -28,7 +28,6 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/pg_lsn.h"
-#include "utils/syscache.h"
#define CHECK_IS_BINARY_UPGRADE \
diff --git a/src/backend/utils/adt/waitfuncs.c b/src/backend/utils/adt/waitfuncs.c
index e135c9e5e45..a083ced251e 100644
--- a/src/backend/utils/adt/waitfuncs.c
+++ b/src/backend/utils/adt/waitfuncs.c
@@ -17,7 +17,7 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/array.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/wait_event.h"
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
diff --git a/src/backend/utils/hash/pg_crc.c b/src/backend/utils/hash/pg_crc.c
index a85e6171864..2fc85405b63 100644
--- a/src/backend/utils/hash/pg_crc.c
+++ b/src/backend/utils/hash/pg_crc.c
@@ -20,7 +20,7 @@
#include "postgres.h"
#include "port/pg_crc32c.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/pg_crc.h"
#include "varatt.h"
diff --git a/src/backend/utils/misc/injection_point.c b/src/backend/utils/misc/injection_point.c
index 7c1a7cf889d..deb7d245ef8 100644
--- a/src/backend/utils/misc/injection_point.c
+++ b/src/backend/utils/misc/injection_point.c
@@ -17,6 +17,10 @@
*/
#include "postgres.h"
+#include "utils/injection_point.h"
+
+#ifdef USE_INJECTION_POINTS
+
#include <sys/stat.h>
#include "fmgr.h"
@@ -25,11 +29,8 @@
#include "storage/lwlock.h"
#include "storage/shmem.h"
#include "utils/hsearch.h"
-#include "utils/injection_point.h"
#include "utils/memutils.h"
-#ifdef USE_INJECTION_POINTS
-
/* Field sizes */
#define INJ_NAME_MAXLEN 64
#define INJ_LIB_MAXLEN 128
diff --git a/src/backend/utils/misc/pg_config.c b/src/backend/utils/misc/pg_config.c
index 2af47e1244f..89dd18355ae 100644
--- a/src/backend/utils/misc/pg_config.c
+++ b/src/backend/utils/misc/pg_config.c
@@ -17,7 +17,6 @@
#include "common/config_info.h"
#include "funcapi.h"
#include "miscadmin.h"
-#include "port.h"
#include "utils/builtins.h"
Datum
base-commit: 11c87216d134a606938531e52edab6189bce6c2d
--
2.47.0
From dc8dab38070ba6005aefb2cfe941a588f5362664 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 22 Jun 2024 10:22:29 +0200
Subject: [PATCH 2/5] Remove unused #include's from contrib .c files
as determined by IWYU
---
contrib/amcheck/verify_heapam.c | 6 +++++-
contrib/amcheck/verify_nbtree.c | 2 --
contrib/auth_delay/auth_delay.c | 2 --
contrib/auto_explain/auto_explain.c | 2 --
contrib/basic_archive/basic_archive.c | 1 -
contrib/bloom/blcost.c | 1 -
contrib/bloom/blinsert.c | 4 +---
contrib/bloom/blscan.c | 4 ----
contrib/bloom/blutils.c | 5 +----
contrib/bloom/blvacuum.c | 4 ----
contrib/bloom/blvalidate.c | 2 --
contrib/btree_gin/btree_gin.c | 3 ---
contrib/btree_gist/btree_bit.c | 3 +--
contrib/btree_gist/btree_bool.c | 1 -
contrib/btree_gist/btree_bytea.c | 3 +--
contrib/btree_gist/btree_date.c | 2 +-
contrib/btree_gist/btree_enum.c | 2 +-
contrib/btree_gist/btree_inet.c | 1 -
contrib/btree_gist/btree_interval.c | 2 +-
contrib/btree_gist/btree_macaddr.c | 2 +-
contrib/btree_gist/btree_macaddr8.c | 2 +-
contrib/btree_gist/btree_text.c | 3 ++-
contrib/btree_gist/btree_time.c | 2 +-
contrib/btree_gist/btree_ts.c | 4 ++--
contrib/btree_gist/btree_utils_num.h | 1 -
contrib/btree_gist/btree_utils_var.c | 3 +--
contrib/btree_gist/btree_utils_var.h | 1 -
contrib/citext/citext.c | 2 +-
contrib/dict_xsyn/dict_xsyn.c | 2 +-
contrib/fuzzystrmatch/fuzzystrmatch.c | 1 -
contrib/intarray/_int_bool.c | 1 -
contrib/intarray/_int_selfuncs.c | 3 +--
contrib/isn/isn.c | 1 -
contrib/lo/lo.c | 2 +-
contrib/ltree/ltree_io.c | 1 -
contrib/ltree/ltree_op.c | 4 +---
contrib/pageinspect/brinfuncs.c | 4 ----
contrib/pageinspect/fsmfuncs.c | 2 +-
contrib/pageinspect/ginfuncs.c | 3 ---
contrib/pageinspect/gistfuncs.c | 3 ---
contrib/pageinspect/heapfuncs.c | 1 -
contrib/pg_freespacemap/pg_freespacemap.c | 2 +-
contrib/pg_prewarm/autoprewarm.c | 9 +--------
contrib/pg_stat_statements/pg_stat_statements.c | 3 ---
contrib/pg_surgery/heap_surgery.c | 4 ++--
contrib/pg_visibility/pg_visibility.c | 2 --
contrib/pgcrypto/pgcrypto.c | 1 -
contrib/pgrowlocks/pgrowlocks.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 6 ------
contrib/pgstattuple/pgstatindex.c | 3 ---
contrib/pgstattuple/pgstattuple.c | 1 -
contrib/postgres_fdw/connection.c | 4 ----
contrib/postgres_fdw/deparse.c | 2 --
contrib/postgres_fdw/option.c | 1 -
contrib/postgres_fdw/postgres_fdw.c | 3 ---
contrib/spi/moddatetime.c | 2 +-
contrib/tablefunc/tablefunc.c | 1 +
contrib/tablefunc/tablefunc.h | 2 --
contrib/tsm_system_rows/tsm_system_rows.c | 1 -
contrib/tsm_system_time/tsm_system_time.c | 1 -
contrib/unaccent/unaccent.c | 2 --
contrib/xml2/xpath.c | 1 -
contrib/xml2/xslt_proc.c | 3 ---
63 files changed, 33 insertions(+), 121 deletions(-)
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index f2526ed63a2..9c74daaceed 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -12,18 +12,22 @@
#include "access/detoast.h"
#include "access/genam.h"
-#include "access/heapam.h"
#include "access/heaptoast.h"
#include "access/multixact.h"
+#include "access/relation.h"
+#include "access/table.h"
#include "access/toast_internals.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_class.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/procarray.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/rel.h"
PG_FUNCTION_INFO_V1(verify_heapam);
diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c
index 7cfb136763f..8b82797c10f 100644
--- a/contrib/amcheck/verify_nbtree.c
+++ b/contrib/amcheck/verify_nbtree.c
@@ -33,11 +33,9 @@
#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opfamily_d.h"
-#include "commands/tablecmds.h"
#include "common/pg_prng.h"
#include "lib/bloomfilter.h"
#include "miscadmin.h"
-#include "storage/lmgr.h"
#include "storage/smgr.h"
#include "utils/guc.h"
#include "utils/memutils.h"
diff --git a/contrib/auth_delay/auth_delay.c b/contrib/auth_delay/auth_delay.c
index ff0e1fd461b..e8dfef3290e 100644
--- a/contrib/auth_delay/auth_delay.c
+++ b/contrib/auth_delay/auth_delay.c
@@ -14,9 +14,7 @@
#include <limits.h>
#include "libpq/auth.h"
-#include "port.h"
#include "utils/guc.h"
-#include "utils/timestamp.h"
PG_MODULE_MAGIC;
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 677c135f59b..623a674f99c 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -18,8 +18,6 @@
#include "commands/explain.h"
#include "common/pg_prng.h"
#include "executor/instrument.h"
-#include "jit/jit.h"
-#include "nodes/params.h"
#include "utils/guc.h"
PG_MODULE_MAGIC;
diff --git a/contrib/basic_archive/basic_archive.c b/contrib/basic_archive/basic_archive.c
index 028cf51c25d..36e2feb4fb8 100644
--- a/contrib/basic_archive/basic_archive.c
+++ b/contrib/basic_archive/basic_archive.c
@@ -36,7 +36,6 @@
#include "storage/copydir.h"
#include "storage/fd.h"
#include "utils/guc.h"
-#include "utils/memutils.h"
PG_MODULE_MAGIC;
diff --git a/contrib/bloom/blcost.c b/contrib/bloom/blcost.c
index 9b629b2c3d5..0f2cf1546c7 100644
--- a/contrib/bloom/blcost.c
+++ b/contrib/bloom/blcost.c
@@ -13,7 +13,6 @@
#include "postgres.h"
#include "bloom.h"
-#include "fmgr.h"
#include "utils/selfuncs.h"
/*
diff --git a/contrib/bloom/blinsert.c b/contrib/bloom/blinsert.c
index f8a1061abb9..dbb79a51edd 100644
--- a/contrib/bloom/blinsert.c
+++ b/contrib/bloom/blinsert.c
@@ -16,11 +16,9 @@
#include "access/generic_xlog.h"
#include "access/tableam.h"
#include "bloom.h"
-#include "catalog/index.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "storage/bufmgr.h"
-#include "storage/indexfsm.h"
-#include "storage/smgr.h"
#include "utils/memutils.h"
#include "utils/rel.h"
diff --git a/contrib/bloom/blscan.c b/contrib/bloom/blscan.c
index 94b6b449c8c..bf455e579fe 100644
--- a/contrib/bloom/blscan.c
+++ b/contrib/bloom/blscan.c
@@ -15,11 +15,7 @@
#include "access/relscan.h"
#include "bloom.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/bufmgr.h"
-#include "storage/lmgr.h"
-#include "utils/memutils.h"
-#include "utils/rel.h"
/*
* Begin scan of bloom index.
diff --git a/contrib/bloom/blutils.c b/contrib/bloom/blutils.c
index a29330afcd3..af10199394a 100644
--- a/contrib/bloom/blutils.c
+++ b/contrib/bloom/blutils.c
@@ -17,14 +17,11 @@
#include "access/generic_xlog.h"
#include "access/reloptions.h"
#include "bloom.h"
-#include "catalog/index.h"
#include "commands/vacuum.h"
-#include "miscadmin.h"
#include "storage/bufmgr.h"
-#include "storage/freespace.h"
#include "storage/indexfsm.h"
-#include "storage/lmgr.h"
#include "utils/memutils.h"
+#include "varatt.h"
/* Signature dealing macros - note i is assumed to be of type int */
#define GETWORD(x,i) ( *( (BloomSignatureWord *)(x) + ( (i) / SIGNWORDBITS ) ) )
diff --git a/contrib/bloom/blvacuum.c b/contrib/bloom/blvacuum.c
index 0998240feaf..2a268fe0070 100644
--- a/contrib/bloom/blvacuum.c
+++ b/contrib/bloom/blvacuum.c
@@ -14,13 +14,9 @@
#include "access/genam.h"
#include "bloom.h"
-#include "catalog/storage.h"
#include "commands/vacuum.h"
-#include "miscadmin.h"
-#include "postmaster/autovacuum.h"
#include "storage/bufmgr.h"
#include "storage/indexfsm.h"
-#include "storage/lmgr.h"
/*
diff --git a/contrib/bloom/blvalidate.c b/contrib/bloom/blvalidate.c
index 88c5a791975..20d3ab9109b 100644
--- a/contrib/bloom/blvalidate.c
+++ b/contrib/bloom/blvalidate.c
@@ -20,8 +20,6 @@
#include "catalog/pg_opclass.h"
#include "catalog/pg_opfamily.h"
#include "catalog/pg_type.h"
-#include "utils/builtins.h"
-#include "utils/lsyscache.h"
#include "utils/regproc.h"
#include "utils/syscache.h"
diff --git a/contrib/btree_gin/btree_gin.c b/contrib/btree_gin/btree_gin.c
index 5e27906f802..533c55e9eaf 100644
--- a/contrib/btree_gin/btree_gin.c
+++ b/contrib/btree_gin/btree_gin.c
@@ -7,15 +7,12 @@
#include "access/stratnum.h"
#include "utils/builtins.h"
-#include "utils/bytea.h"
-#include "utils/cash.h"
#include "utils/date.h"
#include "utils/float.h"
#include "utils/inet.h"
#include "utils/numeric.h"
#include "utils/timestamp.h"
#include "utils/uuid.h"
-#include "utils/varbit.h"
PG_MODULE_MAGIC;
diff --git a/contrib/btree_gist/btree_bit.c b/contrib/btree_gist/btree_bit.c
index 6790f22b4b6..9d5e01a1977 100644
--- a/contrib/btree_gist/btree_bit.c
+++ b/contrib/btree_gist/btree_bit.c
@@ -5,8 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
-#include "utils/bytea.h"
+#include "utils/fmgrprotos.h"
#include "utils/varbit.h"
diff --git a/contrib/btree_gist/btree_bool.c b/contrib/btree_gist/btree_bool.c
index 8b2af129b52..bdc7834bb6a 100644
--- a/contrib/btree_gist/btree_bool.c
+++ b/contrib/btree_gist/btree_bool.c
@@ -5,7 +5,6 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "common/int.h"
typedef struct boolkey
{
diff --git a/contrib/btree_gist/btree_bytea.c b/contrib/btree_gist/btree_bytea.c
index 6b005f0157e..394e40ed16b 100644
--- a/contrib/btree_gist/btree_bytea.c
+++ b/contrib/btree_gist/btree_bytea.c
@@ -5,8 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
-#include "utils/bytea.h"
+#include "utils/fmgrprotos.h"
/*
diff --git a/contrib/btree_gist/btree_date.c b/contrib/btree_gist/btree_date.c
index 68a4107dbf0..5efb5ef96f5 100644
--- a/contrib/btree_gist/btree_date.c
+++ b/contrib/btree_gist/btree_date.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/date.h"
typedef struct
diff --git a/contrib/btree_gist/btree_enum.c b/contrib/btree_gist/btree_enum.c
index d4dc38a38e5..26682122f8e 100644
--- a/contrib/btree_gist/btree_enum.c
+++ b/contrib/btree_gist/btree_enum.c
@@ -6,7 +6,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
#include "fmgr.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
/* enums are really Oids, so we just use the same structure */
diff --git a/contrib/btree_gist/btree_inet.c b/contrib/btree_gist/btree_inet.c
index 2fb952dca83..c66ca99e0c3 100644
--- a/contrib/btree_gist/btree_inet.c
+++ b/contrib/btree_gist/btree_inet.c
@@ -7,7 +7,6 @@
#include "btree_utils_num.h"
#include "catalog/pg_type.h"
#include "utils/builtins.h"
-#include "utils/inet.h"
typedef struct inetkey
{
diff --git a/contrib/btree_gist/btree_interval.c b/contrib/btree_gist/btree_interval.c
index 156f2cebac5..c2f6eede226 100644
--- a/contrib/btree_gist/btree_interval.c
+++ b/contrib/btree_gist/btree_interval.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/timestamp.h"
typedef struct
diff --git a/contrib/btree_gist/btree_macaddr.c b/contrib/btree_gist/btree_macaddr.c
index 17290529c02..1367eaba735 100644
--- a/contrib/btree_gist/btree_macaddr.c
+++ b/contrib/btree_gist/btree_macaddr.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/inet.h"
typedef struct
diff --git a/contrib/btree_gist/btree_macaddr8.c b/contrib/btree_gist/btree_macaddr8.c
index 796cc4efee3..9676e5a15bd 100644
--- a/contrib/btree_gist/btree_macaddr8.c
+++ b/contrib/btree_gist/btree_macaddr8.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/inet.h"
typedef struct
diff --git a/contrib/btree_gist/btree_text.c b/contrib/btree_gist/btree_text.c
index be0eac7975b..28d6b5c9c2b 100644
--- a/contrib/btree_gist/btree_text.c
+++ b/contrib/btree_gist/btree_text.c
@@ -5,7 +5,8 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
+#include "mb/pg_wchar.h"
+#include "utils/fmgrprotos.h"
/*
** Text ops
diff --git a/contrib/btree_gist/btree_time.c b/contrib/btree_gist/btree_time.c
index d89401c0f51..3034bbb456b 100644
--- a/contrib/btree_gist/btree_time.c
+++ b/contrib/btree_gist/btree_time.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/date.h"
#include "utils/timestamp.h"
diff --git a/contrib/btree_gist/btree_ts.c b/contrib/btree_gist/btree_ts.c
index 3f5ba91891d..b3cf2d6f60d 100644
--- a/contrib/btree_gist/btree_ts.c
+++ b/contrib/btree_gist/btree_ts.c
@@ -7,8 +7,8 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
-#include "utils/datetime.h"
+#include "utils/fmgrprotos.h"
+#include "utils/timestamp.h"
#include "utils/float.h"
typedef struct
diff --git a/contrib/btree_gist/btree_utils_num.h b/contrib/btree_gist/btree_utils_num.h
index 11e892311ee..53e477d8b1e 100644
--- a/contrib/btree_gist/btree_utils_num.h
+++ b/contrib/btree_gist/btree_utils_num.h
@@ -9,7 +9,6 @@
#include "access/gist.h"
#include "btree_gist.h"
-#include "utils/rel.h"
typedef char GBT_NUMKEY;
diff --git a/contrib/btree_gist/btree_utils_var.c b/contrib/btree_gist/btree_utils_var.c
index 0c0e952f736..d9df2356cd1 100644
--- a/contrib/btree_gist/btree_utils_var.c
+++ b/contrib/btree_gist/btree_utils_var.c
@@ -9,8 +9,7 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
-#include "utils/pg_locale.h"
+#include "mb/pg_wchar.h"
#include "utils/rel.h"
/* used for key sorting */
diff --git a/contrib/btree_gist/btree_utils_var.h b/contrib/btree_gist/btree_utils_var.h
index 2f8def655c8..9302d41ced6 100644
--- a/contrib/btree_gist/btree_utils_var.h
+++ b/contrib/btree_gist/btree_utils_var.h
@@ -6,7 +6,6 @@
#include "access/gist.h"
#include "btree_gist.h"
-#include "mb/pg_wchar.h"
/* Variable length key */
typedef bytea GBT_VARKEY;
diff --git a/contrib/citext/citext.c b/contrib/citext/citext.c
index 26af935a70f..3c461ff2ff2 100644
--- a/contrib/citext/citext.c
+++ b/contrib/citext/citext.c
@@ -5,7 +5,7 @@
#include "catalog/pg_collation.h"
#include "common/hashfn.h"
-#include "utils/builtins.h"
+#include "fmgr.h"
#include "utils/formatting.h"
#include "utils/varlena.h"
#include "varatt.h"
diff --git a/contrib/dict_xsyn/dict_xsyn.c b/contrib/dict_xsyn/dict_xsyn.c
index 62d2d841417..3635ed1df84 100644
--- a/contrib/dict_xsyn/dict_xsyn.c
+++ b/contrib/dict_xsyn/dict_xsyn.c
@@ -16,7 +16,7 @@
#include "commands/defrem.h"
#include "tsearch/ts_locale.h"
-#include "tsearch/ts_utils.h"
+#include "tsearch/ts_public.h"
PG_MODULE_MAGIC;
diff --git a/contrib/fuzzystrmatch/fuzzystrmatch.c b/contrib/fuzzystrmatch/fuzzystrmatch.c
index 72ae2ab91b5..f243e2caed5 100644
--- a/contrib/fuzzystrmatch/fuzzystrmatch.c
+++ b/contrib/fuzzystrmatch/fuzzystrmatch.c
@@ -40,7 +40,6 @@
#include <ctype.h>
-#include "mb/pg_wchar.h"
#include "utils/builtins.h"
#include "utils/varlena.h"
#include "varatt.h"
diff --git a/contrib/intarray/_int_bool.c b/contrib/intarray/_int_bool.c
index 8fc6ad87fc7..8cf02c0e5da 100644
--- a/contrib/intarray/_int_bool.c
+++ b/contrib/intarray/_int_bool.c
@@ -5,7 +5,6 @@
#include "_int.h"
#include "miscadmin.h"
-#include "utils/builtins.h"
PG_FUNCTION_INFO_V1(bqarr_in);
PG_FUNCTION_INFO_V1(bqarr_out);
diff --git a/contrib/intarray/_int_selfuncs.c b/contrib/intarray/_int_selfuncs.c
index bc1ad3a8048..99179d0c6fc 100644
--- a/contrib/intarray/_int_selfuncs.c
+++ b/contrib/intarray/_int_selfuncs.c
@@ -20,10 +20,9 @@
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
-#include "utils/syscache.h"
PG_FUNCTION_INFO_V1(_int_overlap_sel);
PG_FUNCTION_INFO_V1(_int_contains_sel);
diff --git a/contrib/isn/isn.c b/contrib/isn/isn.c
index 71a543b9478..15bcda200c5 100644
--- a/contrib/isn/isn.c
+++ b/contrib/isn/isn.c
@@ -21,7 +21,6 @@
#include "UPC.h"
#include "fmgr.h"
#include "isn.h"
-#include "utils/builtins.h"
PG_MODULE_MAGIC;
diff --git a/contrib/lo/lo.c b/contrib/lo/lo.c
index 457be26c4ed..62488e45f3a 100644
--- a/contrib/lo/lo.c
+++ b/contrib/lo/lo.c
@@ -9,7 +9,7 @@
#include "commands/trigger.h"
#include "executor/spi.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
PG_MODULE_MAGIC;
diff --git a/contrib/ltree/ltree_io.c b/contrib/ltree/ltree_io.c
index 0a12c77a621..11eefc809b2 100644
--- a/contrib/ltree/ltree_io.c
+++ b/contrib/ltree/ltree_io.c
@@ -10,7 +10,6 @@
#include "crc32.h"
#include "libpq/pqformat.h"
#include "ltree.h"
-#include "utils/memutils.h"
#include "varatt.h"
diff --git a/contrib/ltree/ltree_op.c b/contrib/ltree/ltree_op.c
index 24a21d3ea01..0e30dee4658 100644
--- a/contrib/ltree/ltree_op.c
+++ b/contrib/ltree/ltree_op.c
@@ -7,13 +7,11 @@
#include <ctype.h>
-#include "access/htup_details.h"
-#include "catalog/pg_statistic.h"
#include "common/hashfn.h"
#include "ltree.h"
#include "utils/builtins.h"
-#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
+#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 5a38d926689..22621d584fa 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -9,20 +9,16 @@
*/
#include "postgres.h"
-#include "access/brin.h"
#include "access/brin_internal.h"
#include "access/brin_page.h"
-#include "access/brin_revmap.h"
#include "access/brin_tuple.h"
#include "access/htup_details.h"
-#include "catalog/index.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
#include "pageinspect.h"
-#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
diff --git a/contrib/pageinspect/fsmfuncs.c b/contrib/pageinspect/fsmfuncs.c
index f8ae0886244..31182144380 100644
--- a/contrib/pageinspect/fsmfuncs.c
+++ b/contrib/pageinspect/fsmfuncs.c
@@ -19,7 +19,7 @@
#include "postgres.h"
-#include "funcapi.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
#include "pageinspect.h"
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index 1dabbdb5889..7182d225d96 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -9,17 +9,14 @@
*/
#include "postgres.h"
-#include "access/gin.h"
#include "access/gin_private.h"
#include "access/htup_details.h"
-#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "pageinspect.h"
#include "utils/array.h"
#include "utils/builtins.h"
-#include "utils/rel.h"
PG_FUNCTION_INFO_V1(gin_metapage_info);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index 1cc3b8c9e90..029072567a4 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -10,10 +10,8 @@
#include "postgres.h"
#include "access/gist.h"
-#include "access/gist_private.h"
#include "access/htup.h"
#include "access/relation.h"
-#include "catalog/namespace.h"
#include "catalog/pg_am_d.h"
#include "funcapi.h"
#include "miscadmin.h"
@@ -25,7 +23,6 @@
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/ruleutils.h"
-#include "utils/varlena.h"
PG_FUNCTION_INFO_V1(gist_page_opaque_info);
PG_FUNCTION_INFO_V1(gist_page_items);
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 38a539dad1b..3dc705e43f7 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -32,7 +32,6 @@
#include "funcapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pageinspect.h"
#include "port/pg_bitutils.h"
#include "utils/array.h"
#include "utils/builtins.h"
diff --git a/contrib/pg_freespacemap/pg_freespacemap.c b/contrib/pg_freespacemap/pg_freespacemap.c
index b82cab2d97e..565167aaef0 100644
--- a/contrib/pg_freespacemap/pg_freespacemap.c
+++ b/contrib/pg_freespacemap/pg_freespacemap.c
@@ -9,7 +9,7 @@
#include "postgres.h"
#include "access/relation.h"
-#include "funcapi.h"
+#include "fmgr.h"
#include "storage/freespace.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index d061731706a..fac4051e1aa 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -30,8 +30,6 @@
#include "access/relation.h"
#include "access/xact.h"
-#include "catalog/pg_class.h"
-#include "catalog/pg_type.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -42,18 +40,13 @@
#include "storage/ipc.h"
#include "storage/latch.h"
#include "storage/lwlock.h"
-#include "storage/proc.h"
#include "storage/procsignal.h"
-#include "storage/shmem.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
-#include "utils/acl.h"
-#include "utils/datetime.h"
#include "utils/guc.h"
-#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/relfilenumbermap.h"
-#include "utils/resowner.h"
+#include "utils/timestamp.h"
#define AUTOPREWARM_FILE "autoprewarm.blocks"
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 21b26b7b6ec..1798e1d016f 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -49,7 +49,6 @@
#include "access/parallel.h"
#include "catalog/pg_authid.h"
-#include "common/hashfn.h"
#include "common/int.h"
#include "executor/instrument.h"
#include "funcapi.h"
@@ -59,9 +58,7 @@
#include "nodes/queryjumble.h"
#include "optimizer/planner.h"
#include "parser/analyze.h"
-#include "parser/parsetree.h"
#include "parser/scanner.h"
-#include "parser/scansup.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/ipc.h"
diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index 37dffe3f7d5..1513d76e4b5 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -12,11 +12,11 @@
*/
#include "postgres.h"
-#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
#include "catalog/pg_am_d.h"
-#include "catalog/pg_proc_d.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "utils/acl.h"
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 724122b1bc5..5d0deaba61e 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -19,12 +19,10 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
#include "storage/read_stream.h"
#include "storage/smgr.h"
#include "utils/rel.h"
-#include "utils/snapmgr.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pgcrypto/pgcrypto.c b/contrib/pgcrypto/pgcrypto.c
index 96447c57577..ebd76eed702 100644
--- a/contrib/pgcrypto/pgcrypto.c
+++ b/contrib/pgcrypto/pgcrypto.c
@@ -38,7 +38,6 @@
#include "px-crypt.h"
#include "px.h"
#include "utils/builtins.h"
-#include "utils/uuid.h"
#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pgrowlocks/pgrowlocks.c b/contrib/pgrowlocks/pgrowlocks.c
index adbc8279c3f..7e40ab21dda 100644
--- a/contrib/pgrowlocks/pgrowlocks.c
+++ b/contrib/pgrowlocks/pgrowlocks.c
@@ -37,7 +37,7 @@
#include "storage/bufmgr.h"
#include "storage/procarray.h"
#include "utils/acl.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
#include "utils/varlena.h"
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 04457f4b790..3a5aad30ae2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -14,21 +14,15 @@
#include "access/heapam.h"
#include "access/htup_details.h"
-#include "access/multixact.h"
#include "access/relation.h"
-#include "access/transam.h"
#include "access/visibilitymap.h"
-#include "access/xact.h"
-#include "catalog/namespace.h"
#include "catalog/pg_am_d.h"
#include "commands/vacuum.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/freespace.h"
-#include "storage/lmgr.h"
#include "storage/procarray.h"
-#include "utils/builtins.h"
PG_FUNCTION_INFO_V1(pgstattuple_approx);
PG_FUNCTION_INFO_V1(pgstattuple_approx_v1_5);
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 1b6b768cf80..4b9d76ec4e4 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -32,14 +32,11 @@
#include "access/htup_details.h"
#include "access/nbtree.h"
#include "access/relation.h"
-#include "access/table.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
-#include "storage/lmgr.h"
-#include "utils/builtins.h"
#include "utils/rel.h"
#include "utils/varlena.h"
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 1fff762753b..48cb8f59c4f 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -36,7 +36,6 @@
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/lmgr.h"
-#include "utils/builtins.h"
#include "utils/varlena.h"
PG_MODULE_MAGIC;
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 2e5303eac12..2326f391d34 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -16,7 +16,6 @@
#include <poll.h>
#endif
-#include "access/htup_details.h"
#include "access/xact.h"
#include "catalog/pg_user_mapping.h"
#include "commands/defrem.h"
@@ -27,13 +26,10 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postgres_fdw.h"
-#include "storage/fd.h"
#include "storage/latch.h"
#include "utils/builtins.h"
-#include "utils/datetime.h"
#include "utils/hsearch.h"
#include "utils/inval.h"
-#include "utils/memutils.h"
#include "utils/syscache.h"
/*
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index fb590c87e67..4680d517331 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -47,8 +47,6 @@
#include "catalog/pg_ts_dict.h"
#include "catalog/pg_type.h"
#include "commands/defrem.h"
-#include "commands/tablecmds.h"
-#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "nodes/plannodes.h"
#include "optimizer/optimizer.h"
diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c
index d740893918c..ed11126c09e 100644
--- a/contrib/postgres_fdw/option.c
+++ b/contrib/postgres_fdw/option.c
@@ -20,7 +20,6 @@
#include "commands/extension.h"
#include "libpq/libpq-be.h"
#include "postgres_fdw.h"
-#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/varlena.h"
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index adc62576d1f..53733d642d0 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -17,11 +17,9 @@
#include "access/htup_details.h"
#include "access/sysattr.h"
#include "access/table.h"
-#include "catalog/pg_class.h"
#include "catalog/pg_opfamily.h"
#include "commands/defrem.h"
#include "commands/explain.h"
-#include "commands/vacuum.h"
#include "executor/execAsync.h"
#include "foreign/fdwapi.h"
#include "funcapi.h"
@@ -29,7 +27,6 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/appendinfo.h"
-#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/inherit.h"
#include "optimizer/optimizer.h"
diff --git a/contrib/spi/moddatetime.c b/contrib/spi/moddatetime.c
index 3eb7004de97..5130804ce2a 100644
--- a/contrib/spi/moddatetime.c
+++ b/contrib/spi/moddatetime.c
@@ -19,7 +19,7 @@ OH, me, I'm Terry Mackintosh <[email protected]>
#include "catalog/pg_type.h"
#include "commands/trigger.h"
#include "executor/spi.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
PG_MODULE_MAGIC;
diff --git a/contrib/tablefunc/tablefunc.c b/contrib/tablefunc/tablefunc.c
index 2a25607a2ad..d41b353a6de 100644
--- a/contrib/tablefunc/tablefunc.c
+++ b/contrib/tablefunc/tablefunc.c
@@ -38,6 +38,7 @@
#include "catalog/pg_type.h"
#include "common/pg_prng.h"
#include "executor/spi.h"
+#include "fmgr.h"
#include "funcapi.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
diff --git a/contrib/tablefunc/tablefunc.h b/contrib/tablefunc/tablefunc.h
index 2009382ce7d..b78030044b5 100644
--- a/contrib/tablefunc/tablefunc.h
+++ b/contrib/tablefunc/tablefunc.h
@@ -34,6 +34,4 @@
#ifndef TABLEFUNC_H
#define TABLEFUNC_H
-#include "fmgr.h"
-
#endif /* TABLEFUNC_H */
diff --git a/contrib/tsm_system_rows/tsm_system_rows.c b/contrib/tsm_system_rows/tsm_system_rows.c
index a9f5d657d1c..ecfd1102d7a 100644
--- a/contrib/tsm_system_rows/tsm_system_rows.c
+++ b/contrib/tsm_system_rows/tsm_system_rows.c
@@ -28,7 +28,6 @@
#include "postgres.h"
-#include "access/relscan.h"
#include "access/tsmapi.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
diff --git a/contrib/tsm_system_time/tsm_system_time.c b/contrib/tsm_system_time/tsm_system_time.c
index f187fd77655..093682a9155 100644
--- a/contrib/tsm_system_time/tsm_system_time.c
+++ b/contrib/tsm_system_time/tsm_system_time.c
@@ -26,7 +26,6 @@
#include <math.h>
-#include "access/relscan.h"
#include "access/tsmapi.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
diff --git a/contrib/unaccent/unaccent.c b/contrib/unaccent/unaccent.c
index 707962305f8..0217696aac1 100644
--- a/contrib/unaccent/unaccent.c
+++ b/contrib/unaccent/unaccent.c
@@ -13,7 +13,6 @@
#include "postgres.h"
-#include "catalog/namespace.h"
#include "catalog/pg_ts_dict.h"
#include "commands/defrem.h"
#include "lib/stringinfo.h"
@@ -22,7 +21,6 @@
#include "tsearch/ts_public.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-#include "utils/regproc.h"
#include "utils/syscache.h"
PG_MODULE_MAGIC;
diff --git a/contrib/xml2/xpath.c b/contrib/xml2/xpath.c
index ef78aa00c88..f7e3f485fe1 100644
--- a/contrib/xml2/xpath.c
+++ b/contrib/xml2/xpath.c
@@ -11,7 +11,6 @@
#include "fmgr.h"
#include "funcapi.h"
#include "lib/stringinfo.h"
-#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/xml.h"
diff --git a/contrib/xml2/xslt_proc.c b/contrib/xml2/xslt_proc.c
index e761ca5cb5f..b720d89f754 100644
--- a/contrib/xml2/xslt_proc.c
+++ b/contrib/xml2/xslt_proc.c
@@ -7,10 +7,7 @@
*/
#include "postgres.h"
-#include "executor/spi.h"
#include "fmgr.h"
-#include "funcapi.h"
-#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/xml.h"
--
2.47.0
From 4a547fed9b6c09b396861f195e8fefff76040c84 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 21 Sep 2024 22:25:52 +0200
Subject: [PATCH 3/5] Remove unused #include's from pl .c files
as determined by IWYU
---
src/pl/plperl/plperl.c | 1 -
src/pl/plpgsql/src/pl_comp.c | 3 +--
src/pl/plpgsql/src/pl_exec.c | 5 -----
src/pl/plpython/plpy_cursorobject.c | 2 --
src/pl/plpython/plpy_exec.c | 4 +---
src/pl/plpython/plpy_planobject.c | 1 -
src/pl/plpython/plpy_plpymodule.c | 3 ---
src/pl/plpython/plpy_procedure.c | 3 ---
src/pl/plpython/plpy_spi.c | 3 ---
src/pl/plpython/plpy_util.c | 1 -
src/pl/tcl/pltcl.c | 3 +--
11 files changed, 3 insertions(+), 26 deletions(-)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index fe719935c67..1b1677e333b 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -24,7 +24,6 @@
#include "executor/spi.h"
#include "funcapi.h"
#include "miscadmin.h"
-#include "nodes/makefuncs.h"
#include "parser/parse_type.h"
#include "storage/ipc.h"
#include "tcop/tcopprot.h"
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index f1bce708d62..5633e3c7905 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -23,7 +23,7 @@
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "nodes/makefuncs.h"
-#include "parser/parse_type.h"
+#include "parser/parse_node.h"
#include "plpgsql.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
@@ -31,7 +31,6 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/regproc.h"
-#include "utils/rel.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index ea9740e3f89..b51b3d13a08 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -19,11 +19,9 @@
#include "access/detoast.h"
#include "access/htup_details.h"
-#include "access/transam.h"
#include "access/tupconvert.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
-#include "commands/defrem.h"
#include "executor/execExpr.h"
#include "executor/spi.h"
#include "executor/tstoreReceiver.h"
@@ -34,13 +32,10 @@
#include "optimizer/optimizer.h"
#include "parser/parse_coerce.h"
#include "parser/parse_type.h"
-#include "parser/scansup.h"
#include "plpgsql.h"
#include "storage/proc.h"
#include "tcop/cmdtag.h"
#include "tcop/pquery.h"
-#include "tcop/tcopprot.h"
-#include "tcop/utility.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 24f2ac8c46b..6108384c9a5 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -8,14 +8,12 @@
#include <limits.h>
-#include "access/xact.h"
#include "catalog/pg_type.h"
#include "mb/pg_wchar.h"
#include "plpy_cursorobject.h"
#include "plpy_elog.h"
#include "plpy_main.h"
#include "plpy_planobject.h"
-#include "plpy_procedure.h"
#include "plpy_resultobject.h"
#include "plpy_spi.h"
#include "plpython.h"
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 157229e96f6..9d59473d6ad 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -18,10 +18,8 @@
#include "plpy_procedure.h"
#include "plpy_subxactobject.h"
#include "plpython.h"
-#include "utils/builtins.h"
-#include "utils/lsyscache.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
-#include "utils/typcache.h"
/* saved state for a set-returning function */
typedef struct PLySRFState
diff --git a/src/pl/plpython/plpy_planobject.c b/src/pl/plpython/plpy_planobject.c
index ec2439c6a1f..bbef889329e 100644
--- a/src/pl/plpython/plpy_planobject.c
+++ b/src/pl/plpython/plpy_planobject.c
@@ -7,7 +7,6 @@
#include "postgres.h"
#include "plpy_cursorobject.h"
-#include "plpy_elog.h"
#include "plpy_planobject.h"
#include "plpy_spi.h"
#include "plpython.h"
diff --git a/src/pl/plpython/plpy_plpymodule.c b/src/pl/plpython/plpy_plpymodule.c
index fa08f0dbfb3..ea06d9a52b1 100644
--- a/src/pl/plpython/plpy_plpymodule.c
+++ b/src/pl/plpython/plpy_plpymodule.c
@@ -6,11 +6,9 @@
#include "postgres.h"
-#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "plpy_cursorobject.h"
#include "plpy_elog.h"
-#include "plpy_main.h"
#include "plpy_planobject.h"
#include "plpy_plpymodule.h"
#include "plpy_resultobject.h"
@@ -18,7 +16,6 @@
#include "plpy_subxactobject.h"
#include "plpython.h"
#include "utils/builtins.h"
-#include "utils/snapmgr.h"
HTAB *PLy_spi_exceptions = NULL;
diff --git a/src/pl/plpython/plpy_procedure.c b/src/pl/plpython/plpy_procedure.c
index ba7786d31c3..c35a3b801ab 100644
--- a/src/pl/plpython/plpy_procedure.c
+++ b/src/pl/plpython/plpy_procedure.c
@@ -7,7 +7,6 @@
#include "postgres.h"
#include "access/htup_details.h"
-#include "access/transam.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
@@ -17,8 +16,6 @@
#include "plpython.h"
#include "utils/builtins.h"
#include "utils/hsearch.h"
-#include "utils/inval.h"
-#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/syscache.h"
diff --git a/src/pl/plpython/plpy_spi.c b/src/pl/plpython/plpy_spi.c
index ff87b27de09..bcbd07b70ae 100644
--- a/src/pl/plpython/plpy_spi.c
+++ b/src/pl/plpython/plpy_spi.c
@@ -8,7 +8,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "executor/spi.h"
@@ -18,12 +17,10 @@
#include "plpy_main.h"
#include "plpy_planobject.h"
#include "plpy_plpymodule.h"
-#include "plpy_procedure.h"
#include "plpy_resultobject.h"
#include "plpy_spi.h"
#include "plpython.h"
#include "utils/memutils.h"
-#include "utils/syscache.h"
static PyObject *PLy_spi_execute_query(char *query, long limit);
static PyObject *PLy_spi_execute_fetch_result(SPITupleTable *tuptable,
diff --git a/src/pl/plpython/plpy_util.c b/src/pl/plpython/plpy_util.c
index 22e2a599ad9..6d89b1cb60a 100644
--- a/src/pl/plpython/plpy_util.c
+++ b/src/pl/plpython/plpy_util.c
@@ -10,7 +10,6 @@
#include "plpy_elog.h"
#include "plpy_util.h"
#include "plpython.h"
-#include "utils/memutils.h"
/*
* Convert a Python unicode object to a Python string/bytes object in
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e2ccaa84f3f..80de0db40de 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -25,13 +25,12 @@
#include "funcapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/makefuncs.h"
#include "parser/parse_func.h"
#include "parser/parse_type.h"
#include "pgstat.h"
-#include "tcop/tcopprot.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/guc.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/regproc.h"
--
2.47.0
From 95c5e4eef4e914db1d44a80ca56733b08fd59054 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 21 Sep 2024 22:24:01 +0200
Subject: [PATCH 4/5] Remove unused #include's from src/test/ .c files
as determined by IWYU
---
src/test/modules/delay_execution/delay_execution.c | 2 +-
src/test/modules/dummy_index_am/dummy_index_am.c | 2 --
src/test/modules/ldap_password_func/ldap_password_func.c | 4 +---
src/test/modules/libpq_pipeline/libpq_pipeline.c | 2 --
src/test/modules/plsample/plsample.c | 2 +-
src/test/modules/test_ddl_deparse/test_ddl_deparse.c | 1 -
src/test/modules/test_integerset/test_integerset.c | 4 ----
src/test/modules/test_oat_hooks/test_oat_hooks.c | 1 -
src/test/modules/test_radixtree/test_radixtree.c | 2 --
src/test/modules/test_regex/test_regex.c | 1 -
src/test/modules/test_resowner/test_resowner_basic.c | 2 --
src/test/modules/test_resowner/test_resowner_many.c | 1 -
src/test/modules/test_rls_hooks/test_rls_hooks.c | 1 -
src/test/modules/test_shm_mq/setup.c | 1 -
src/test/modules/test_slru/test_multixact.c | 2 +-
src/test/modules/test_tidstore/test_tidstore.c | 1 -
src/test/modules/worker_spi/worker_spi.c | 4 ----
src/test/regress/pg_regress.c | 1 -
src/test/regress/regress.c | 3 ---
19 files changed, 4 insertions(+), 33 deletions(-)
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 155c8a8d556..fa4693a3f57 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -23,7 +23,7 @@
#include <limits.h>
#include "optimizer/planner.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/guc.h"
#include "utils/inval.h"
diff --git a/src/test/modules/dummy_index_am/dummy_index_am.c b/src/test/modules/dummy_index_am/dummy_index_am.c
index 2841cf2eb4b..beb2c1d2542 100644
--- a/src/test/modules/dummy_index_am/dummy_index_am.c
+++ b/src/test/modules/dummy_index_am/dummy_index_am.c
@@ -18,8 +18,6 @@
#include "catalog/index.h"
#include "commands/vacuum.h"
#include "nodes/pathnodes.h"
-#include "utils/guc.h"
-#include "utils/rel.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/ldap_password_func/ldap_password_func.c b/src/test/modules/ldap_password_func/ldap_password_func.c
index 24d9c637818..84057d955aa 100644
--- a/src/test/modules/ldap_password_func/ldap_password_func.c
+++ b/src/test/modules/ldap_password_func/ldap_password_func.c
@@ -15,10 +15,8 @@
#include <float.h>
#include <stdio.h>
+#include "fmgr.h"
#include "libpq/auth.h"
-#include "libpq/libpq.h"
-#include "libpq/libpq-be.h"
-#include "utils/guc.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 999a7f57a7e..1323e4c598d 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -19,10 +19,8 @@
#include <sys/time.h>
#include "catalog/pg_type_d.h"
-#include "common/fe_memutils.h"
#include "libpq-fe.h"
#include "pg_getopt.h"
-#include "portability/instr_time.h"
static void exit_nicely(PGconn *conn);
diff --git a/src/test/modules/plsample/plsample.c b/src/test/modules/plsample/plsample.c
index 89ea166a676..25f7814f1f7 100644
--- a/src/test/modules/plsample/plsample.c
+++ b/src/test/modules/plsample/plsample.c
@@ -21,7 +21,7 @@
#include "commands/trigger.h"
#include "executor/spi.h"
#include "funcapi.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
#include "utils/syscache.h"
diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c
index 2758ae82d7b..97cf52d133c 100644
--- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c
+++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c
@@ -10,7 +10,6 @@
*/
#include "postgres.h"
-#include "catalog/pg_type.h"
#include "funcapi.h"
#include "nodes/execnodes.h"
#include "tcop/deparse_utility.h"
diff --git a/src/test/modules/test_integerset/test_integerset.c b/src/test/modules/test_integerset/test_integerset.c
index 61f8d0b663a..73a97b81167 100644
--- a/src/test/modules/test_integerset/test_integerset.c
+++ b/src/test/modules/test_integerset/test_integerset.c
@@ -15,10 +15,6 @@
#include "common/pg_prng.h"
#include "fmgr.h"
#include "lib/integerset.h"
-#include "miscadmin.h"
-#include "nodes/bitmapset.h"
-#include "storage/block.h"
-#include "storage/itemptr.h"
#include "utils/memutils.h"
#include "utils/timestamp.h"
diff --git a/src/test/modules/test_oat_hooks/test_oat_hooks.c b/src/test/modules/test_oat_hooks/test_oat_hooks.c
index 05c33712254..ca4aa7e6696 100644
--- a/src/test/modules/test_oat_hooks/test_oat_hooks.c
+++ b/src/test/modules/test_oat_hooks/test_oat_hooks.c
@@ -16,7 +16,6 @@
#include "access/parallel.h"
#include "catalog/dependency.h"
#include "catalog/objectaccess.h"
-#include "catalog/pg_proc.h"
#include "executor/executor.h"
#include "fmgr.h"
#include "miscadmin.h"
diff --git a/src/test/modules/test_radixtree/test_radixtree.c b/src/test/modules/test_radixtree/test_radixtree.c
index 1d9165a3a23..3e072fa5ec6 100644
--- a/src/test/modules/test_radixtree/test_radixtree.c
+++ b/src/test/modules/test_radixtree/test_radixtree.c
@@ -15,8 +15,6 @@
#include "common/int.h"
#include "common/pg_prng.h"
#include "fmgr.h"
-#include "miscadmin.h"
-#include "storage/lwlock.h"
#include "utils/memutils.h"
#include "utils/timestamp.h"
diff --git a/src/test/modules/test_regex/test_regex.c b/src/test/modules/test_regex/test_regex.c
index c4ef15d7425..cd503a2cd81 100644
--- a/src/test/modules/test_regex/test_regex.c
+++ b/src/test/modules/test_regex/test_regex.c
@@ -15,7 +15,6 @@
#include "postgres.h"
#include "funcapi.h"
-#include "miscadmin.h"
#include "regex/regex.h"
#include "utils/array.h"
#include "utils/builtins.h"
diff --git a/src/test/modules/test_resowner/test_resowner_basic.c b/src/test/modules/test_resowner/test_resowner_basic.c
index 558cf1de56e..1d9c1cd0259 100644
--- a/src/test/modules/test_resowner/test_resowner_basic.c
+++ b/src/test/modules/test_resowner/test_resowner_basic.c
@@ -13,8 +13,6 @@
#include "postgres.h"
#include "fmgr.h"
-#include "lib/ilist.h"
-#include "utils/memutils.h"
#include "utils/resowner.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/test_resowner/test_resowner_many.c b/src/test/modules/test_resowner/test_resowner_many.c
index 7808c9e5df3..4f3f5ebf87e 100644
--- a/src/test/modules/test_resowner/test_resowner_many.c
+++ b/src/test/modules/test_resowner/test_resowner_many.c
@@ -14,7 +14,6 @@
#include "fmgr.h"
#include "lib/ilist.h"
-#include "utils/memutils.h"
#include "utils/resowner.h"
/*
diff --git a/src/test/modules/test_rls_hooks/test_rls_hooks.c b/src/test/modules/test_rls_hooks/test_rls_hooks.c
index 358e28edb45..d18d5815cfe 100644
--- a/src/test/modules/test_rls_hooks/test_rls_hooks.c
+++ b/src/test/modules/test_rls_hooks/test_rls_hooks.c
@@ -15,7 +15,6 @@
#include "catalog/pg_type.h"
#include "fmgr.h"
-#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
diff --git a/src/test/modules/test_shm_mq/setup.c b/src/test/modules/test_shm_mq/setup.c
index b3dac44d97a..fb235604394 100644
--- a/src/test/modules/test_shm_mq/setup.c
+++ b/src/test/modules/test_shm_mq/setup.c
@@ -18,7 +18,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
-#include "storage/procsignal.h"
#include "storage/shm_toc.h"
#include "test_shm_mq.h"
#include "utils/memutils.h"
diff --git a/src/test/modules/test_slru/test_multixact.c b/src/test/modules/test_slru/test_multixact.c
index 22cc441a385..1cc89aa116c 100644
--- a/src/test/modules/test_slru/test_multixact.c
+++ b/src/test/modules/test_slru/test_multixact.c
@@ -16,7 +16,7 @@
#include "access/multixact.h"
#include "access/xact.h"
-#include "utils/builtins.h"
+#include "fmgr.h"
#include "utils/injection_point.h"
PG_FUNCTION_INFO_V1(test_create_multixact);
diff --git a/src/test/modules/test_tidstore/test_tidstore.c b/src/test/modules/test_tidstore/test_tidstore.c
index 94ddcf1de82..6da1cb24527 100644
--- a/src/test/modules/test_tidstore/test_tidstore.c
+++ b/src/test/modules/test_tidstore/test_tidstore.c
@@ -18,7 +18,6 @@
#include "access/tidstore.h"
#include "fmgr.h"
-#include "funcapi.h"
#include "storage/block.h"
#include "storage/itemptr.h"
#include "storage/lwlock.h"
diff --git a/src/test/modules/worker_spi/worker_spi.c b/src/test/modules/worker_spi/worker_spi.c
index de8f46902b4..d4403b24d98 100644
--- a/src/test/modules/worker_spi/worker_spi.c
+++ b/src/test/modules/worker_spi/worker_spi.c
@@ -26,11 +26,7 @@
#include "miscadmin.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
-#include "storage/ipc.h"
#include "storage/latch.h"
-#include "storage/lwlock.h"
-#include "storage/proc.h"
-#include "storage/shmem.h"
/* these headers are used by this particular worker's code */
#include "access/xact.h"
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 6c188954b14..0e40ed32a21 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -28,7 +28,6 @@
#include "common/logging.h"
#include "common/restricted_token.h"
-#include "common/string.h"
#include "common/username.h"
#include "getopt_long.h"
#include "lib/stringinfo.h"
diff --git a/src/test/regress/regress.c b/src/test/regress/regress.c
index 9e81371be4b..8bae56a9777 100644
--- a/src/test/regress/regress.c
+++ b/src/test/regress/regress.c
@@ -21,8 +21,6 @@
#include "access/detoast.h"
#include "access/htup_details.h"
-#include "access/transam.h"
-#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
@@ -39,7 +37,6 @@
#include "parser/parse_coerce.h"
#include "port/atomics.h"
#include "storage/spin.h"
-#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/geo_decls.h"
#include "utils/memutils.h"
--
2.47.0
From c0133bb5b7573d0ec4d94b2f5f3d04e0261329ac Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 22 Jun 2024 10:27:40 +0200
Subject: [PATCH 5/5] Remove unused #include's from bin .c files
as determined by IWYU
---
src/bin/pg_basebackup/pg_createsubscriber.c | 3 +--
src/bin/pg_basebackup/pg_recvlogical.c | 2 --
src/bin/pg_basebackup/receivelog.c | 1 -
src/bin/pg_basebackup/streamutil.c | 2 --
src/bin/pg_basebackup/walmethods.c | 4 ++--
src/bin/pg_checksums/pg_checksums.c | 1 -
src/bin/pg_combinebackup/copy_file.h | 1 -
src/bin/pg_combinebackup/pg_combinebackup.c | 2 +-
src/bin/pg_config/pg_config.c | 1 -
src/bin/pg_dump/common.c | 2 --
src/bin/pg_dump/compress_io.c | 1 -
src/bin/pg_dump/filter.c | 1 -
src/bin/pg_dump/parallel.c | 2 ++
src/bin/pg_dump/pg_backup_custom.c | 1 -
src/bin/pg_dump/pg_backup_db.c | 2 --
src/bin/pg_dump/pg_backup_utils.c | 2 ++
src/bin/pg_dump/pg_dump.c | 2 --
src/bin/pg_dump/pg_dump_sort.c | 1 -
src/bin/pg_dump/pg_restore.c | 1 -
src/bin/pg_rewind/datapagemap.h | 1 -
src/bin/pg_rewind/libpq_source.c | 1 -
src/bin/pg_rewind/local_source.c | 4 +---
src/bin/pg_rewind/pg_rewind.h | 4 ----
src/bin/pg_test_fsync/pg_test_fsync.c | 1 -
src/bin/pg_upgrade/check.c | 2 --
src/bin/pg_upgrade/relfilenumber.c | 2 --
src/bin/pg_walsummary/pg_walsummary.c | 1 -
src/bin/psql/command.c | 3 ---
src/bin/psql/describe.c | 1 -
src/bin/psql/help.c | 3 ---
src/bin/scripts/clusterdb.c | 1 -
src/bin/scripts/common.c | 1 -
src/bin/scripts/reindexdb.c | 1 -
src/common/controldata_utils.c | 2 +-
src/common/string.c | 1 -
src/common/unicode_case.c | 1 -
src/fe_utils/astreamer_file.c | 1 -
src/fe_utils/astreamer_gzip.c | 2 --
src/fe_utils/astreamer_lz4.c | 2 --
src/fe_utils/cancel.c | 2 --
40 files changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index e804b2a63ef..e96370a9ec1 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -13,14 +13,13 @@
#include "postgres_fe.h"
+#include <sys/stat.h>
#include <sys/time.h>
#include <sys/wait.h>
#include <time.h>
-#include "catalog/pg_authid_d.h"
#include "common/connect.h"
#include "common/controldata_utils.h"
-#include "common/file_perm.h"
#include "common/logging.h"
#include "common/pg_prng.h"
#include "common/restricted_token.h"
diff --git a/src/bin/pg_basebackup/pg_recvlogical.c b/src/bin/pg_basebackup/pg_recvlogical.c
index 3db520ed38b..42b2a7bb9d8 100644
--- a/src/bin/pg_basebackup/pg_recvlogical.c
+++ b/src/bin/pg_basebackup/pg_recvlogical.c
@@ -18,8 +18,6 @@
#include <sys/stat.h>
#include <unistd.h>
-#include "access/xlog_internal.h"
-#include "common/fe_memutils.h"
#include "common/file_perm.h"
#include "common/logging.h"
#include "fe_utils/option_utils.h"
diff --git a/src/bin/pg_basebackup/receivelog.c b/src/bin/pg_basebackup/receivelog.c
index 70f42467649..9fe90cb38fd 100644
--- a/src/bin/pg_basebackup/receivelog.c
+++ b/src/bin/pg_basebackup/receivelog.c
@@ -19,7 +19,6 @@
#include <unistd.h>
#include "access/xlog_internal.h"
-#include "common/file_utils.h"
#include "common/logging.h"
#include "libpq-fe.h"
#include "receivelog.h"
diff --git a/src/bin/pg_basebackup/streamutil.c b/src/bin/pg_basebackup/streamutil.c
index 30b3d9a377e..3f4d157d583 100644
--- a/src/bin/pg_basebackup/streamutil.c
+++ b/src/bin/pg_basebackup/streamutil.c
@@ -19,14 +19,12 @@
#include "access/xlog_internal.h"
#include "common/connect.h"
-#include "common/fe_memutils.h"
#include "common/file_perm.h"
#include "common/logging.h"
#include "common/string.h"
#include "datatype/timestamp.h"
#include "port/pg_bswap.h"
#include "pqexpbuffer.h"
-#include "receivelog.h"
#include "streamutil.h"
#define ERRCODE_DUPLICATE_OBJECT "42710"
diff --git a/src/bin/pg_basebackup/walmethods.c b/src/bin/pg_basebackup/walmethods.c
index 832aafd2973..215b24597fe 100644
--- a/src/bin/pg_basebackup/walmethods.c
+++ b/src/bin/pg_basebackup/walmethods.c
@@ -11,6 +11,7 @@
#include "postgres_fe.h"
+#include <fcntl.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
@@ -26,8 +27,7 @@
#include "common/file_utils.h"
#include "common/logging.h"
#include "pgtar.h"
-#include "receivelog.h"
-#include "streamutil.h"
+#include "walmethods.h"
/* Size of zlib buffer for .tar.gz */
#define ZLIB_OUT_SIZE 4096
diff --git a/src/bin/pg_checksums/pg_checksums.c b/src/bin/pg_checksums/pg_checksums.c
index f5f7ff10455..b86bc417c9c 100644
--- a/src/bin/pg_checksums/pg_checksums.c
+++ b/src/bin/pg_checksums/pg_checksums.c
@@ -21,7 +21,6 @@
#include <unistd.h>
#include "common/controldata_utils.h"
-#include "common/file_perm.h"
#include "common/file_utils.h"
#include "common/logging.h"
#include "common/relpath.h"
diff --git a/src/bin/pg_combinebackup/copy_file.h b/src/bin/pg_combinebackup/copy_file.h
index cedb7607385..d5ddc7e53d7 100644
--- a/src/bin/pg_combinebackup/copy_file.h
+++ b/src/bin/pg_combinebackup/copy_file.h
@@ -13,7 +13,6 @@
#include "c.h"
#include "common/checksum_helper.h"
-#include "common/file_utils.h"
/*
* Enumeration to denote copy modes.
diff --git a/src/bin/pg_combinebackup/pg_combinebackup.c b/src/bin/pg_combinebackup/pg_combinebackup.c
index 6183d317151..5f1f62f1db6 100644
--- a/src/bin/pg_combinebackup/pg_combinebackup.c
+++ b/src/bin/pg_combinebackup/pg_combinebackup.c
@@ -25,12 +25,12 @@
#endif
#include "backup_label.h"
-#include "common/blkreftable.h"
#include "common/checksum_helper.h"
#include "common/controldata_utils.h"
#include "common/file_perm.h"
#include "common/file_utils.h"
#include "common/logging.h"
+#include "common/relpath.h"
#include "copy_file.h"
#include "fe_utils/option_utils.h"
#include "getopt_long.h"
diff --git a/src/bin/pg_config/pg_config.c b/src/bin/pg_config/pg_config.c
index 77d09ccfc47..504e6c58009 100644
--- a/src/bin/pg_config/pg_config.c
+++ b/src/bin/pg_config/pg_config.c
@@ -25,7 +25,6 @@
#include "postgres_fe.h"
#include "common/config_info.h"
-#include "port.h"
static const char *progname;
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index c323b5bd3d9..9b2d34e2815 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -27,8 +27,6 @@
#include "catalog/pg_subscription_d.h"
#include "catalog/pg_type_d.h"
#include "common/hashfn.h"
-#include "fe_utils/string_utils.h"
-#include "pg_backup_archiver.h"
#include "pg_backup_utils.h"
#include "pg_dump.h"
diff --git a/src/bin/pg_dump/compress_io.c b/src/bin/pg_dump/compress_io.c
index e2edf8bf4f6..47a13099ccf 100644
--- a/src/bin/pg_dump/compress_io.c
+++ b/src/bin/pg_dump/compress_io.c
@@ -70,7 +70,6 @@
#include "compress_lz4.h"
#include "compress_none.h"
#include "compress_zstd.h"
-#include "pg_backup_utils.h"
/*----------------------
* Generic functions
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index 5815cd23748..9dc6d22fcda 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -13,7 +13,6 @@
*/
#include "postgres_fe.h"
-#include "common/fe_memutils.h"
#include "common/logging.h"
#include "common/string.h"
#include "filter.h"
diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c
index a09247fae47..78e91f6e2dc 100644
--- a/src/bin/pg_dump/parallel.c
+++ b/src/bin/pg_dump/parallel.c
@@ -63,7 +63,9 @@
#include "fe_utils/string_utils.h"
#include "parallel.h"
#include "pg_backup_utils.h"
+#ifdef WIN32
#include "port/pg_bswap.h"
+#endif
/* Mnemonic macros for indexing the fd array returned by pipe(2) */
#define PIPE_READ 0
diff --git a/src/bin/pg_dump/pg_backup_custom.c b/src/bin/pg_dump/pg_backup_custom.c
index 55107b20058..ecaad7321a3 100644
--- a/src/bin/pg_dump/pg_backup_custom.c
+++ b/src/bin/pg_dump/pg_backup_custom.c
@@ -27,7 +27,6 @@
#include "common/file_utils.h"
#include "compress_io.h"
-#include "parallel.h"
#include "pg_backup_utils.h"
/*--------
diff --git a/src/bin/pg_dump/pg_backup_db.c b/src/bin/pg_dump/pg_backup_db.c
index 4d1957a7eb3..71c55d2466a 100644
--- a/src/bin/pg_dump/pg_backup_db.c
+++ b/src/bin/pg_dump/pg_backup_db.c
@@ -19,8 +19,6 @@
#include "common/connect.h"
#include "common/string.h"
-#include "dumputils.h"
-#include "fe_utils/string_utils.h"
#include "parallel.h"
#include "pg_backup_archiver.h"
#include "pg_backup_db.h"
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index a0045cf5e58..80715979a1a 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -13,7 +13,9 @@
*/
#include "postgres_fe.h"
+#ifdef WIN32
#include "parallel.h"
+#endif
#include "pg_backup_utils.h"
/* Globals exported by this file */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ced..ea759c5f78a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -49,10 +49,8 @@
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
-#include "catalog/pg_largeobject_metadata_d.h"
#include "catalog/pg_proc_d.h"
#include "catalog/pg_subscription.h"
-#include "catalog/pg_trigger_d.h"
#include "catalog/pg_type_d.h"
#include "common/connect.h"
#include "common/int.h"
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 4cb754caa55..d88ce06ff46 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -18,7 +18,6 @@
#include "catalog/pg_class_d.h"
#include "common/int.h"
#include "lib/binaryheap.h"
-#include "pg_backup_archiver.h"
#include "pg_backup_utils.h"
#include "pg_dump.h"
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index df119591cca..f2c1020d053 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -45,7 +45,6 @@
#include <termios.h>
#endif
-#include "dumputils.h"
#include "fe_utils/option_utils.h"
#include "filter.h"
#include "getopt_long.h"
diff --git a/src/bin/pg_rewind/datapagemap.h b/src/bin/pg_rewind/datapagemap.h
index 49124834508..be59ea23fc5 100644
--- a/src/bin/pg_rewind/datapagemap.h
+++ b/src/bin/pg_rewind/datapagemap.h
@@ -10,7 +10,6 @@
#define DATAPAGEMAP_H
#include "storage/block.h"
-#include "storage/relfilelocator.h"
struct datapagemap
{
diff --git a/src/bin/pg_rewind/libpq_source.c b/src/bin/pg_rewind/libpq_source.c
index 0239fb47644..62610a4efaf 100644
--- a/src/bin/pg_rewind/libpq_source.c
+++ b/src/bin/pg_rewind/libpq_source.c
@@ -11,7 +11,6 @@
#include "catalog/pg_type_d.h"
#include "common/connect.h"
-#include "datapagemap.h"
#include "file_ops.h"
#include "filemap.h"
#include "lib/stringinfo.h"
diff --git a/src/bin/pg_rewind/local_source.c b/src/bin/pg_rewind/local_source.c
index 46c6b72bd71..4aab2ba0ac9 100644
--- a/src/bin/pg_rewind/local_source.c
+++ b/src/bin/pg_rewind/local_source.c
@@ -12,10 +12,8 @@
#include <fcntl.h>
#include <unistd.h>
-#include "datapagemap.h"
+#include "common/logging.h"
#include "file_ops.h"
-#include "filemap.h"
-#include "pg_rewind.h"
#include "rewind_source.h"
typedef struct
diff --git a/src/bin/pg_rewind/pg_rewind.h b/src/bin/pg_rewind/pg_rewind.h
index ec43cbe2c67..d937542c6ec 100644
--- a/src/bin/pg_rewind/pg_rewind.h
+++ b/src/bin/pg_rewind/pg_rewind.h
@@ -14,10 +14,6 @@
#include "access/timeline.h"
#include "common/logging.h"
#include "common/file_utils.h"
-#include "datapagemap.h"
-#include "libpq-fe.h"
-#include "storage/block.h"
-#include "storage/relfilelocator.h"
/* Configuration options */
extern char *datadir_target;
diff --git a/src/bin/pg_test_fsync/pg_test_fsync.c b/src/bin/pg_test_fsync/pg_test_fsync.c
index cbf587116ea..65d05136f89 100644
--- a/src/bin/pg_test_fsync/pg_test_fsync.c
+++ b/src/bin/pg_test_fsync/pg_test_fsync.c
@@ -19,7 +19,6 @@
#include <unistd.h>
#include <signal.h>
-#include "access/xlogdefs.h"
#include "common/logging.h"
#include "common/pg_prng.h"
#include "getopt_long.h"
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 12735a42687..cba5031af87 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -11,9 +11,7 @@
#include "catalog/pg_authid_d.h"
#include "catalog/pg_class_d.h"
-#include "catalog/pg_collation.h"
#include "fe_utils/string_utils.h"
-#include "mb/pg_wchar.h"
#include "pg_upgrade.h"
static void check_new_cluster_is_empty(void);
diff --git a/src/bin/pg_upgrade/relfilenumber.c b/src/bin/pg_upgrade/relfilenumber.c
index 1d3054d78bd..07baa49a025 100644
--- a/src/bin/pg_upgrade/relfilenumber.c
+++ b/src/bin/pg_upgrade/relfilenumber.c
@@ -11,8 +11,6 @@
#include <sys/stat.h>
-#include "access/transam.h"
-#include "catalog/pg_class_d.h"
#include "pg_upgrade.h"
static void transfer_single_new_db(FileNameMap *maps, int size, char *old_tablespace);
diff --git a/src/bin/pg_walsummary/pg_walsummary.c b/src/bin/pg_walsummary/pg_walsummary.c
index f6a262d3184..267174d07d0 100644
--- a/src/bin/pg_walsummary/pg_walsummary.c
+++ b/src/bin/pg_walsummary/pg_walsummary.c
@@ -20,7 +20,6 @@
#include "common/logging.h"
#include "fe_utils/option_utils.h"
#include "getopt_long.h"
-#include "lib/stringinfo.h"
typedef struct ws_options
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 328d78c73f9..1f3cbb11f7c 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -30,7 +30,6 @@
#include "common/logging.h"
#include "common/string.h"
#include "copy.h"
-#include "crosstabview.h"
#include "describe.h"
#include "fe_utils/cancel.h"
#include "fe_utils/print.h"
@@ -38,10 +37,8 @@
#include "help.h"
#include "input.h"
#include "large_obj.h"
-#include "libpq-fe.h"
#include "libpq/pqcomm.h"
#include "mainloop.h"
-#include "portability/instr_time.h"
#include "pqexpbuffer.h"
#include "psqlscanslash.h"
#include "settings.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 363a66e7185..37b43fb1221 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -27,7 +27,6 @@
#include "fe_utils/print.h"
#include "fe_utils/string_utils.h"
#include "settings.h"
-#include "variables.h"
static const char *map_typename_pattern(const char *pattern);
static bool describeOneTableDetails(const char *schemaname,
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index 19d20c58781..02fe5d151e0 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -21,9 +21,6 @@
#include <termios.h>
#endif
-#include "common.h"
-#include "common/logging.h"
-#include "common/username.h"
#include "help.h"
#include "input.h"
#include "settings.h"
diff --git a/src/bin/scripts/clusterdb.c b/src/bin/scripts/clusterdb.c
index 3503a3bb584..7dd80d24137 100644
--- a/src/bin/scripts/clusterdb.c
+++ b/src/bin/scripts/clusterdb.c
@@ -16,7 +16,6 @@
#include "fe_utils/option_utils.h"
#include "fe_utils/query_utils.h"
#include "fe_utils/simple_list.h"
-#include "fe_utils/string_utils.h"
static void cluster_one_database(const ConnParams *cparams, const char *table,
diff --git a/src/bin/scripts/common.c b/src/bin/scripts/common.c
index d16381eda90..e3acd1e67a2 100644
--- a/src/bin/scripts/common.c
+++ b/src/bin/scripts/common.c
@@ -21,7 +21,6 @@
#include "common/connect.h"
#include "common/logging.h"
#include "common/string.h"
-#include "fe_utils/cancel.h"
#include "fe_utils/query_utils.h"
#include "fe_utils/string_utils.h"
diff --git a/src/bin/scripts/reindexdb.c b/src/bin/scripts/reindexdb.c
index af0738d9334..659d8d5ba58 100644
--- a/src/bin/scripts/reindexdb.c
+++ b/src/bin/scripts/reindexdb.c
@@ -15,7 +15,6 @@
#include "catalog/pg_class_d.h"
#include "common.h"
-#include "common/connect.h"
#include "common/logging.h"
#include "fe_utils/cancel.h"
#include "fe_utils/option_utils.h"
diff --git a/src/common/controldata_utils.c b/src/common/controldata_utils.c
index 82309b25107..52d4702ce87 100644
--- a/src/common/controldata_utils.c
+++ b/src/common/controldata_utils.c
@@ -28,8 +28,8 @@
#include "access/xlog_internal.h"
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
-#include "common/file_perm.h"
#ifdef FRONTEND
+#include "common/file_perm.h"
#include "common/logging.h"
#endif
#include "port/pg_crc32c.h"
diff --git a/src/common/string.c b/src/common/string.c
index ce9f2bc7ca5..4c42ae299b5 100644
--- a/src/common/string.c
+++ b/src/common/string.c
@@ -22,7 +22,6 @@
#endif
#include "common/string.h"
-#include "lib/stringinfo.h"
/*
diff --git a/src/common/unicode_case.c b/src/common/unicode_case.c
index 291e256e639..542e7ea5378 100644
--- a/src/common/unicode_case.c
+++ b/src/common/unicode_case.c
@@ -17,7 +17,6 @@
#include "common/unicode_case.h"
#include "common/unicode_case_table.h"
-#include "common/unicode_category.h"
#include "mb/pg_wchar.h"
static const pg_case_map *find_case_map(pg_wchar ucs);
diff --git a/src/fe_utils/astreamer_file.c b/src/fe_utils/astreamer_file.c
index c9a030853bc..47568da2dbb 100644
--- a/src/fe_utils/astreamer_file.c
+++ b/src/fe_utils/astreamer_file.c
@@ -19,7 +19,6 @@
#include "common/file_perm.h"
#include "common/logging.h"
-#include "common/string.h"
#include "fe_utils/astreamer.h"
typedef struct astreamer_plain_writer
diff --git a/src/fe_utils/astreamer_gzip.c b/src/fe_utils/astreamer_gzip.c
index ca5be6423a1..e0b755317cb 100644
--- a/src/fe_utils/astreamer_gzip.c
+++ b/src/fe_utils/astreamer_gzip.c
@@ -32,9 +32,7 @@
#include <zlib.h>
#endif
-#include "common/file_perm.h"
#include "common/logging.h"
-#include "common/string.h"
#include "fe_utils/astreamer.h"
#ifdef HAVE_LIBZ
diff --git a/src/fe_utils/astreamer_lz4.c b/src/fe_utils/astreamer_lz4.c
index 2bf14084e7f..a628088edfa 100644
--- a/src/fe_utils/astreamer_lz4.c
+++ b/src/fe_utils/astreamer_lz4.c
@@ -21,9 +21,7 @@
#include <lz4frame.h>
#endif
-#include "common/file_perm.h"
#include "common/logging.h"
-#include "common/string.h"
#include "fe_utils/astreamer.h"
#ifdef USE_LZ4
diff --git a/src/fe_utils/cancel.c b/src/fe_utils/cancel.c
index dcff9a85641..0df07fa4aa4 100644
--- a/src/fe_utils/cancel.c
+++ b/src/fe_utils/cancel.c
@@ -18,9 +18,7 @@
#include <unistd.h>
-#include "common/connect.h"
#include "fe_utils/cancel.h"
-#include "fe_utils/string_utils.h"
/*
--
2.47.0
Attachments:
[text/plain] 0001-Remove-unused-include-s-from-backend-.c-files.patch (32.7K, ../../[email protected]/2-0001-Remove-unused-include-s-from-backend-.c-files.patch)
download | inline diff:
From 365b6f236589b02ada99b55acdc4cfff33c71a05 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Thu, 8 Feb 2024 16:26:33 +0100
Subject: [PATCH 1/5] Remove unused #include's from backend .c files
as determined by IWYU
---
src/backend/access/brin/brin_bloom.c | 1 -
src/backend/access/gin/ginvacuum.c | 1 -
src/backend/access/heap/vacuumlazy.c | 2 --
src/backend/access/nbtree/nbtree.c | 3 ---
src/backend/access/nbtree/nbtsort.c | 1 -
src/backend/access/sequence/sequence.c | 2 +-
src/backend/access/transam/slru.c | 2 +-
src/backend/access/transam/xlog.c | 2 --
src/backend/backup/basebackup.c | 1 -
src/backend/backup/basebackup_incremental.c | 2 --
src/backend/catalog/pg_depend.c | 1 -
src/backend/commands/analyze.c | 2 --
src/backend/commands/cluster.c | 1 -
src/backend/commands/copyfromparse.c | 1 -
src/backend/commands/createas.c | 2 --
src/backend/commands/indexcmds.c | 1 -
src/backend/commands/vacuum.c | 1 -
src/backend/commands/waitlsn.c | 2 --
src/backend/executor/nodeBitmapHeapscan.c | 1 -
src/backend/executor/nodeHash.c | 1 -
src/backend/lib/integerset.c | 1 -
src/backend/libpq/be-secure-gssapi.c | 2 +-
src/backend/libpq/be-secure-openssl.c | 3 +--
src/backend/main/main.c | 1 -
src/backend/parser/gram.y | 4 ---
src/backend/parser/parse_clause.c | 2 --
src/backend/parser/parse_expr.c | 2 --
src/backend/parser/parse_jsontable.c | 4 +--
src/backend/postmaster/auxprocess.c | 5 ----
src/backend/postmaster/launch_backend.c | 26 +++++++------------
src/backend/postmaster/postmaster.c | 5 +---
src/backend/postmaster/walwriter.c | 2 --
src/backend/replication/logical/slotsync.c | 2 --
src/backend/replication/slotfuncs.c | 4 ---
src/backend/storage/aio/read_stream.c | 1 -
src/backend/storage/buffer/buf_init.c | 1 -
src/backend/storage/file/fd.c | 1 -
src/backend/storage/ipc/ipci.c | 2 --
src/backend/storage/ipc/sinvaladt.c | 1 -
src/backend/storage/lmgr/lmgr.c | 1 -
src/backend/storage/lmgr/lock.c | 1 -
src/backend/storage/lmgr/s_lock.c | 1 -
src/backend/storage/page/checksum.c | 2 +-
src/backend/storage/smgr/bulk_write.c | 1 -
src/backend/tcop/fastpath.c | 1 -
src/backend/tcop/postgres.c | 1 -
src/backend/utils/Gen_fmgrtab.pl | 1 -
src/backend/utils/activity/backend_progress.c | 1 -
src/backend/utils/activity/backend_status.c | 2 --
src/backend/utils/activity/pgstat.c | 1 -
src/backend/utils/activity/pgstat_relation.c | 1 -
src/backend/utils/activity/wait_event.c | 1 -
src/backend/utils/adt/float.c | 1 -
src/backend/utils/adt/jsonpath_exec.c | 2 --
src/backend/utils/adt/pg_upgrade_support.c | 1 -
src/backend/utils/adt/waitfuncs.c | 2 +-
src/backend/utils/hash/pg_crc.c | 2 +-
src/backend/utils/misc/injection_point.c | 7 ++---
src/backend/utils/misc/pg_config.c | 1 -
59 files changed, 22 insertions(+), 108 deletions(-)
diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c
index f94e2b0bfbd..2e210d3e227 100644
--- a/src/backend/access/brin/brin_bloom.c
+++ b/src/backend/access/brin/brin_bloom.c
@@ -124,7 +124,6 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/pg_am.h"
-#include "catalog/pg_amop.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
#include "utils/fmgrprotos.h"
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index b3f415e2849..c6e8b39d4c9 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -19,7 +19,6 @@
#include "access/xloginsert.h"
#include "commands/vacuum.h"
#include "miscadmin.h"
-#include "postmaster/autovacuum.h"
#include "storage/indexfsm.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index d82aa3d4896..793bd33cb4d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -35,7 +35,6 @@
#include "access/genam.h"
#include "access/heapam.h"
-#include "access/heapam_xlog.h"
#include "access/htup_details.h"
#include "access/multixact.h"
#include "access/tidstore.h"
@@ -56,7 +55,6 @@
#include "storage/freespace.h"
#include "storage/lmgr.h"
#include "utils/lsyscache.h"
-#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/timestamp.h"
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index f4f79f27062..484ede8c2e1 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -20,10 +20,8 @@
#include "access/nbtree.h"
#include "access/relscan.h"
-#include "access/xloginsert.h"
#include "commands/progress.h"
#include "commands/vacuum.h"
-#include "miscadmin.h"
#include "nodes/execnodes.h"
#include "pgstat.h"
#include "storage/bulk_write.h"
@@ -31,7 +29,6 @@
#include "storage/indexfsm.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
-#include "storage/smgr.h"
#include "utils/fmgrprotos.h"
#include "utils/index_selfuncs.h"
#include "utils/memutils.h"
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 5cca0d4f520..fb9a05f7af1 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -45,7 +45,6 @@
#include "access/relscan.h"
#include "access/table.h"
#include "access/xact.h"
-#include "access/xloginsert.h"
#include "catalog/index.h"
#include "commands/progress.h"
#include "executor/instrument.h"
diff --git a/src/backend/access/sequence/sequence.c b/src/backend/access/sequence/sequence.c
index 8d6b7bb5dc2..d2cf95aadc2 100644
--- a/src/backend/access/sequence/sequence.c
+++ b/src/backend/access/sequence/sequence.c
@@ -22,7 +22,7 @@
#include "access/relation.h"
#include "access/sequence.h"
-#include "storage/lmgr.h"
+#include "utils/rel.h"
static inline void validate_relation_kind(Relation r);
diff --git a/src/backend/access/transam/slru.c b/src/backend/access/transam/slru.c
index 889eff1815b..f249c3cd05e 100644
--- a/src/backend/access/transam/slru.c
+++ b/src/backend/access/transam/slru.c
@@ -70,7 +70,7 @@
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/shmem.h"
-#include "utils/guc_hooks.h"
+#include "utils/guc.h"
/*
* Converts segment number to the filename of the segment.
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 9102c8d772e..11f623cabd1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -74,7 +74,6 @@
#include "pg_trace.h"
#include "pgstat.h"
#include "port/atomics.h"
-#include "port/pg_iovec.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
#include "postmaster/walsummarizer.h"
@@ -98,7 +97,6 @@
#include "utils/guc_hooks.h"
#include "utils/guc_tables.h"
#include "utils/injection_point.h"
-#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
diff --git a/src/backend/backup/basebackup.c b/src/backend/backup/basebackup.c
index 0f8cddcbeeb..e2ed9081d1c 100644
--- a/src/backend/backup/basebackup.c
+++ b/src/backend/backup/basebackup.c
@@ -33,7 +33,6 @@
#include "nodes/pg_list.h"
#include "pgstat.h"
#include "pgtar.h"
-#include "port.h"
#include "postmaster/syslogger.h"
#include "postmaster/walsummarizer.h"
#include "replication/slot.h"
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index f581a5f9e68..275615877eb 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -27,9 +27,7 @@
#include "common/hashfn.h"
#include "common/int.h"
#include "common/parse_manifest.h"
-#include "datatype/timestamp.h"
#include "postmaster/walsummarizer.h"
-#include "utils/timestamp.h"
#define BLOCKS_PER_READ 512
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index cfd7ef51dfa..2b4514e8a35 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -28,7 +28,6 @@
#include "miscadmin.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
-#include "utils/syscache.h"
#include "utils/rel.h"
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 38fb4c3ef23..9a56de2282f 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -41,7 +41,6 @@
#include "parser/parse_oper.h"
#include "parser/parse_relation.h"
#include "pgstat.h"
-#include "postmaster/autovacuum.h"
#include "statistics/extended_stats_internal.h"
#include "statistics/statistics.h"
#include "storage/bufmgr.h"
@@ -54,7 +53,6 @@
#include "utils/pg_rusage.h"
#include "utils/sampling.h"
#include "utils/sortsupport.h"
-#include "utils/spccache.h"
#include "utils/syscache.h"
#include "utils/timestamp.h"
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 78f96789b0e..ae0863d9a25 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -32,7 +32,6 @@
#include "catalog/namespace.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_am.h"
-#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
#include "catalog/toasting.h"
#include "commands/cluster.h"
diff --git a/src/backend/commands/copyfromparse.c b/src/backend/commands/copyfromparse.c
index 654fecb1b14..d1d43b53d83 100644
--- a/src/backend/commands/copyfromparse.c
+++ b/src/backend/commands/copyfromparse.c
@@ -70,7 +70,6 @@
#include "libpq/pqformat.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/miscnodes.h"
#include "pgstat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 68ec122dbf9..aaeaaffe405 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -35,14 +35,12 @@
#include "commands/prepare.h"
#include "commands/tablecmds.h"
#include "commands/view.h"
-#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "rewrite/rewriteHandler.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-#include "utils/rel.h"
#include "utils/rls.h"
#include "utils/snapmgr.h"
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index e33ad815295..2f652463e3c 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -58,7 +58,6 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ac8f5d9c259..86f36b36954 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -34,7 +34,6 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
-#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
diff --git a/src/backend/commands/waitlsn.c b/src/backend/commands/waitlsn.c
index 501938f4330..ce1537136c8 100644
--- a/src/backend/commands/waitlsn.c
+++ b/src/backend/commands/waitlsn.c
@@ -21,7 +21,6 @@
#include "access/xlog.h"
#include "access/xlogrecovery.h"
#include "commands/waitlsn.h"
-#include "funcapi.h"
#include "miscadmin.h"
#include "storage/latch.h"
#include "storage/proc.h"
@@ -29,7 +28,6 @@
#include "utils/fmgrprotos.h"
#include "utils/pg_lsn.h"
#include "utils/snapmgr.h"
-#include "utils/wait_event_types.h"
static int waitlsn_cmp(const pairingheap_node *a, const pairingheap_node *b,
void *arg);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3c63bdd93df..01d825c47bd 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -46,7 +46,6 @@
#include "pgstat.h"
#include "storage/bufmgr.h"
#include "utils/rel.h"
-#include "utils/snapmgr.h"
#include "utils/spccache.h"
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 0456a017dc6..3e22d50e3a4 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -35,7 +35,6 @@
#include "executor/nodeHash.h"
#include "executor/nodeHashjoin.h"
#include "miscadmin.h"
-#include "port/atomics.h"
#include "port/pg_bitutils.h"
#include "utils/dynahash.h"
#include "utils/lsyscache.h"
diff --git a/src/backend/lib/integerset.c b/src/backend/lib/integerset.c
index 9f1d4db52b2..bd9b28e19bd 100644
--- a/src/backend/lib/integerset.c
+++ b/src/backend/lib/integerset.c
@@ -72,7 +72,6 @@
#include "postgres.h"
#include "lib/integerset.h"
-#include "port/pg_bitutils.h"
#include "utils/memutils.h"
diff --git a/src/backend/libpq/be-secure-gssapi.c b/src/backend/libpq/be-secure-gssapi.c
index 2d36c76324a..5a009776d12 100644
--- a/src/backend/libpq/be-secure-gssapi.c
+++ b/src/backend/libpq/be-secure-gssapi.c
@@ -18,9 +18,9 @@
#include "libpq/auth.h"
#include "libpq/be-gssapi-common.h"
#include "libpq/libpq.h"
-#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "port/pg_bswap.h"
#include "utils/injection_point.h"
#include "utils/memutils.h"
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index 9d503104be3..b3d3f83a157 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -33,8 +33,7 @@
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/latch.h"
-#include "tcop/tcopprot.h"
-#include "utils/builtins.h"
+#include "utils/guc.h"
#include "utils/memutils.h"
/*
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index 4672aab8378..aea93a02298 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -32,7 +32,6 @@
#include "bootstrap/bootstrap.h"
#include "common/username.h"
-#include "port/atomics.h"
#include "postmaster/postmaster.h"
#include "tcop/tcopprot.h"
#include "utils/help_config.h"
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4aa8646af7b..f63d1538df7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -48,7 +48,6 @@
#include <ctype.h>
#include <limits.h>
-#include "access/tableam.h"
#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -59,10 +58,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parser.h"
-#include "storage/lmgr.h"
-#include "utils/date.h"
#include "utils/datetime.h"
-#include "utils/numeric.h"
#include "utils/xml.h"
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 4c976909088..979926b6052 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -40,11 +40,9 @@
#include "parser/parse_target.h"
#include "parser/parse_type.h"
#include "parser/parser.h"
-#include "parser/parsetree.h"
#include "rewrite/rewriteManip.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
-#include "utils/guc.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
#include "utils/syscache.h"
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index ef0b560f5e3..c2806297aa4 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -16,7 +16,6 @@
#include "postgres.h"
#include "catalog/pg_aggregate.h"
-#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "commands/dbcommands.h"
#include "miscadmin.h"
@@ -37,7 +36,6 @@
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/fmgroids.h"
-#include "utils/jsonb.h"
#include "utils/lsyscache.h"
#include "utils/timestamp.h"
#include "utils/xml.h"
diff --git a/src/backend/parser/parse_jsontable.c b/src/backend/parser/parse_jsontable.c
index 6d4e4968202..ecb140e6e71 100644
--- a/src/backend/parser/parse_jsontable.c
+++ b/src/backend/parser/parse_jsontable.c
@@ -15,9 +15,7 @@
#include "postgres.h"
-#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
-#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
@@ -26,7 +24,7 @@
#include "parser/parse_expr.h"
#include "parser/parse_relation.h"
#include "parser/parse_type.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/json.h"
#include "utils/lsyscache.h"
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 74b8a00c94b..d19174bda39 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -18,11 +18,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/auxprocess.h"
-#include "postmaster/bgwriter.h"
-#include "postmaster/startup.h"
-#include "postmaster/walsummarizer.h"
-#include "postmaster/walwriter.h"
-#include "replication/walreceiver.h"
#include "storage/condition_variable.h"
#include "storage/ipc.h"
#include "storage/proc.h"
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 0ae23fdf55e..25020cedad0 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -33,15 +33,9 @@
#include <unistd.h>
-#include "access/xlog.h"
-#include "common/file_utils.h"
#include "libpq/libpq-be.h"
-#include "libpq/pqsignal.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
-#include "port.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
#include "postmaster/fork_process.h"
@@ -54,20 +48,9 @@
#include "replication/slotsync.h"
#include "replication/walreceiver.h"
#include "storage/dsm.h"
-#include "storage/fd.h"
-#include "storage/ipc.h"
#include "storage/pg_shmem.h"
-#include "storage/pmsignal.h"
-#include "storage/proc.h"
-#include "storage/procsignal.h"
#include "tcop/backend_startup.h"
-#include "tcop/tcopprot.h"
-#include "utils/builtins.h"
-#include "utils/datetime.h"
-#include "utils/guc.h"
-#include "utils/injection_point.h"
#include "utils/memutils.h"
-#include "utils/timestamp.h"
#ifdef EXEC_BACKEND
#include "nodes/queryjumble.h"
@@ -78,6 +61,15 @@
#ifdef EXEC_BACKEND
+#include "common/file_utils.h"
+#include "storage/fd.h"
+#include "storage/lwlock.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "storage/procsignal.h"
+#include "tcop/tcopprot.h"
+#include "utils/injection_point.h"
+
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
typedef struct
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 85fd24e8287..407f55996ba 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -92,8 +92,6 @@
#include "access/xlog.h"
#include "access/xlogrecovery.h"
#include "common/file_perm.h"
-#include "common/file_utils.h"
-#include "common/ip.h"
#include "common/pg_prng.h"
#include "lib/ilist.h"
#include "libpq/libpq.h"
@@ -102,7 +100,6 @@
#include "pgstat.h"
#include "port/pg_bswap.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -114,7 +111,6 @@
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pmsignal.h"
-#include "storage/proc.h"
#include "tcop/backend_startup.h"
#include "tcop/tcopprot.h"
#include "utils/datetime.h"
@@ -124,6 +120,7 @@
#include "utils/varlena.h"
#ifdef EXEC_BACKEND
+#include "common/file_utils.h"
#include "storage/pg_shmem.h"
#endif
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 6e7918a78d4..91013f6e936 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -54,12 +54,10 @@
#include "storage/bufmgr.h"
#include "storage/condition_variable.h"
#include "storage/fd.h"
-#include "storage/ipc.h"
#include "storage/lwlock.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/smgr.h"
-#include "utils/guc.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
#include "utils/resowner.h"
diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c
index f9649eec1a5..d62186a5107 100644
--- a/src/backend/replication/logical/slotsync.c
+++ b/src/backend/replication/logical/slotsync.c
@@ -55,9 +55,7 @@
#include "commands/dbcommands.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
-#include "postmaster/postmaster.h"
#include "replication/logical.h"
#include "replication/slotsync.h"
#include "replication/snapbuild.h"
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index c7bfbb15e0a..488a161b3e7 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -17,16 +17,12 @@
#include "access/xlogrecovery.h"
#include "access/xlogutils.h"
#include "funcapi.h"
-#include "miscadmin.h"
-#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/slot.h"
#include "replication/slotsync.h"
#include "utils/builtins.h"
#include "utils/guc.h"
-#include "utils/inval.h"
#include "utils/pg_lsn.h"
-#include "utils/resowner.h"
/*
* Helper function for creating a new physical replication slot with
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index b8ea1180e25..3d30e6224f7 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -89,7 +89,6 @@
*/
#include "postgres.h"
-#include "catalog/pg_tablespace.h"
#include "miscadmin.h"
#include "storage/fd.h"
#include "storage/smgr.h"
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 09bec6449b6..56761a8eedc 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -16,7 +16,6 @@
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
-#include "storage/proc.h"
BufferDescPadded *BufferDescriptors;
char *BufferBlocks;
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index ec1505802b9..42bf857e873 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -93,7 +93,6 @@
#include "common/pg_prng.h"
#include "miscadmin.h"
#include "pgstat.h"
-#include "portability/mem.h"
#include "postmaster/startup.h"
#include "storage/fd.h"
#include "storage/ipc.h"
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 10fc18f2529..b0852564388 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -31,7 +31,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
-#include "postmaster/postmaster.h"
#include "postmaster/walsummarizer.h"
#include "replication/logicallauncher.h"
#include "replication/origin.h"
@@ -50,7 +49,6 @@
#include "storage/procarray.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
-#include "storage/spin.h"
#include "utils/guc.h"
#include "utils/injection_point.h"
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index db59637c438..ff817449a44 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -17,7 +17,6 @@
#include <signal.h>
#include <unistd.h>
-#include "access/transam.h"
#include "miscadmin.h"
#include "storage/ipc.h"
#include "storage/proc.h"
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index 094522acb41..9f43620e472 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -24,7 +24,6 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "utils/inval.h"
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 09a8ac15784..97c0fd469dc 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -41,7 +41,6 @@
#include "pg_trace.h"
#include "storage/proc.h"
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "storage/spin.h"
#include "storage/standby.h"
#include "utils/memutils.h"
diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c
index 69549a65dba..f37d338566f 100644
--- a/src/backend/storage/lmgr/s_lock.c
+++ b/src/backend/storage/lmgr/s_lock.c
@@ -51,7 +51,6 @@
#include <unistd.h>
#include "common/pg_prng.h"
-#include "port/atomics.h"
#include "storage/s_lock.h"
#include "utils/wait_event.h"
diff --git a/src/backend/storage/page/checksum.c b/src/backend/storage/page/checksum.c
index 243878fe9e0..ffc3d245887 100644
--- a/src/backend/storage/page/checksum.c
+++ b/src/backend/storage/page/checksum.c
@@ -19,4 +19,4 @@
* external programs can incorporate the checksum code by #include'ing
* that file from the exported Postgres headers. (Compare our CRC code.)
*/
-#include "storage/checksum_impl.h"
+#include "storage/checksum_impl.h" /* IWYU pragma: keep */
diff --git a/src/backend/storage/smgr/bulk_write.c b/src/backend/storage/smgr/bulk_write.c
index 1a5f3ce96e1..f0a65bfe242 100644
--- a/src/backend/storage/smgr/bulk_write.c
+++ b/src/backend/storage/smgr/bulk_write.c
@@ -36,7 +36,6 @@
#include "access/xloginsert.h"
#include "access/xlogrecord.h"
-#include "storage/bufmgr.h"
#include "storage/bufpage.h"
#include "storage/bulk_write.h"
#include "storage/proc.h"
diff --git a/src/backend/tcop/fastpath.c b/src/backend/tcop/fastpath.c
index deac71335c2..c6281cf86d3 100644
--- a/src/backend/tcop/fastpath.c
+++ b/src/backend/tcop/fastpath.c
@@ -26,7 +26,6 @@
#include "libpq/protocol.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "port/pg_bswap.h"
#include "tcop/fastpath.h"
#include "tcop/tcopprot.h"
#include "utils/acl.h"
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7f5eada9d45..8cc23a9cef8 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -53,7 +53,6 @@
#include "pg_getopt.h"
#include "pg_trace.h"
#include "pgstat.h"
-#include "postmaster/autovacuum.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/logicallauncher.h"
diff --git a/src/backend/utils/Gen_fmgrtab.pl b/src/backend/utils/Gen_fmgrtab.pl
index 37628a85c73..738828b95c6 100644
--- a/src/backend/utils/Gen_fmgrtab.pl
+++ b/src/backend/utils/Gen_fmgrtab.pl
@@ -182,7 +182,6 @@
#include "postgres.h"
-#include "access/transam.h"
#include "utils/fmgrtab.h"
#include "utils/fmgrprotos.h"
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index c78c5eb5076..b80249a79e7 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -12,7 +12,6 @@
#include "access/parallel.h"
#include "libpq/pqformat.h"
-#include "port/atomics.h" /* for memory barriers */
#include "utils/backend_progress.h"
#include "utils/backend_status.h"
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 34a55e2177d..bdb3a296ca6 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -16,11 +16,9 @@
#include "miscadmin.h"
#include "pg_trace.h"
#include "pgstat.h"
-#include "port/atomics.h" /* for memory barriers */
#include "storage/ipc.h"
#include "storage/proc.h" /* for MyProc */
#include "storage/procarray.h"
-#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h" /* for application_name */
#include "utils/memutils.h"
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index d1768a89f6e..be48432cc38 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -105,7 +105,6 @@
#include "access/xlog.h"
#include "lib/dshash.h"
#include "pgstat.h"
-#include "port/atomics.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/lwlock.h"
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..36d3adf7310 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -20,7 +20,6 @@
#include "access/twophase_rmgr.h"
#include "access/xact.h"
#include "catalog/catalog.h"
-#include "postmaster/autovacuum.h"
#include "utils/memutils.h"
#include "utils/pgstat_internal.h"
#include "utils/rel.h"
diff --git a/src/backend/utils/activity/wait_event.c b/src/backend/utils/activity/wait_event.c
index bbf59482be1..d930277140c 100644
--- a/src/backend/utils/activity/wait_event.c
+++ b/src/backend/utils/activity/wait_event.c
@@ -22,7 +22,6 @@
*/
#include "postgres.h"
-#include "port/pg_bitutils.h"
#include "storage/lmgr.h" /* for GetLockNameFromTagType */
#include "storage/lwlock.h" /* for GetLWLockIdentifier */
#include "storage/spin.h"
diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c
index 6fa6ffb51f5..f5927bcc564 100644
--- a/src/backend/utils/adt/float.c
+++ b/src/backend/utils/adt/float.c
@@ -27,7 +27,6 @@
#include "utils/float.h"
#include "utils/fmgrprotos.h"
#include "utils/sortsupport.h"
-#include "utils/timestamp.h"
/*
diff --git a/src/backend/utils/adt/jsonpath_exec.c b/src/backend/utils/adt/jsonpath_exec.c
index 1184cba983a..ac214598037 100644
--- a/src/backend/utils/adt/jsonpath_exec.c
+++ b/src/backend/utils/adt/jsonpath_exec.c
@@ -61,7 +61,6 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
-#include "executor/execExpr.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "nodes/miscnodes.h"
@@ -74,7 +73,6 @@
#include "utils/formatting.h"
#include "utils/json.h"
#include "utils/jsonpath.h"
-#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/timestamp.h"
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index c54b08fe180..8a45b5827e3 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -28,7 +28,6 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/pg_lsn.h"
-#include "utils/syscache.h"
#define CHECK_IS_BINARY_UPGRADE \
diff --git a/src/backend/utils/adt/waitfuncs.c b/src/backend/utils/adt/waitfuncs.c
index e135c9e5e45..a083ced251e 100644
--- a/src/backend/utils/adt/waitfuncs.c
+++ b/src/backend/utils/adt/waitfuncs.c
@@ -17,7 +17,7 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/array.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/wait_event.h"
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
diff --git a/src/backend/utils/hash/pg_crc.c b/src/backend/utils/hash/pg_crc.c
index a85e6171864..2fc85405b63 100644
--- a/src/backend/utils/hash/pg_crc.c
+++ b/src/backend/utils/hash/pg_crc.c
@@ -20,7 +20,7 @@
#include "postgres.h"
#include "port/pg_crc32c.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/pg_crc.h"
#include "varatt.h"
diff --git a/src/backend/utils/misc/injection_point.c b/src/backend/utils/misc/injection_point.c
index 7c1a7cf889d..deb7d245ef8 100644
--- a/src/backend/utils/misc/injection_point.c
+++ b/src/backend/utils/misc/injection_point.c
@@ -17,6 +17,10 @@
*/
#include "postgres.h"
+#include "utils/injection_point.h"
+
+#ifdef USE_INJECTION_POINTS
+
#include <sys/stat.h>
#include "fmgr.h"
@@ -25,11 +29,8 @@
#include "storage/lwlock.h"
#include "storage/shmem.h"
#include "utils/hsearch.h"
-#include "utils/injection_point.h"
#include "utils/memutils.h"
-#ifdef USE_INJECTION_POINTS
-
/* Field sizes */
#define INJ_NAME_MAXLEN 64
#define INJ_LIB_MAXLEN 128
diff --git a/src/backend/utils/misc/pg_config.c b/src/backend/utils/misc/pg_config.c
index 2af47e1244f..89dd18355ae 100644
--- a/src/backend/utils/misc/pg_config.c
+++ b/src/backend/utils/misc/pg_config.c
@@ -17,7 +17,6 @@
#include "common/config_info.h"
#include "funcapi.h"
#include "miscadmin.h"
-#include "port.h"
#include "utils/builtins.h"
Datum
base-commit: 11c87216d134a606938531e52edab6189bce6c2d
--
2.47.0
[text/plain] 0002-Remove-unused-include-s-from-contrib-.c-files.patch (31.6K, ../../[email protected]/3-0002-Remove-unused-include-s-from-contrib-.c-files.patch)
download | inline diff:
From dc8dab38070ba6005aefb2cfe941a588f5362664 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 22 Jun 2024 10:22:29 +0200
Subject: [PATCH 2/5] Remove unused #include's from contrib .c files
as determined by IWYU
---
contrib/amcheck/verify_heapam.c | 6 +++++-
contrib/amcheck/verify_nbtree.c | 2 --
contrib/auth_delay/auth_delay.c | 2 --
contrib/auto_explain/auto_explain.c | 2 --
contrib/basic_archive/basic_archive.c | 1 -
contrib/bloom/blcost.c | 1 -
contrib/bloom/blinsert.c | 4 +---
contrib/bloom/blscan.c | 4 ----
contrib/bloom/blutils.c | 5 +----
contrib/bloom/blvacuum.c | 4 ----
contrib/bloom/blvalidate.c | 2 --
contrib/btree_gin/btree_gin.c | 3 ---
contrib/btree_gist/btree_bit.c | 3 +--
contrib/btree_gist/btree_bool.c | 1 -
contrib/btree_gist/btree_bytea.c | 3 +--
contrib/btree_gist/btree_date.c | 2 +-
contrib/btree_gist/btree_enum.c | 2 +-
contrib/btree_gist/btree_inet.c | 1 -
contrib/btree_gist/btree_interval.c | 2 +-
contrib/btree_gist/btree_macaddr.c | 2 +-
contrib/btree_gist/btree_macaddr8.c | 2 +-
contrib/btree_gist/btree_text.c | 3 ++-
contrib/btree_gist/btree_time.c | 2 +-
contrib/btree_gist/btree_ts.c | 4 ++--
contrib/btree_gist/btree_utils_num.h | 1 -
contrib/btree_gist/btree_utils_var.c | 3 +--
contrib/btree_gist/btree_utils_var.h | 1 -
contrib/citext/citext.c | 2 +-
contrib/dict_xsyn/dict_xsyn.c | 2 +-
contrib/fuzzystrmatch/fuzzystrmatch.c | 1 -
contrib/intarray/_int_bool.c | 1 -
contrib/intarray/_int_selfuncs.c | 3 +--
contrib/isn/isn.c | 1 -
contrib/lo/lo.c | 2 +-
contrib/ltree/ltree_io.c | 1 -
contrib/ltree/ltree_op.c | 4 +---
contrib/pageinspect/brinfuncs.c | 4 ----
contrib/pageinspect/fsmfuncs.c | 2 +-
contrib/pageinspect/ginfuncs.c | 3 ---
contrib/pageinspect/gistfuncs.c | 3 ---
contrib/pageinspect/heapfuncs.c | 1 -
contrib/pg_freespacemap/pg_freespacemap.c | 2 +-
contrib/pg_prewarm/autoprewarm.c | 9 +--------
contrib/pg_stat_statements/pg_stat_statements.c | 3 ---
contrib/pg_surgery/heap_surgery.c | 4 ++--
contrib/pg_visibility/pg_visibility.c | 2 --
contrib/pgcrypto/pgcrypto.c | 1 -
contrib/pgrowlocks/pgrowlocks.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 6 ------
contrib/pgstattuple/pgstatindex.c | 3 ---
contrib/pgstattuple/pgstattuple.c | 1 -
contrib/postgres_fdw/connection.c | 4 ----
contrib/postgres_fdw/deparse.c | 2 --
contrib/postgres_fdw/option.c | 1 -
contrib/postgres_fdw/postgres_fdw.c | 3 ---
contrib/spi/moddatetime.c | 2 +-
contrib/tablefunc/tablefunc.c | 1 +
contrib/tablefunc/tablefunc.h | 2 --
contrib/tsm_system_rows/tsm_system_rows.c | 1 -
contrib/tsm_system_time/tsm_system_time.c | 1 -
contrib/unaccent/unaccent.c | 2 --
contrib/xml2/xpath.c | 1 -
contrib/xml2/xslt_proc.c | 3 ---
63 files changed, 33 insertions(+), 121 deletions(-)
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index f2526ed63a2..9c74daaceed 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -12,18 +12,22 @@
#include "access/detoast.h"
#include "access/genam.h"
-#include "access/heapam.h"
#include "access/heaptoast.h"
#include "access/multixact.h"
+#include "access/relation.h"
+#include "access/table.h"
#include "access/toast_internals.h"
#include "access/visibilitymap.h"
+#include "access/xact.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_class.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/procarray.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/rel.h"
PG_FUNCTION_INFO_V1(verify_heapam);
diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c
index 7cfb136763f..8b82797c10f 100644
--- a/contrib/amcheck/verify_nbtree.c
+++ b/contrib/amcheck/verify_nbtree.c
@@ -33,11 +33,9 @@
#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opfamily_d.h"
-#include "commands/tablecmds.h"
#include "common/pg_prng.h"
#include "lib/bloomfilter.h"
#include "miscadmin.h"
-#include "storage/lmgr.h"
#include "storage/smgr.h"
#include "utils/guc.h"
#include "utils/memutils.h"
diff --git a/contrib/auth_delay/auth_delay.c b/contrib/auth_delay/auth_delay.c
index ff0e1fd461b..e8dfef3290e 100644
--- a/contrib/auth_delay/auth_delay.c
+++ b/contrib/auth_delay/auth_delay.c
@@ -14,9 +14,7 @@
#include <limits.h>
#include "libpq/auth.h"
-#include "port.h"
#include "utils/guc.h"
-#include "utils/timestamp.h"
PG_MODULE_MAGIC;
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 677c135f59b..623a674f99c 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -18,8 +18,6 @@
#include "commands/explain.h"
#include "common/pg_prng.h"
#include "executor/instrument.h"
-#include "jit/jit.h"
-#include "nodes/params.h"
#include "utils/guc.h"
PG_MODULE_MAGIC;
diff --git a/contrib/basic_archive/basic_archive.c b/contrib/basic_archive/basic_archive.c
index 028cf51c25d..36e2feb4fb8 100644
--- a/contrib/basic_archive/basic_archive.c
+++ b/contrib/basic_archive/basic_archive.c
@@ -36,7 +36,6 @@
#include "storage/copydir.h"
#include "storage/fd.h"
#include "utils/guc.h"
-#include "utils/memutils.h"
PG_MODULE_MAGIC;
diff --git a/contrib/bloom/blcost.c b/contrib/bloom/blcost.c
index 9b629b2c3d5..0f2cf1546c7 100644
--- a/contrib/bloom/blcost.c
+++ b/contrib/bloom/blcost.c
@@ -13,7 +13,6 @@
#include "postgres.h"
#include "bloom.h"
-#include "fmgr.h"
#include "utils/selfuncs.h"
/*
diff --git a/contrib/bloom/blinsert.c b/contrib/bloom/blinsert.c
index f8a1061abb9..dbb79a51edd 100644
--- a/contrib/bloom/blinsert.c
+++ b/contrib/bloom/blinsert.c
@@ -16,11 +16,9 @@
#include "access/generic_xlog.h"
#include "access/tableam.h"
#include "bloom.h"
-#include "catalog/index.h"
#include "miscadmin.h"
+#include "nodes/execnodes.h"
#include "storage/bufmgr.h"
-#include "storage/indexfsm.h"
-#include "storage/smgr.h"
#include "utils/memutils.h"
#include "utils/rel.h"
diff --git a/contrib/bloom/blscan.c b/contrib/bloom/blscan.c
index 94b6b449c8c..bf455e579fe 100644
--- a/contrib/bloom/blscan.c
+++ b/contrib/bloom/blscan.c
@@ -15,11 +15,7 @@
#include "access/relscan.h"
#include "bloom.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/bufmgr.h"
-#include "storage/lmgr.h"
-#include "utils/memutils.h"
-#include "utils/rel.h"
/*
* Begin scan of bloom index.
diff --git a/contrib/bloom/blutils.c b/contrib/bloom/blutils.c
index a29330afcd3..af10199394a 100644
--- a/contrib/bloom/blutils.c
+++ b/contrib/bloom/blutils.c
@@ -17,14 +17,11 @@
#include "access/generic_xlog.h"
#include "access/reloptions.h"
#include "bloom.h"
-#include "catalog/index.h"
#include "commands/vacuum.h"
-#include "miscadmin.h"
#include "storage/bufmgr.h"
-#include "storage/freespace.h"
#include "storage/indexfsm.h"
-#include "storage/lmgr.h"
#include "utils/memutils.h"
+#include "varatt.h"
/* Signature dealing macros - note i is assumed to be of type int */
#define GETWORD(x,i) ( *( (BloomSignatureWord *)(x) + ( (i) / SIGNWORDBITS ) ) )
diff --git a/contrib/bloom/blvacuum.c b/contrib/bloom/blvacuum.c
index 0998240feaf..2a268fe0070 100644
--- a/contrib/bloom/blvacuum.c
+++ b/contrib/bloom/blvacuum.c
@@ -14,13 +14,9 @@
#include "access/genam.h"
#include "bloom.h"
-#include "catalog/storage.h"
#include "commands/vacuum.h"
-#include "miscadmin.h"
-#include "postmaster/autovacuum.h"
#include "storage/bufmgr.h"
#include "storage/indexfsm.h"
-#include "storage/lmgr.h"
/*
diff --git a/contrib/bloom/blvalidate.c b/contrib/bloom/blvalidate.c
index 88c5a791975..20d3ab9109b 100644
--- a/contrib/bloom/blvalidate.c
+++ b/contrib/bloom/blvalidate.c
@@ -20,8 +20,6 @@
#include "catalog/pg_opclass.h"
#include "catalog/pg_opfamily.h"
#include "catalog/pg_type.h"
-#include "utils/builtins.h"
-#include "utils/lsyscache.h"
#include "utils/regproc.h"
#include "utils/syscache.h"
diff --git a/contrib/btree_gin/btree_gin.c b/contrib/btree_gin/btree_gin.c
index 5e27906f802..533c55e9eaf 100644
--- a/contrib/btree_gin/btree_gin.c
+++ b/contrib/btree_gin/btree_gin.c
@@ -7,15 +7,12 @@
#include "access/stratnum.h"
#include "utils/builtins.h"
-#include "utils/bytea.h"
-#include "utils/cash.h"
#include "utils/date.h"
#include "utils/float.h"
#include "utils/inet.h"
#include "utils/numeric.h"
#include "utils/timestamp.h"
#include "utils/uuid.h"
-#include "utils/varbit.h"
PG_MODULE_MAGIC;
diff --git a/contrib/btree_gist/btree_bit.c b/contrib/btree_gist/btree_bit.c
index 6790f22b4b6..9d5e01a1977 100644
--- a/contrib/btree_gist/btree_bit.c
+++ b/contrib/btree_gist/btree_bit.c
@@ -5,8 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
-#include "utils/bytea.h"
+#include "utils/fmgrprotos.h"
#include "utils/varbit.h"
diff --git a/contrib/btree_gist/btree_bool.c b/contrib/btree_gist/btree_bool.c
index 8b2af129b52..bdc7834bb6a 100644
--- a/contrib/btree_gist/btree_bool.c
+++ b/contrib/btree_gist/btree_bool.c
@@ -5,7 +5,6 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "common/int.h"
typedef struct boolkey
{
diff --git a/contrib/btree_gist/btree_bytea.c b/contrib/btree_gist/btree_bytea.c
index 6b005f0157e..394e40ed16b 100644
--- a/contrib/btree_gist/btree_bytea.c
+++ b/contrib/btree_gist/btree_bytea.c
@@ -5,8 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
-#include "utils/bytea.h"
+#include "utils/fmgrprotos.h"
/*
diff --git a/contrib/btree_gist/btree_date.c b/contrib/btree_gist/btree_date.c
index 68a4107dbf0..5efb5ef96f5 100644
--- a/contrib/btree_gist/btree_date.c
+++ b/contrib/btree_gist/btree_date.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/date.h"
typedef struct
diff --git a/contrib/btree_gist/btree_enum.c b/contrib/btree_gist/btree_enum.c
index d4dc38a38e5..26682122f8e 100644
--- a/contrib/btree_gist/btree_enum.c
+++ b/contrib/btree_gist/btree_enum.c
@@ -6,7 +6,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
#include "fmgr.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
/* enums are really Oids, so we just use the same structure */
diff --git a/contrib/btree_gist/btree_inet.c b/contrib/btree_gist/btree_inet.c
index 2fb952dca83..c66ca99e0c3 100644
--- a/contrib/btree_gist/btree_inet.c
+++ b/contrib/btree_gist/btree_inet.c
@@ -7,7 +7,6 @@
#include "btree_utils_num.h"
#include "catalog/pg_type.h"
#include "utils/builtins.h"
-#include "utils/inet.h"
typedef struct inetkey
{
diff --git a/contrib/btree_gist/btree_interval.c b/contrib/btree_gist/btree_interval.c
index 156f2cebac5..c2f6eede226 100644
--- a/contrib/btree_gist/btree_interval.c
+++ b/contrib/btree_gist/btree_interval.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/timestamp.h"
typedef struct
diff --git a/contrib/btree_gist/btree_macaddr.c b/contrib/btree_gist/btree_macaddr.c
index 17290529c02..1367eaba735 100644
--- a/contrib/btree_gist/btree_macaddr.c
+++ b/contrib/btree_gist/btree_macaddr.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/inet.h"
typedef struct
diff --git a/contrib/btree_gist/btree_macaddr8.c b/contrib/btree_gist/btree_macaddr8.c
index 796cc4efee3..9676e5a15bd 100644
--- a/contrib/btree_gist/btree_macaddr8.c
+++ b/contrib/btree_gist/btree_macaddr8.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/inet.h"
typedef struct
diff --git a/contrib/btree_gist/btree_text.c b/contrib/btree_gist/btree_text.c
index be0eac7975b..28d6b5c9c2b 100644
--- a/contrib/btree_gist/btree_text.c
+++ b/contrib/btree_gist/btree_text.c
@@ -5,7 +5,8 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
+#include "mb/pg_wchar.h"
+#include "utils/fmgrprotos.h"
/*
** Text ops
diff --git a/contrib/btree_gist/btree_time.c b/contrib/btree_gist/btree_time.c
index d89401c0f51..3034bbb456b 100644
--- a/contrib/btree_gist/btree_time.c
+++ b/contrib/btree_gist/btree_time.c
@@ -5,7 +5,7 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/date.h"
#include "utils/timestamp.h"
diff --git a/contrib/btree_gist/btree_ts.c b/contrib/btree_gist/btree_ts.c
index 3f5ba91891d..b3cf2d6f60d 100644
--- a/contrib/btree_gist/btree_ts.c
+++ b/contrib/btree_gist/btree_ts.c
@@ -7,8 +7,8 @@
#include "btree_gist.h"
#include "btree_utils_num.h"
-#include "utils/builtins.h"
-#include "utils/datetime.h"
+#include "utils/fmgrprotos.h"
+#include "utils/timestamp.h"
#include "utils/float.h"
typedef struct
diff --git a/contrib/btree_gist/btree_utils_num.h b/contrib/btree_gist/btree_utils_num.h
index 11e892311ee..53e477d8b1e 100644
--- a/contrib/btree_gist/btree_utils_num.h
+++ b/contrib/btree_gist/btree_utils_num.h
@@ -9,7 +9,6 @@
#include "access/gist.h"
#include "btree_gist.h"
-#include "utils/rel.h"
typedef char GBT_NUMKEY;
diff --git a/contrib/btree_gist/btree_utils_var.c b/contrib/btree_gist/btree_utils_var.c
index 0c0e952f736..d9df2356cd1 100644
--- a/contrib/btree_gist/btree_utils_var.c
+++ b/contrib/btree_gist/btree_utils_var.c
@@ -9,8 +9,7 @@
#include "btree_gist.h"
#include "btree_utils_var.h"
-#include "utils/builtins.h"
-#include "utils/pg_locale.h"
+#include "mb/pg_wchar.h"
#include "utils/rel.h"
/* used for key sorting */
diff --git a/contrib/btree_gist/btree_utils_var.h b/contrib/btree_gist/btree_utils_var.h
index 2f8def655c8..9302d41ced6 100644
--- a/contrib/btree_gist/btree_utils_var.h
+++ b/contrib/btree_gist/btree_utils_var.h
@@ -6,7 +6,6 @@
#include "access/gist.h"
#include "btree_gist.h"
-#include "mb/pg_wchar.h"
/* Variable length key */
typedef bytea GBT_VARKEY;
diff --git a/contrib/citext/citext.c b/contrib/citext/citext.c
index 26af935a70f..3c461ff2ff2 100644
--- a/contrib/citext/citext.c
+++ b/contrib/citext/citext.c
@@ -5,7 +5,7 @@
#include "catalog/pg_collation.h"
#include "common/hashfn.h"
-#include "utils/builtins.h"
+#include "fmgr.h"
#include "utils/formatting.h"
#include "utils/varlena.h"
#include "varatt.h"
diff --git a/contrib/dict_xsyn/dict_xsyn.c b/contrib/dict_xsyn/dict_xsyn.c
index 62d2d841417..3635ed1df84 100644
--- a/contrib/dict_xsyn/dict_xsyn.c
+++ b/contrib/dict_xsyn/dict_xsyn.c
@@ -16,7 +16,7 @@
#include "commands/defrem.h"
#include "tsearch/ts_locale.h"
-#include "tsearch/ts_utils.h"
+#include "tsearch/ts_public.h"
PG_MODULE_MAGIC;
diff --git a/contrib/fuzzystrmatch/fuzzystrmatch.c b/contrib/fuzzystrmatch/fuzzystrmatch.c
index 72ae2ab91b5..f243e2caed5 100644
--- a/contrib/fuzzystrmatch/fuzzystrmatch.c
+++ b/contrib/fuzzystrmatch/fuzzystrmatch.c
@@ -40,7 +40,6 @@
#include <ctype.h>
-#include "mb/pg_wchar.h"
#include "utils/builtins.h"
#include "utils/varlena.h"
#include "varatt.h"
diff --git a/contrib/intarray/_int_bool.c b/contrib/intarray/_int_bool.c
index 8fc6ad87fc7..8cf02c0e5da 100644
--- a/contrib/intarray/_int_bool.c
+++ b/contrib/intarray/_int_bool.c
@@ -5,7 +5,6 @@
#include "_int.h"
#include "miscadmin.h"
-#include "utils/builtins.h"
PG_FUNCTION_INFO_V1(bqarr_in);
PG_FUNCTION_INFO_V1(bqarr_out);
diff --git a/contrib/intarray/_int_selfuncs.c b/contrib/intarray/_int_selfuncs.c
index bc1ad3a8048..99179d0c6fc 100644
--- a/contrib/intarray/_int_selfuncs.c
+++ b/contrib/intarray/_int_selfuncs.c
@@ -20,10 +20,9 @@
#include "catalog/pg_statistic.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
-#include "utils/syscache.h"
PG_FUNCTION_INFO_V1(_int_overlap_sel);
PG_FUNCTION_INFO_V1(_int_contains_sel);
diff --git a/contrib/isn/isn.c b/contrib/isn/isn.c
index 71a543b9478..15bcda200c5 100644
--- a/contrib/isn/isn.c
+++ b/contrib/isn/isn.c
@@ -21,7 +21,6 @@
#include "UPC.h"
#include "fmgr.h"
#include "isn.h"
-#include "utils/builtins.h"
PG_MODULE_MAGIC;
diff --git a/contrib/lo/lo.c b/contrib/lo/lo.c
index 457be26c4ed..62488e45f3a 100644
--- a/contrib/lo/lo.c
+++ b/contrib/lo/lo.c
@@ -9,7 +9,7 @@
#include "commands/trigger.h"
#include "executor/spi.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
PG_MODULE_MAGIC;
diff --git a/contrib/ltree/ltree_io.c b/contrib/ltree/ltree_io.c
index 0a12c77a621..11eefc809b2 100644
--- a/contrib/ltree/ltree_io.c
+++ b/contrib/ltree/ltree_io.c
@@ -10,7 +10,6 @@
#include "crc32.h"
#include "libpq/pqformat.h"
#include "ltree.h"
-#include "utils/memutils.h"
#include "varatt.h"
diff --git a/contrib/ltree/ltree_op.c b/contrib/ltree/ltree_op.c
index 24a21d3ea01..0e30dee4658 100644
--- a/contrib/ltree/ltree_op.c
+++ b/contrib/ltree/ltree_op.c
@@ -7,13 +7,11 @@
#include <ctype.h>
-#include "access/htup_details.h"
-#include "catalog/pg_statistic.h"
#include "common/hashfn.h"
#include "ltree.h"
#include "utils/builtins.h"
-#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
+#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 5a38d926689..22621d584fa 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -9,20 +9,16 @@
*/
#include "postgres.h"
-#include "access/brin.h"
#include "access/brin_internal.h"
#include "access/brin_page.h"
-#include "access/brin_revmap.h"
#include "access/brin_tuple.h"
#include "access/htup_details.h"
-#include "catalog/index.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
#include "pageinspect.h"
-#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
diff --git a/contrib/pageinspect/fsmfuncs.c b/contrib/pageinspect/fsmfuncs.c
index f8ae0886244..31182144380 100644
--- a/contrib/pageinspect/fsmfuncs.c
+++ b/contrib/pageinspect/fsmfuncs.c
@@ -19,7 +19,7 @@
#include "postgres.h"
-#include "funcapi.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
#include "pageinspect.h"
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index 1dabbdb5889..7182d225d96 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -9,17 +9,14 @@
*/
#include "postgres.h"
-#include "access/gin.h"
#include "access/gin_private.h"
#include "access/htup_details.h"
-#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "pageinspect.h"
#include "utils/array.h"
#include "utils/builtins.h"
-#include "utils/rel.h"
PG_FUNCTION_INFO_V1(gin_metapage_info);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index 1cc3b8c9e90..029072567a4 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -10,10 +10,8 @@
#include "postgres.h"
#include "access/gist.h"
-#include "access/gist_private.h"
#include "access/htup.h"
#include "access/relation.h"
-#include "catalog/namespace.h"
#include "catalog/pg_am_d.h"
#include "funcapi.h"
#include "miscadmin.h"
@@ -25,7 +23,6 @@
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/ruleutils.h"
-#include "utils/varlena.h"
PG_FUNCTION_INFO_V1(gist_page_opaque_info);
PG_FUNCTION_INFO_V1(gist_page_items);
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 38a539dad1b..3dc705e43f7 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -32,7 +32,6 @@
#include "funcapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pageinspect.h"
#include "port/pg_bitutils.h"
#include "utils/array.h"
#include "utils/builtins.h"
diff --git a/contrib/pg_freespacemap/pg_freespacemap.c b/contrib/pg_freespacemap/pg_freespacemap.c
index b82cab2d97e..565167aaef0 100644
--- a/contrib/pg_freespacemap/pg_freespacemap.c
+++ b/contrib/pg_freespacemap/pg_freespacemap.c
@@ -9,7 +9,7 @@
#include "postgres.h"
#include "access/relation.h"
-#include "funcapi.h"
+#include "fmgr.h"
#include "storage/freespace.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index d061731706a..fac4051e1aa 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -30,8 +30,6 @@
#include "access/relation.h"
#include "access/xact.h"
-#include "catalog/pg_class.h"
-#include "catalog/pg_type.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -42,18 +40,13 @@
#include "storage/ipc.h"
#include "storage/latch.h"
#include "storage/lwlock.h"
-#include "storage/proc.h"
#include "storage/procsignal.h"
-#include "storage/shmem.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h"
-#include "utils/acl.h"
-#include "utils/datetime.h"
#include "utils/guc.h"
-#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/relfilenumbermap.h"
-#include "utils/resowner.h"
+#include "utils/timestamp.h"
#define AUTOPREWARM_FILE "autoprewarm.blocks"
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 21b26b7b6ec..1798e1d016f 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -49,7 +49,6 @@
#include "access/parallel.h"
#include "catalog/pg_authid.h"
-#include "common/hashfn.h"
#include "common/int.h"
#include "executor/instrument.h"
#include "funcapi.h"
@@ -59,9 +58,7 @@
#include "nodes/queryjumble.h"
#include "optimizer/planner.h"
#include "parser/analyze.h"
-#include "parser/parsetree.h"
#include "parser/scanner.h"
-#include "parser/scansup.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/ipc.h"
diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index 37dffe3f7d5..1513d76e4b5 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -12,11 +12,11 @@
*/
#include "postgres.h"
-#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
#include "catalog/pg_am_d.h"
-#include "catalog/pg_proc_d.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "utils/acl.h"
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 724122b1bc5..5d0deaba61e 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -19,12 +19,10 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
#include "storage/read_stream.h"
#include "storage/smgr.h"
#include "utils/rel.h"
-#include "utils/snapmgr.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pgcrypto/pgcrypto.c b/contrib/pgcrypto/pgcrypto.c
index 96447c57577..ebd76eed702 100644
--- a/contrib/pgcrypto/pgcrypto.c
+++ b/contrib/pgcrypto/pgcrypto.c
@@ -38,7 +38,6 @@
#include "px-crypt.h"
#include "px.h"
#include "utils/builtins.h"
-#include "utils/uuid.h"
#include "varatt.h"
PG_MODULE_MAGIC;
diff --git a/contrib/pgrowlocks/pgrowlocks.c b/contrib/pgrowlocks/pgrowlocks.c
index adbc8279c3f..7e40ab21dda 100644
--- a/contrib/pgrowlocks/pgrowlocks.c
+++ b/contrib/pgrowlocks/pgrowlocks.c
@@ -37,7 +37,7 @@
#include "storage/bufmgr.h"
#include "storage/procarray.h"
#include "utils/acl.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
#include "utils/varlena.h"
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 04457f4b790..3a5aad30ae2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -14,21 +14,15 @@
#include "access/heapam.h"
#include "access/htup_details.h"
-#include "access/multixact.h"
#include "access/relation.h"
-#include "access/transam.h"
#include "access/visibilitymap.h"
-#include "access/xact.h"
-#include "catalog/namespace.h"
#include "catalog/pg_am_d.h"
#include "commands/vacuum.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/freespace.h"
-#include "storage/lmgr.h"
#include "storage/procarray.h"
-#include "utils/builtins.h"
PG_FUNCTION_INFO_V1(pgstattuple_approx);
PG_FUNCTION_INFO_V1(pgstattuple_approx_v1_5);
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 1b6b768cf80..4b9d76ec4e4 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -32,14 +32,11 @@
#include "access/htup_details.h"
#include "access/nbtree.h"
#include "access/relation.h"
-#include "access/table.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "storage/bufmgr.h"
-#include "storage/lmgr.h"
-#include "utils/builtins.h"
#include "utils/rel.h"
#include "utils/varlena.h"
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 1fff762753b..48cb8f59c4f 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -36,7 +36,6 @@
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/lmgr.h"
-#include "utils/builtins.h"
#include "utils/varlena.h"
PG_MODULE_MAGIC;
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 2e5303eac12..2326f391d34 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -16,7 +16,6 @@
#include <poll.h>
#endif
-#include "access/htup_details.h"
#include "access/xact.h"
#include "catalog/pg_user_mapping.h"
#include "commands/defrem.h"
@@ -27,13 +26,10 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postgres_fdw.h"
-#include "storage/fd.h"
#include "storage/latch.h"
#include "utils/builtins.h"
-#include "utils/datetime.h"
#include "utils/hsearch.h"
#include "utils/inval.h"
-#include "utils/memutils.h"
#include "utils/syscache.h"
/*
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index fb590c87e67..4680d517331 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -47,8 +47,6 @@
#include "catalog/pg_ts_dict.h"
#include "catalog/pg_type.h"
#include "commands/defrem.h"
-#include "commands/tablecmds.h"
-#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "nodes/plannodes.h"
#include "optimizer/optimizer.h"
diff --git a/contrib/postgres_fdw/option.c b/contrib/postgres_fdw/option.c
index d740893918c..ed11126c09e 100644
--- a/contrib/postgres_fdw/option.c
+++ b/contrib/postgres_fdw/option.c
@@ -20,7 +20,6 @@
#include "commands/extension.h"
#include "libpq/libpq-be.h"
#include "postgres_fdw.h"
-#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/varlena.h"
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index adc62576d1f..53733d642d0 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -17,11 +17,9 @@
#include "access/htup_details.h"
#include "access/sysattr.h"
#include "access/table.h"
-#include "catalog/pg_class.h"
#include "catalog/pg_opfamily.h"
#include "commands/defrem.h"
#include "commands/explain.h"
-#include "commands/vacuum.h"
#include "executor/execAsync.h"
#include "foreign/fdwapi.h"
#include "funcapi.h"
@@ -29,7 +27,6 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/appendinfo.h"
-#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/inherit.h"
#include "optimizer/optimizer.h"
diff --git a/contrib/spi/moddatetime.c b/contrib/spi/moddatetime.c
index 3eb7004de97..5130804ce2a 100644
--- a/contrib/spi/moddatetime.c
+++ b/contrib/spi/moddatetime.c
@@ -19,7 +19,7 @@ OH, me, I'm Terry Mackintosh <[email protected]>
#include "catalog/pg_type.h"
#include "commands/trigger.h"
#include "executor/spi.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
PG_MODULE_MAGIC;
diff --git a/contrib/tablefunc/tablefunc.c b/contrib/tablefunc/tablefunc.c
index 2a25607a2ad..d41b353a6de 100644
--- a/contrib/tablefunc/tablefunc.c
+++ b/contrib/tablefunc/tablefunc.c
@@ -38,6 +38,7 @@
#include "catalog/pg_type.h"
#include "common/pg_prng.h"
#include "executor/spi.h"
+#include "fmgr.h"
#include "funcapi.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
diff --git a/contrib/tablefunc/tablefunc.h b/contrib/tablefunc/tablefunc.h
index 2009382ce7d..b78030044b5 100644
--- a/contrib/tablefunc/tablefunc.h
+++ b/contrib/tablefunc/tablefunc.h
@@ -34,6 +34,4 @@
#ifndef TABLEFUNC_H
#define TABLEFUNC_H
-#include "fmgr.h"
-
#endif /* TABLEFUNC_H */
diff --git a/contrib/tsm_system_rows/tsm_system_rows.c b/contrib/tsm_system_rows/tsm_system_rows.c
index a9f5d657d1c..ecfd1102d7a 100644
--- a/contrib/tsm_system_rows/tsm_system_rows.c
+++ b/contrib/tsm_system_rows/tsm_system_rows.c
@@ -28,7 +28,6 @@
#include "postgres.h"
-#include "access/relscan.h"
#include "access/tsmapi.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
diff --git a/contrib/tsm_system_time/tsm_system_time.c b/contrib/tsm_system_time/tsm_system_time.c
index f187fd77655..093682a9155 100644
--- a/contrib/tsm_system_time/tsm_system_time.c
+++ b/contrib/tsm_system_time/tsm_system_time.c
@@ -26,7 +26,6 @@
#include <math.h>
-#include "access/relscan.h"
#include "access/tsmapi.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
diff --git a/contrib/unaccent/unaccent.c b/contrib/unaccent/unaccent.c
index 707962305f8..0217696aac1 100644
--- a/contrib/unaccent/unaccent.c
+++ b/contrib/unaccent/unaccent.c
@@ -13,7 +13,6 @@
#include "postgres.h"
-#include "catalog/namespace.h"
#include "catalog/pg_ts_dict.h"
#include "commands/defrem.h"
#include "lib/stringinfo.h"
@@ -22,7 +21,6 @@
#include "tsearch/ts_public.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-#include "utils/regproc.h"
#include "utils/syscache.h"
PG_MODULE_MAGIC;
diff --git a/contrib/xml2/xpath.c b/contrib/xml2/xpath.c
index ef78aa00c88..f7e3f485fe1 100644
--- a/contrib/xml2/xpath.c
+++ b/contrib/xml2/xpath.c
@@ -11,7 +11,6 @@
#include "fmgr.h"
#include "funcapi.h"
#include "lib/stringinfo.h"
-#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/xml.h"
diff --git a/contrib/xml2/xslt_proc.c b/contrib/xml2/xslt_proc.c
index e761ca5cb5f..b720d89f754 100644
--- a/contrib/xml2/xslt_proc.c
+++ b/contrib/xml2/xslt_proc.c
@@ -7,10 +7,7 @@
*/
#include "postgres.h"
-#include "executor/spi.h"
#include "fmgr.h"
-#include "funcapi.h"
-#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/xml.h"
--
2.47.0
[text/plain] 0003-Remove-unused-include-s-from-pl-.c-files.patch (6.8K, ../../[email protected]/4-0003-Remove-unused-include-s-from-pl-.c-files.patch)
download | inline diff:
From 4a547fed9b6c09b396861f195e8fefff76040c84 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 21 Sep 2024 22:25:52 +0200
Subject: [PATCH 3/5] Remove unused #include's from pl .c files
as determined by IWYU
---
src/pl/plperl/plperl.c | 1 -
src/pl/plpgsql/src/pl_comp.c | 3 +--
src/pl/plpgsql/src/pl_exec.c | 5 -----
src/pl/plpython/plpy_cursorobject.c | 2 --
src/pl/plpython/plpy_exec.c | 4 +---
src/pl/plpython/plpy_planobject.c | 1 -
src/pl/plpython/plpy_plpymodule.c | 3 ---
src/pl/plpython/plpy_procedure.c | 3 ---
src/pl/plpython/plpy_spi.c | 3 ---
src/pl/plpython/plpy_util.c | 1 -
src/pl/tcl/pltcl.c | 3 +--
11 files changed, 3 insertions(+), 26 deletions(-)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index fe719935c67..1b1677e333b 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -24,7 +24,6 @@
#include "executor/spi.h"
#include "funcapi.h"
#include "miscadmin.h"
-#include "nodes/makefuncs.h"
#include "parser/parse_type.h"
#include "storage/ipc.h"
#include "tcop/tcopprot.h"
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index f1bce708d62..5633e3c7905 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -23,7 +23,7 @@
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "nodes/makefuncs.h"
-#include "parser/parse_type.h"
+#include "parser/parse_node.h"
#include "plpgsql.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
@@ -31,7 +31,6 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/regproc.h"
-#include "utils/rel.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index ea9740e3f89..b51b3d13a08 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -19,11 +19,9 @@
#include "access/detoast.h"
#include "access/htup_details.h"
-#include "access/transam.h"
#include "access/tupconvert.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
-#include "commands/defrem.h"
#include "executor/execExpr.h"
#include "executor/spi.h"
#include "executor/tstoreReceiver.h"
@@ -34,13 +32,10 @@
#include "optimizer/optimizer.h"
#include "parser/parse_coerce.h"
#include "parser/parse_type.h"
-#include "parser/scansup.h"
#include "plpgsql.h"
#include "storage/proc.h"
#include "tcop/cmdtag.h"
#include "tcop/pquery.h"
-#include "tcop/tcopprot.h"
-#include "tcop/utility.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 24f2ac8c46b..6108384c9a5 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -8,14 +8,12 @@
#include <limits.h>
-#include "access/xact.h"
#include "catalog/pg_type.h"
#include "mb/pg_wchar.h"
#include "plpy_cursorobject.h"
#include "plpy_elog.h"
#include "plpy_main.h"
#include "plpy_planobject.h"
-#include "plpy_procedure.h"
#include "plpy_resultobject.h"
#include "plpy_spi.h"
#include "plpython.h"
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 157229e96f6..9d59473d6ad 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -18,10 +18,8 @@
#include "plpy_procedure.h"
#include "plpy_subxactobject.h"
#include "plpython.h"
-#include "utils/builtins.h"
-#include "utils/lsyscache.h"
+#include "utils/fmgrprotos.h"
#include "utils/rel.h"
-#include "utils/typcache.h"
/* saved state for a set-returning function */
typedef struct PLySRFState
diff --git a/src/pl/plpython/plpy_planobject.c b/src/pl/plpython/plpy_planobject.c
index ec2439c6a1f..bbef889329e 100644
--- a/src/pl/plpython/plpy_planobject.c
+++ b/src/pl/plpython/plpy_planobject.c
@@ -7,7 +7,6 @@
#include "postgres.h"
#include "plpy_cursorobject.h"
-#include "plpy_elog.h"
#include "plpy_planobject.h"
#include "plpy_spi.h"
#include "plpython.h"
diff --git a/src/pl/plpython/plpy_plpymodule.c b/src/pl/plpython/plpy_plpymodule.c
index fa08f0dbfb3..ea06d9a52b1 100644
--- a/src/pl/plpython/plpy_plpymodule.c
+++ b/src/pl/plpython/plpy_plpymodule.c
@@ -6,11 +6,9 @@
#include "postgres.h"
-#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "plpy_cursorobject.h"
#include "plpy_elog.h"
-#include "plpy_main.h"
#include "plpy_planobject.h"
#include "plpy_plpymodule.h"
#include "plpy_resultobject.h"
@@ -18,7 +16,6 @@
#include "plpy_subxactobject.h"
#include "plpython.h"
#include "utils/builtins.h"
-#include "utils/snapmgr.h"
HTAB *PLy_spi_exceptions = NULL;
diff --git a/src/pl/plpython/plpy_procedure.c b/src/pl/plpython/plpy_procedure.c
index ba7786d31c3..c35a3b801ab 100644
--- a/src/pl/plpython/plpy_procedure.c
+++ b/src/pl/plpython/plpy_procedure.c
@@ -7,7 +7,6 @@
#include "postgres.h"
#include "access/htup_details.h"
-#include "access/transam.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
@@ -17,8 +16,6 @@
#include "plpython.h"
#include "utils/builtins.h"
#include "utils/hsearch.h"
-#include "utils/inval.h"
-#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/syscache.h"
diff --git a/src/pl/plpython/plpy_spi.c b/src/pl/plpython/plpy_spi.c
index ff87b27de09..bcbd07b70ae 100644
--- a/src/pl/plpython/plpy_spi.c
+++ b/src/pl/plpython/plpy_spi.c
@@ -8,7 +8,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "executor/spi.h"
@@ -18,12 +17,10 @@
#include "plpy_main.h"
#include "plpy_planobject.h"
#include "plpy_plpymodule.h"
-#include "plpy_procedure.h"
#include "plpy_resultobject.h"
#include "plpy_spi.h"
#include "plpython.h"
#include "utils/memutils.h"
-#include "utils/syscache.h"
static PyObject *PLy_spi_execute_query(char *query, long limit);
static PyObject *PLy_spi_execute_fetch_result(SPITupleTable *tuptable,
diff --git a/src/pl/plpython/plpy_util.c b/src/pl/plpython/plpy_util.c
index 22e2a599ad9..6d89b1cb60a 100644
--- a/src/pl/plpython/plpy_util.c
+++ b/src/pl/plpython/plpy_util.c
@@ -10,7 +10,6 @@
#include "plpy_elog.h"
#include "plpy_util.h"
#include "plpython.h"
-#include "utils/memutils.h"
/*
* Convert a Python unicode object to a Python string/bytes object in
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e2ccaa84f3f..80de0db40de 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -25,13 +25,12 @@
#include "funcapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/makefuncs.h"
#include "parser/parse_func.h"
#include "parser/parse_type.h"
#include "pgstat.h"
-#include "tcop/tcopprot.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/guc.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/regproc.h"
--
2.47.0
[text/plain] 0004-Remove-unused-include-s-from-src-test-.c-files.patch (10.1K, ../../[email protected]/5-0004-Remove-unused-include-s-from-src-test-.c-files.patch)
download | inline diff:
From 95c5e4eef4e914db1d44a80ca56733b08fd59054 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 21 Sep 2024 22:24:01 +0200
Subject: [PATCH 4/5] Remove unused #include's from src/test/ .c files
as determined by IWYU
---
src/test/modules/delay_execution/delay_execution.c | 2 +-
src/test/modules/dummy_index_am/dummy_index_am.c | 2 --
src/test/modules/ldap_password_func/ldap_password_func.c | 4 +---
src/test/modules/libpq_pipeline/libpq_pipeline.c | 2 --
src/test/modules/plsample/plsample.c | 2 +-
src/test/modules/test_ddl_deparse/test_ddl_deparse.c | 1 -
src/test/modules/test_integerset/test_integerset.c | 4 ----
src/test/modules/test_oat_hooks/test_oat_hooks.c | 1 -
src/test/modules/test_radixtree/test_radixtree.c | 2 --
src/test/modules/test_regex/test_regex.c | 1 -
src/test/modules/test_resowner/test_resowner_basic.c | 2 --
src/test/modules/test_resowner/test_resowner_many.c | 1 -
src/test/modules/test_rls_hooks/test_rls_hooks.c | 1 -
src/test/modules/test_shm_mq/setup.c | 1 -
src/test/modules/test_slru/test_multixact.c | 2 +-
src/test/modules/test_tidstore/test_tidstore.c | 1 -
src/test/modules/worker_spi/worker_spi.c | 4 ----
src/test/regress/pg_regress.c | 1 -
src/test/regress/regress.c | 3 ---
19 files changed, 4 insertions(+), 33 deletions(-)
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 155c8a8d556..fa4693a3f57 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -23,7 +23,7 @@
#include <limits.h>
#include "optimizer/planner.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/guc.h"
#include "utils/inval.h"
diff --git a/src/test/modules/dummy_index_am/dummy_index_am.c b/src/test/modules/dummy_index_am/dummy_index_am.c
index 2841cf2eb4b..beb2c1d2542 100644
--- a/src/test/modules/dummy_index_am/dummy_index_am.c
+++ b/src/test/modules/dummy_index_am/dummy_index_am.c
@@ -18,8 +18,6 @@
#include "catalog/index.h"
#include "commands/vacuum.h"
#include "nodes/pathnodes.h"
-#include "utils/guc.h"
-#include "utils/rel.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/ldap_password_func/ldap_password_func.c b/src/test/modules/ldap_password_func/ldap_password_func.c
index 24d9c637818..84057d955aa 100644
--- a/src/test/modules/ldap_password_func/ldap_password_func.c
+++ b/src/test/modules/ldap_password_func/ldap_password_func.c
@@ -15,10 +15,8 @@
#include <float.h>
#include <stdio.h>
+#include "fmgr.h"
#include "libpq/auth.h"
-#include "libpq/libpq.h"
-#include "libpq/libpq-be.h"
-#include "utils/guc.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c
index 999a7f57a7e..1323e4c598d 100644
--- a/src/test/modules/libpq_pipeline/libpq_pipeline.c
+++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c
@@ -19,10 +19,8 @@
#include <sys/time.h>
#include "catalog/pg_type_d.h"
-#include "common/fe_memutils.h"
#include "libpq-fe.h"
#include "pg_getopt.h"
-#include "portability/instr_time.h"
static void exit_nicely(PGconn *conn);
diff --git a/src/test/modules/plsample/plsample.c b/src/test/modules/plsample/plsample.c
index 89ea166a676..25f7814f1f7 100644
--- a/src/test/modules/plsample/plsample.c
+++ b/src/test/modules/plsample/plsample.c
@@ -21,7 +21,7 @@
#include "commands/trigger.h"
#include "executor/spi.h"
#include "funcapi.h"
-#include "utils/builtins.h"
+#include "utils/fmgrprotos.h"
#include "utils/lsyscache.h"
#include "utils/syscache.h"
diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c
index 2758ae82d7b..97cf52d133c 100644
--- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c
+++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c
@@ -10,7 +10,6 @@
*/
#include "postgres.h"
-#include "catalog/pg_type.h"
#include "funcapi.h"
#include "nodes/execnodes.h"
#include "tcop/deparse_utility.h"
diff --git a/src/test/modules/test_integerset/test_integerset.c b/src/test/modules/test_integerset/test_integerset.c
index 61f8d0b663a..73a97b81167 100644
--- a/src/test/modules/test_integerset/test_integerset.c
+++ b/src/test/modules/test_integerset/test_integerset.c
@@ -15,10 +15,6 @@
#include "common/pg_prng.h"
#include "fmgr.h"
#include "lib/integerset.h"
-#include "miscadmin.h"
-#include "nodes/bitmapset.h"
-#include "storage/block.h"
-#include "storage/itemptr.h"
#include "utils/memutils.h"
#include "utils/timestamp.h"
diff --git a/src/test/modules/test_oat_hooks/test_oat_hooks.c b/src/test/modules/test_oat_hooks/test_oat_hooks.c
index 05c33712254..ca4aa7e6696 100644
--- a/src/test/modules/test_oat_hooks/test_oat_hooks.c
+++ b/src/test/modules/test_oat_hooks/test_oat_hooks.c
@@ -16,7 +16,6 @@
#include "access/parallel.h"
#include "catalog/dependency.h"
#include "catalog/objectaccess.h"
-#include "catalog/pg_proc.h"
#include "executor/executor.h"
#include "fmgr.h"
#include "miscadmin.h"
diff --git a/src/test/modules/test_radixtree/test_radixtree.c b/src/test/modules/test_radixtree/test_radixtree.c
index 1d9165a3a23..3e072fa5ec6 100644
--- a/src/test/modules/test_radixtree/test_radixtree.c
+++ b/src/test/modules/test_radixtree/test_radixtree.c
@@ -15,8 +15,6 @@
#include "common/int.h"
#include "common/pg_prng.h"
#include "fmgr.h"
-#include "miscadmin.h"
-#include "storage/lwlock.h"
#include "utils/memutils.h"
#include "utils/timestamp.h"
diff --git a/src/test/modules/test_regex/test_regex.c b/src/test/modules/test_regex/test_regex.c
index c4ef15d7425..cd503a2cd81 100644
--- a/src/test/modules/test_regex/test_regex.c
+++ b/src/test/modules/test_regex/test_regex.c
@@ -15,7 +15,6 @@
#include "postgres.h"
#include "funcapi.h"
-#include "miscadmin.h"
#include "regex/regex.h"
#include "utils/array.h"
#include "utils/builtins.h"
diff --git a/src/test/modules/test_resowner/test_resowner_basic.c b/src/test/modules/test_resowner/test_resowner_basic.c
index 558cf1de56e..1d9c1cd0259 100644
--- a/src/test/modules/test_resowner/test_resowner_basic.c
+++ b/src/test/modules/test_resowner/test_resowner_basic.c
@@ -13,8 +13,6 @@
#include "postgres.h"
#include "fmgr.h"
-#include "lib/ilist.h"
-#include "utils/memutils.h"
#include "utils/resowner.h"
PG_MODULE_MAGIC;
diff --git a/src/test/modules/test_resowner/test_resowner_many.c b/src/test/modules/test_resowner/test_resowner_many.c
index 7808c9e5df3..4f3f5ebf87e 100644
--- a/src/test/modules/test_resowner/test_resowner_many.c
+++ b/src/test/modules/test_resowner/test_resowner_many.c
@@ -14,7 +14,6 @@
#include "fmgr.h"
#include "lib/ilist.h"
-#include "utils/memutils.h"
#include "utils/resowner.h"
/*
diff --git a/src/test/modules/test_rls_hooks/test_rls_hooks.c b/src/test/modules/test_rls_hooks/test_rls_hooks.c
index 358e28edb45..d18d5815cfe 100644
--- a/src/test/modules/test_rls_hooks/test_rls_hooks.c
+++ b/src/test/modules/test_rls_hooks/test_rls_hooks.c
@@ -15,7 +15,6 @@
#include "catalog/pg_type.h"
#include "fmgr.h"
-#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
diff --git a/src/test/modules/test_shm_mq/setup.c b/src/test/modules/test_shm_mq/setup.c
index b3dac44d97a..fb235604394 100644
--- a/src/test/modules/test_shm_mq/setup.c
+++ b/src/test/modules/test_shm_mq/setup.c
@@ -18,7 +18,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
-#include "storage/procsignal.h"
#include "storage/shm_toc.h"
#include "test_shm_mq.h"
#include "utils/memutils.h"
diff --git a/src/test/modules/test_slru/test_multixact.c b/src/test/modules/test_slru/test_multixact.c
index 22cc441a385..1cc89aa116c 100644
--- a/src/test/modules/test_slru/test_multixact.c
+++ b/src/test/modules/test_slru/test_multixact.c
@@ -16,7 +16,7 @@
#include "access/multixact.h"
#include "access/xact.h"
-#include "utils/builtins.h"
+#include "fmgr.h"
#include "utils/injection_point.h"
PG_FUNCTION_INFO_V1(test_create_multixact);
diff --git a/src/test/modules/test_tidstore/test_tidstore.c b/src/test/modules/test_tidstore/test_tidstore.c
index 94ddcf1de82..6da1cb24527 100644
--- a/src/test/modules/test_tidstore/test_tidstore.c
+++ b/src/test/modules/test_tidstore/test_tidstore.c
@@ -18,7 +18,6 @@
#include "access/tidstore.h"
#include "fmgr.h"
-#include "funcapi.h"
#include "storage/block.h"
#include "storage/itemptr.h"
#include "storage/lwlock.h"
diff --git a/src/test/modules/worker_spi/worker_spi.c b/src/test/modules/worker_spi/worker_spi.c
index de8f46902b4..d4403b24d98 100644
--- a/src/test/modules/worker_spi/worker_spi.c
+++ b/src/test/modules/worker_spi/worker_spi.c
@@ -26,11 +26,7 @@
#include "miscadmin.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
-#include "storage/ipc.h"
#include "storage/latch.h"
-#include "storage/lwlock.h"
-#include "storage/proc.h"
-#include "storage/shmem.h"
/* these headers are used by this particular worker's code */
#include "access/xact.h"
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 6c188954b14..0e40ed32a21 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -28,7 +28,6 @@
#include "common/logging.h"
#include "common/restricted_token.h"
-#include "common/string.h"
#include "common/username.h"
#include "getopt_long.h"
#include "lib/stringinfo.h"
diff --git a/src/test/regress/regress.c b/src/test/regress/regress.c
index 9e81371be4b..8bae56a9777 100644
--- a/src/test/regress/regress.c
+++ b/src/test/regress/regress.c
@@ -21,8 +21,6 @@
#include "access/detoast.h"
#include "access/htup_details.h"
-#include "access/transam.h"
-#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
@@ -39,7 +37,6 @@
#include "parser/parse_coerce.h"
#include "port/atomics.h"
#include "storage/spin.h"
-#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/geo_decls.h"
#include "utils/memutils.h"
--
2.47.0
[text/plain] 0005-Remove-unused-include-s-from-bin-.c-files.patch (18.5K, ../../[email protected]/6-0005-Remove-unused-include-s-from-bin-.c-files.patch)
download | inline diff:
From c0133bb5b7573d0ec4d94b2f5f3d04e0261329ac Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sat, 22 Jun 2024 10:27:40 +0200
Subject: [PATCH 5/5] Remove unused #include's from bin .c files
as determined by IWYU
---
src/bin/pg_basebackup/pg_createsubscriber.c | 3 +--
src/bin/pg_basebackup/pg_recvlogical.c | 2 --
src/bin/pg_basebackup/receivelog.c | 1 -
src/bin/pg_basebackup/streamutil.c | 2 --
src/bin/pg_basebackup/walmethods.c | 4 ++--
src/bin/pg_checksums/pg_checksums.c | 1 -
src/bin/pg_combinebackup/copy_file.h | 1 -
src/bin/pg_combinebackup/pg_combinebackup.c | 2 +-
src/bin/pg_config/pg_config.c | 1 -
src/bin/pg_dump/common.c | 2 --
src/bin/pg_dump/compress_io.c | 1 -
src/bin/pg_dump/filter.c | 1 -
src/bin/pg_dump/parallel.c | 2 ++
src/bin/pg_dump/pg_backup_custom.c | 1 -
src/bin/pg_dump/pg_backup_db.c | 2 --
src/bin/pg_dump/pg_backup_utils.c | 2 ++
src/bin/pg_dump/pg_dump.c | 2 --
src/bin/pg_dump/pg_dump_sort.c | 1 -
src/bin/pg_dump/pg_restore.c | 1 -
src/bin/pg_rewind/datapagemap.h | 1 -
src/bin/pg_rewind/libpq_source.c | 1 -
src/bin/pg_rewind/local_source.c | 4 +---
src/bin/pg_rewind/pg_rewind.h | 4 ----
src/bin/pg_test_fsync/pg_test_fsync.c | 1 -
src/bin/pg_upgrade/check.c | 2 --
src/bin/pg_upgrade/relfilenumber.c | 2 --
src/bin/pg_walsummary/pg_walsummary.c | 1 -
src/bin/psql/command.c | 3 ---
src/bin/psql/describe.c | 1 -
src/bin/psql/help.c | 3 ---
src/bin/scripts/clusterdb.c | 1 -
src/bin/scripts/common.c | 1 -
src/bin/scripts/reindexdb.c | 1 -
src/common/controldata_utils.c | 2 +-
src/common/string.c | 1 -
src/common/unicode_case.c | 1 -
src/fe_utils/astreamer_file.c | 1 -
src/fe_utils/astreamer_gzip.c | 2 --
src/fe_utils/astreamer_lz4.c | 2 --
src/fe_utils/cancel.c | 2 --
40 files changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index e804b2a63ef..e96370a9ec1 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -13,14 +13,13 @@
#include "postgres_fe.h"
+#include <sys/stat.h>
#include <sys/time.h>
#include <sys/wait.h>
#include <time.h>
-#include "catalog/pg_authid_d.h"
#include "common/connect.h"
#include "common/controldata_utils.h"
-#include "common/file_perm.h"
#include "common/logging.h"
#include "common/pg_prng.h"
#include "common/restricted_token.h"
diff --git a/src/bin/pg_basebackup/pg_recvlogical.c b/src/bin/pg_basebackup/pg_recvlogical.c
index 3db520ed38b..42b2a7bb9d8 100644
--- a/src/bin/pg_basebackup/pg_recvlogical.c
+++ b/src/bin/pg_basebackup/pg_recvlogical.c
@@ -18,8 +18,6 @@
#include <sys/stat.h>
#include <unistd.h>
-#include "access/xlog_internal.h"
-#include "common/fe_memutils.h"
#include "common/file_perm.h"
#include "common/logging.h"
#include "fe_utils/option_utils.h"
diff --git a/src/bin/pg_basebackup/receivelog.c b/src/bin/pg_basebackup/receivelog.c
index 70f42467649..9fe90cb38fd 100644
--- a/src/bin/pg_basebackup/receivelog.c
+++ b/src/bin/pg_basebackup/receivelog.c
@@ -19,7 +19,6 @@
#include <unistd.h>
#include "access/xlog_internal.h"
-#include "common/file_utils.h"
#include "common/logging.h"
#include "libpq-fe.h"
#include "receivelog.h"
diff --git a/src/bin/pg_basebackup/streamutil.c b/src/bin/pg_basebackup/streamutil.c
index 30b3d9a377e..3f4d157d583 100644
--- a/src/bin/pg_basebackup/streamutil.c
+++ b/src/bin/pg_basebackup/streamutil.c
@@ -19,14 +19,12 @@
#include "access/xlog_internal.h"
#include "common/connect.h"
-#include "common/fe_memutils.h"
#include "common/file_perm.h"
#include "common/logging.h"
#include "common/string.h"
#include "datatype/timestamp.h"
#include "port/pg_bswap.h"
#include "pqexpbuffer.h"
-#include "receivelog.h"
#include "streamutil.h"
#define ERRCODE_DUPLICATE_OBJECT "42710"
diff --git a/src/bin/pg_basebackup/walmethods.c b/src/bin/pg_basebackup/walmethods.c
index 832aafd2973..215b24597fe 100644
--- a/src/bin/pg_basebackup/walmethods.c
+++ b/src/bin/pg_basebackup/walmethods.c
@@ -11,6 +11,7 @@
#include "postgres_fe.h"
+#include <fcntl.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
@@ -26,8 +27,7 @@
#include "common/file_utils.h"
#include "common/logging.h"
#include "pgtar.h"
-#include "receivelog.h"
-#include "streamutil.h"
+#include "walmethods.h"
/* Size of zlib buffer for .tar.gz */
#define ZLIB_OUT_SIZE 4096
diff --git a/src/bin/pg_checksums/pg_checksums.c b/src/bin/pg_checksums/pg_checksums.c
index f5f7ff10455..b86bc417c9c 100644
--- a/src/bin/pg_checksums/pg_checksums.c
+++ b/src/bin/pg_checksums/pg_checksums.c
@@ -21,7 +21,6 @@
#include <unistd.h>
#include "common/controldata_utils.h"
-#include "common/file_perm.h"
#include "common/file_utils.h"
#include "common/logging.h"
#include "common/relpath.h"
diff --git a/src/bin/pg_combinebackup/copy_file.h b/src/bin/pg_combinebackup/copy_file.h
index cedb7607385..d5ddc7e53d7 100644
--- a/src/bin/pg_combinebackup/copy_file.h
+++ b/src/bin/pg_combinebackup/copy_file.h
@@ -13,7 +13,6 @@
#include "c.h"
#include "common/checksum_helper.h"
-#include "common/file_utils.h"
/*
* Enumeration to denote copy modes.
diff --git a/src/bin/pg_combinebackup/pg_combinebackup.c b/src/bin/pg_combinebackup/pg_combinebackup.c
index 6183d317151..5f1f62f1db6 100644
--- a/src/bin/pg_combinebackup/pg_combinebackup.c
+++ b/src/bin/pg_combinebackup/pg_combinebackup.c
@@ -25,12 +25,12 @@
#endif
#include "backup_label.h"
-#include "common/blkreftable.h"
#include "common/checksum_helper.h"
#include "common/controldata_utils.h"
#include "common/file_perm.h"
#include "common/file_utils.h"
#include "common/logging.h"
+#include "common/relpath.h"
#include "copy_file.h"
#include "fe_utils/option_utils.h"
#include "getopt_long.h"
diff --git a/src/bin/pg_config/pg_config.c b/src/bin/pg_config/pg_config.c
index 77d09ccfc47..504e6c58009 100644
--- a/src/bin/pg_config/pg_config.c
+++ b/src/bin/pg_config/pg_config.c
@@ -25,7 +25,6 @@
#include "postgres_fe.h"
#include "common/config_info.h"
-#include "port.h"
static const char *progname;
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index c323b5bd3d9..9b2d34e2815 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -27,8 +27,6 @@
#include "catalog/pg_subscription_d.h"
#include "catalog/pg_type_d.h"
#include "common/hashfn.h"
-#include "fe_utils/string_utils.h"
-#include "pg_backup_archiver.h"
#include "pg_backup_utils.h"
#include "pg_dump.h"
diff --git a/src/bin/pg_dump/compress_io.c b/src/bin/pg_dump/compress_io.c
index e2edf8bf4f6..47a13099ccf 100644
--- a/src/bin/pg_dump/compress_io.c
+++ b/src/bin/pg_dump/compress_io.c
@@ -70,7 +70,6 @@
#include "compress_lz4.h"
#include "compress_none.h"
#include "compress_zstd.h"
-#include "pg_backup_utils.h"
/*----------------------
* Generic functions
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index 5815cd23748..9dc6d22fcda 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -13,7 +13,6 @@
*/
#include "postgres_fe.h"
-#include "common/fe_memutils.h"
#include "common/logging.h"
#include "common/string.h"
#include "filter.h"
diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c
index a09247fae47..78e91f6e2dc 100644
--- a/src/bin/pg_dump/parallel.c
+++ b/src/bin/pg_dump/parallel.c
@@ -63,7 +63,9 @@
#include "fe_utils/string_utils.h"
#include "parallel.h"
#include "pg_backup_utils.h"
+#ifdef WIN32
#include "port/pg_bswap.h"
+#endif
/* Mnemonic macros for indexing the fd array returned by pipe(2) */
#define PIPE_READ 0
diff --git a/src/bin/pg_dump/pg_backup_custom.c b/src/bin/pg_dump/pg_backup_custom.c
index 55107b20058..ecaad7321a3 100644
--- a/src/bin/pg_dump/pg_backup_custom.c
+++ b/src/bin/pg_dump/pg_backup_custom.c
@@ -27,7 +27,6 @@
#include "common/file_utils.h"
#include "compress_io.h"
-#include "parallel.h"
#include "pg_backup_utils.h"
/*--------
diff --git a/src/bin/pg_dump/pg_backup_db.c b/src/bin/pg_dump/pg_backup_db.c
index 4d1957a7eb3..71c55d2466a 100644
--- a/src/bin/pg_dump/pg_backup_db.c
+++ b/src/bin/pg_dump/pg_backup_db.c
@@ -19,8 +19,6 @@
#include "common/connect.h"
#include "common/string.h"
-#include "dumputils.h"
-#include "fe_utils/string_utils.h"
#include "parallel.h"
#include "pg_backup_archiver.h"
#include "pg_backup_db.h"
diff --git a/src/bin/pg_dump/pg_backup_utils.c b/src/bin/pg_dump/pg_backup_utils.c
index a0045cf5e58..80715979a1a 100644
--- a/src/bin/pg_dump/pg_backup_utils.c
+++ b/src/bin/pg_dump/pg_backup_utils.c
@@ -13,7 +13,9 @@
*/
#include "postgres_fe.h"
+#ifdef WIN32
#include "parallel.h"
+#endif
#include "pg_backup_utils.h"
/* Globals exported by this file */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1b47c388ced..ea759c5f78a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -49,10 +49,8 @@
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
-#include "catalog/pg_largeobject_metadata_d.h"
#include "catalog/pg_proc_d.h"
#include "catalog/pg_subscription.h"
-#include "catalog/pg_trigger_d.h"
#include "catalog/pg_type_d.h"
#include "common/connect.h"
#include "common/int.h"
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 4cb754caa55..d88ce06ff46 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -18,7 +18,6 @@
#include "catalog/pg_class_d.h"
#include "common/int.h"
#include "lib/binaryheap.h"
-#include "pg_backup_archiver.h"
#include "pg_backup_utils.h"
#include "pg_dump.h"
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index df119591cca..f2c1020d053 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -45,7 +45,6 @@
#include <termios.h>
#endif
-#include "dumputils.h"
#include "fe_utils/option_utils.h"
#include "filter.h"
#include "getopt_long.h"
diff --git a/src/bin/pg_rewind/datapagemap.h b/src/bin/pg_rewind/datapagemap.h
index 49124834508..be59ea23fc5 100644
--- a/src/bin/pg_rewind/datapagemap.h
+++ b/src/bin/pg_rewind/datapagemap.h
@@ -10,7 +10,6 @@
#define DATAPAGEMAP_H
#include "storage/block.h"
-#include "storage/relfilelocator.h"
struct datapagemap
{
diff --git a/src/bin/pg_rewind/libpq_source.c b/src/bin/pg_rewind/libpq_source.c
index 0239fb47644..62610a4efaf 100644
--- a/src/bin/pg_rewind/libpq_source.c
+++ b/src/bin/pg_rewind/libpq_source.c
@@ -11,7 +11,6 @@
#include "catalog/pg_type_d.h"
#include "common/connect.h"
-#include "datapagemap.h"
#include "file_ops.h"
#include "filemap.h"
#include "lib/stringinfo.h"
diff --git a/src/bin/pg_rewind/local_source.c b/src/bin/pg_rewind/local_source.c
index 46c6b72bd71..4aab2ba0ac9 100644
--- a/src/bin/pg_rewind/local_source.c
+++ b/src/bin/pg_rewind/local_source.c
@@ -12,10 +12,8 @@
#include <fcntl.h>
#include <unistd.h>
-#include "datapagemap.h"
+#include "common/logging.h"
#include "file_ops.h"
-#include "filemap.h"
-#include "pg_rewind.h"
#include "rewind_source.h"
typedef struct
diff --git a/src/bin/pg_rewind/pg_rewind.h b/src/bin/pg_rewind/pg_rewind.h
index ec43cbe2c67..d937542c6ec 100644
--- a/src/bin/pg_rewind/pg_rewind.h
+++ b/src/bin/pg_rewind/pg_rewind.h
@@ -14,10 +14,6 @@
#include "access/timeline.h"
#include "common/logging.h"
#include "common/file_utils.h"
-#include "datapagemap.h"
-#include "libpq-fe.h"
-#include "storage/block.h"
-#include "storage/relfilelocator.h"
/* Configuration options */
extern char *datadir_target;
diff --git a/src/bin/pg_test_fsync/pg_test_fsync.c b/src/bin/pg_test_fsync/pg_test_fsync.c
index cbf587116ea..65d05136f89 100644
--- a/src/bin/pg_test_fsync/pg_test_fsync.c
+++ b/src/bin/pg_test_fsync/pg_test_fsync.c
@@ -19,7 +19,6 @@
#include <unistd.h>
#include <signal.h>
-#include "access/xlogdefs.h"
#include "common/logging.h"
#include "common/pg_prng.h"
#include "getopt_long.h"
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 12735a42687..cba5031af87 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -11,9 +11,7 @@
#include "catalog/pg_authid_d.h"
#include "catalog/pg_class_d.h"
-#include "catalog/pg_collation.h"
#include "fe_utils/string_utils.h"
-#include "mb/pg_wchar.h"
#include "pg_upgrade.h"
static void check_new_cluster_is_empty(void);
diff --git a/src/bin/pg_upgrade/relfilenumber.c b/src/bin/pg_upgrade/relfilenumber.c
index 1d3054d78bd..07baa49a025 100644
--- a/src/bin/pg_upgrade/relfilenumber.c
+++ b/src/bin/pg_upgrade/relfilenumber.c
@@ -11,8 +11,6 @@
#include <sys/stat.h>
-#include "access/transam.h"
-#include "catalog/pg_class_d.h"
#include "pg_upgrade.h"
static void transfer_single_new_db(FileNameMap *maps, int size, char *old_tablespace);
diff --git a/src/bin/pg_walsummary/pg_walsummary.c b/src/bin/pg_walsummary/pg_walsummary.c
index f6a262d3184..267174d07d0 100644
--- a/src/bin/pg_walsummary/pg_walsummary.c
+++ b/src/bin/pg_walsummary/pg_walsummary.c
@@ -20,7 +20,6 @@
#include "common/logging.h"
#include "fe_utils/option_utils.h"
#include "getopt_long.h"
-#include "lib/stringinfo.h"
typedef struct ws_options
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 328d78c73f9..1f3cbb11f7c 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -30,7 +30,6 @@
#include "common/logging.h"
#include "common/string.h"
#include "copy.h"
-#include "crosstabview.h"
#include "describe.h"
#include "fe_utils/cancel.h"
#include "fe_utils/print.h"
@@ -38,10 +37,8 @@
#include "help.h"
#include "input.h"
#include "large_obj.h"
-#include "libpq-fe.h"
#include "libpq/pqcomm.h"
#include "mainloop.h"
-#include "portability/instr_time.h"
#include "pqexpbuffer.h"
#include "psqlscanslash.h"
#include "settings.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 363a66e7185..37b43fb1221 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -27,7 +27,6 @@
#include "fe_utils/print.h"
#include "fe_utils/string_utils.h"
#include "settings.h"
-#include "variables.h"
static const char *map_typename_pattern(const char *pattern);
static bool describeOneTableDetails(const char *schemaname,
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index 19d20c58781..02fe5d151e0 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -21,9 +21,6 @@
#include <termios.h>
#endif
-#include "common.h"
-#include "common/logging.h"
-#include "common/username.h"
#include "help.h"
#include "input.h"
#include "settings.h"
diff --git a/src/bin/scripts/clusterdb.c b/src/bin/scripts/clusterdb.c
index 3503a3bb584..7dd80d24137 100644
--- a/src/bin/scripts/clusterdb.c
+++ b/src/bin/scripts/clusterdb.c
@@ -16,7 +16,6 @@
#include "fe_utils/option_utils.h"
#include "fe_utils/query_utils.h"
#include "fe_utils/simple_list.h"
-#include "fe_utils/string_utils.h"
static void cluster_one_database(const ConnParams *cparams, const char *table,
diff --git a/src/bin/scripts/common.c b/src/bin/scripts/common.c
index d16381eda90..e3acd1e67a2 100644
--- a/src/bin/scripts/common.c
+++ b/src/bin/scripts/common.c
@@ -21,7 +21,6 @@
#include "common/connect.h"
#include "common/logging.h"
#include "common/string.h"
-#include "fe_utils/cancel.h"
#include "fe_utils/query_utils.h"
#include "fe_utils/string_utils.h"
diff --git a/src/bin/scripts/reindexdb.c b/src/bin/scripts/reindexdb.c
index af0738d9334..659d8d5ba58 100644
--- a/src/bin/scripts/reindexdb.c
+++ b/src/bin/scripts/reindexdb.c
@@ -15,7 +15,6 @@
#include "catalog/pg_class_d.h"
#include "common.h"
-#include "common/connect.h"
#include "common/logging.h"
#include "fe_utils/cancel.h"
#include "fe_utils/option_utils.h"
diff --git a/src/common/controldata_utils.c b/src/common/controldata_utils.c
index 82309b25107..52d4702ce87 100644
--- a/src/common/controldata_utils.c
+++ b/src/common/controldata_utils.c
@@ -28,8 +28,8 @@
#include "access/xlog_internal.h"
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
-#include "common/file_perm.h"
#ifdef FRONTEND
+#include "common/file_perm.h"
#include "common/logging.h"
#endif
#include "port/pg_crc32c.h"
diff --git a/src/common/string.c b/src/common/string.c
index ce9f2bc7ca5..4c42ae299b5 100644
--- a/src/common/string.c
+++ b/src/common/string.c
@@ -22,7 +22,6 @@
#endif
#include "common/string.h"
-#include "lib/stringinfo.h"
/*
diff --git a/src/common/unicode_case.c b/src/common/unicode_case.c
index 291e256e639..542e7ea5378 100644
--- a/src/common/unicode_case.c
+++ b/src/common/unicode_case.c
@@ -17,7 +17,6 @@
#include "common/unicode_case.h"
#include "common/unicode_case_table.h"
-#include "common/unicode_category.h"
#include "mb/pg_wchar.h"
static const pg_case_map *find_case_map(pg_wchar ucs);
diff --git a/src/fe_utils/astreamer_file.c b/src/fe_utils/astreamer_file.c
index c9a030853bc..47568da2dbb 100644
--- a/src/fe_utils/astreamer_file.c
+++ b/src/fe_utils/astreamer_file.c
@@ -19,7 +19,6 @@
#include "common/file_perm.h"
#include "common/logging.h"
-#include "common/string.h"
#include "fe_utils/astreamer.h"
typedef struct astreamer_plain_writer
diff --git a/src/fe_utils/astreamer_gzip.c b/src/fe_utils/astreamer_gzip.c
index ca5be6423a1..e0b755317cb 100644
--- a/src/fe_utils/astreamer_gzip.c
+++ b/src/fe_utils/astreamer_gzip.c
@@ -32,9 +32,7 @@
#include <zlib.h>
#endif
-#include "common/file_perm.h"
#include "common/logging.h"
-#include "common/string.h"
#include "fe_utils/astreamer.h"
#ifdef HAVE_LIBZ
diff --git a/src/fe_utils/astreamer_lz4.c b/src/fe_utils/astreamer_lz4.c
index 2bf14084e7f..a628088edfa 100644
--- a/src/fe_utils/astreamer_lz4.c
+++ b/src/fe_utils/astreamer_lz4.c
@@ -21,9 +21,7 @@
#include <lz4frame.h>
#endif
-#include "common/file_perm.h"
#include "common/logging.h"
-#include "common/string.h"
#include "fe_utils/astreamer.h"
#ifdef USE_LZ4
diff --git a/src/fe_utils/cancel.c b/src/fe_utils/cancel.c
index dcff9a85641..0df07fa4aa4 100644
--- a/src/fe_utils/cancel.c
+++ b/src/fe_utils/cancel.c
@@ -18,9 +18,7 @@
#include <unistd.h>
-#include "common/connect.h"
#include "fe_utils/cancel.h"
-#include "fe_utils/string_utils.h"
/*
--
2.47.0
^ permalink raw reply [nested|flat] 29+ messages in thread
* Re: further #include cleanup (IWYU)
2024-10-20 07:50 further #include cleanup (IWYU) Peter Eisentraut <[email protected]>
@ 2024-10-20 09:37 ` Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 29+ messages in thread
From: Alvaro Herrera @ 2024-10-20 09:37 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: pgsql-hackers
On 2024-Oct-20, Peter Eisentraut wrote:
> diff --git a/contrib/tablefunc/tablefunc.h b/contrib/tablefunc/tablefunc.h
> index 2009382ce7d..b78030044b5 100644
> --- a/contrib/tablefunc/tablefunc.h
> +++ b/contrib/tablefunc/tablefunc.h
> @@ -34,6 +34,4 @@
> #ifndef TABLEFUNC_H
> #define TABLEFUNC_H
>
> -#include "fmgr.h"
> -
> #endif /* TABLEFUNC_H */
You could as well just delete this file.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 29+ messages in thread
end of thread, other threads:[~2024-10-20 09:37 UTC | newest]
Thread overview: 29+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v30 4/6] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v34 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v32 4/6] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v29 4/6] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v31 4/6] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v35 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v32 4/6] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2019-02-01 10:47 [PATCH v33 4/5] Subscripting documentation erthalion <[email protected]>
2024-10-20 07:50 further #include cleanup (IWYU) Peter Eisentraut <[email protected]>
2024-10-20 09:37 ` Re: further #include cleanup (IWYU) Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox