public inbox for [email protected]help / color / mirror / Atom feed
[PATCH] fixes to docs and comments 2+ messages / 2 participants [nested] [flat]
* [PATCH] fixes to docs and comments @ 2021-01-09 15:49 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Justin Pryzby @ 2021-01-09 15:49 UTC (permalink / raw) --- doc/src/sgml/config.sgml | 10 +++--- doc/src/sgml/libpq.sgml | 23 ++++++++---- doc/src/sgml/protocol.sgml | 54 +++++++++++++---------------- src/backend/libpq/pqcomm.c | 2 +- src/common/zpq_stream.c | 16 ++++----- src/include/common/zpq_stream.h | 2 +- src/interfaces/libpq/fe-connect.c | 2 +- src/interfaces/libpq/fe-protocol3.c | 26 +++++++------- 8 files changed, 69 insertions(+), 66 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 3b132dcf27..8b5a6525ac 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -1004,12 +1004,10 @@ include_dir 'conf.d' </term> <listitem> <para> - When this parameter is <literal>on</literal> (default), the <productname>PostgreSQL</productname> - server can switch on compression of traffic between server and client if it is requested by client. - Client sends to the server list of compression algorithms supported by frontend library, - server chooses one which is supported by backend library and sends it in compression acknowledgement message - to the client. This option allows to reject compression request even if it is supported by server - (due to security, CPU consumption or whatever else reasons...). + This parameter enables compression of libpq traffic between client and server. + The default is <literal>on</literal>. + This option allows rejecting compression requests even if it is supported by server + (for example, due to security, or CPU consumption). </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 2934908520..d4da390ca9 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -1244,13 +1244,22 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <term><literal>compression</literal></term> <listitem> <para> - Request compression of libpq traffic. Client sends to the server list of compression algorithms, supported by client library. - If server supports one of this algorithms, then it acknowledges use of this algorithm and then all libpq messages send both from client to server and - visa versa will be compressed. If server is not supporting any of the suggested algorithms, then it rejects client request to use compression - and it is up to the client whether to continue work without compression or report error. - Supported compression algorithms are chosen at configure time. Right now two libraries are supported: zlib (default) and zstd (if Postgres was - configured with --with-zstd option). In both cases streaming mode is used. - By default compression is disabled. Please notice that using compression together with SSL may add extra vulnerabilities: + Request compression of libpq traffic. The client sends a request with a list of compression algorithms. + Compression can be requested by a client by including the "compression" option in its connection string. + This can either be a boolean value to enable or disable compression + ("true"/"false", "on"/"off", "yes"/"no", "1"/"0"), "auto", or an explicit list of comma-separated compression algorithms + which can optionally include compression level ("zlib,zstd:5"). + If compression is enabled but an algorithm is not explicitly specified, the client library sends its full list of + supported algorithms and the server chooses a preferred algorithm. + + If the server accepts one of the algorithms, it replies with an acknowledgment and all future libpq messages between client and server + will be compressed. + If the server rejects the compression request, it is up to the client whether to continue without compression or to report an error. + Support for compression algorithms must be enabled when the server is compiled. + Currently, two libraries are supported: zlib (default) and zstd (if Postgres was + configured with --with-zstd option). In both cases, streaming mode is used. + By default, compression is not requested by the client. + Please note that using compression together with SSL may expose extra vulnerabilities: <ulink url="https://en.wikipedia.org/wiki/CRIME">CRIME</ulink; </para> </listitem> diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 9f30aad8bd..a69f9bfc26 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -94,10 +94,10 @@ <para> It is possible to compress protocol data to reduce traffic and speed-up client-server interaction. - Compression is especially useful for importing/exporting data to/from database using COPY command - and for replication (both physical and logical). Also compression can reduce server response time - in case of queries returning large amount of data (for example returning JSON, BLOBs, text,...) - Right now two libraries are supported: zlib (default) and zstd (if Postgres was + Compression is especially useful for importing/exporting data to/from the database using the <literal>COPY</literal> command + and for replication (both physical and logical). Compression can also reduce the server's response time + for queries returning a large amount of data (for example, JSON, BLOBs, text, ...). + Currently, two libraries are supported: zlib (default) and zstd (if Postgres was configured with --with-zstd option). </para> @@ -275,19 +275,14 @@ <term>CompressionAck</term> <listitem> <para> - Server acknowledges using compression for client-server communication protocol. - Compression can be requested by client by including "compression" option in connection string. - It can be just boolean values enabling or disabling compression - ("true"/"false", "on"/"off", "yes"/"no", "1"/"0"), "auto" or explicit list of compression algorithms - separated by comma with optional specification of compression level: "zlib,zstd:5". - If compression algorithm is not explicitly specified the most efficient one supported both by - client and server is chosen. Client sends to the server list of compression algorithms, - supported by client library. - If server supports one of this algorithms, then it acknowledges use of this algorithm and - all subsequent libpq messages send both from client to server and - visa versa will be compressed. Server selects most efficient algorithm among list specified by client and returns to the client - index of chosen algorithm in this list. If server is not supporting any of the suggested algorithms, then it replies with -1 - and it is up to the client whether to continue work without compression or report error. + The server accepts the client's compression request. + Compression is requested when a client connection includes the "compression" option, which includes + a list of requested compression algorithms. + If the server accepts one of these algorithms, it acknowledges use of compression and + all subsequent libpq messages between the client and server will be compressed. + The server chooses an algorithm from the list specified by client and responds with the index of the chosen algorithm from the client-supplied list. + If the server does not accept any of the requested algorithms, then it replies with an index of -1 + and it is up to the client whether to continue without compression or to report an error. </para> </listitem> </varlistentry> @@ -3460,12 +3455,14 @@ CompressionAck (B) </term> <listitem> <para> - Acknowledge use of compression for protocol data. Client sends to the server list of compression algorithms, supported by client library. - If server supports one of this algorithms, then it acknowledges use of this algorithm and all subsequent libpq messages send both from client to server and - visa versa will be compressed. Server selects most efficient algorithm among list specified by client and returns to the client - index of chosen algorithm in this list. If server is not supporting any of the suggested algorithms, then it replies with -1 - and it is up to the client whether to continue work without compression or report error. - After receiving this message with algorithm index other than -1, both server and client are switched to compression mode + Acknowledge use of compression for protocol data. The client sends to the server a list of requested compression algorithms. + If the server supports any of these algorithms, it acknowledges use of this algorithm and all subsequent libpq messages between client and server + will be compressed. + The server selects the preferred algorithm from the list specified by client and responds with the + index of the chosen algorithm in this list. + If the server does not support any of the requested algorithms, it replies with -1 + and it is up to the client whether to continue without compression or to report an error. + After receiving this message with algorithm index other than -1, both server and client switch to compressed mode and exchange compressed messages. </para> </listitem> @@ -3486,7 +3483,7 @@ CompressionAck (B) </term> <listitem> <para> - Index of algorithm in the list of supported algotihms specified by client or -1 if none of them is supported. + Index of algorithm in the list of supported algorithms specified by client or -1 if none of them are supported. </para> </listitem> </varlistentry> @@ -6069,12 +6066,11 @@ StartupMessage (F) </term> <listitem> <para> - Request compression of libpq traffic. Value is list of compression algorithms supported by client with optional + Request compression of libpq traffic. The value is a list of compression algorithms requested by the client with an optional specification of compression level: <literal>"zlib,zstd:5"</literal>. - When connecting to an older backend, which does not support compression, or in case when the backend support compression - but for some reason wants to disable it, the backend will just ignore the _pq_.compression parameter and won’t send - the compressionAck message to the frontend. - By default compression is disabled. Please notice that using compression together with SSL may add extra vulnerabilities: + If the server does not accept compression, the backend will ignore the _pq_.compression + parameter and will not send the CompressionAck message to the frontend. + By default, compression is disabled. Please note that using compression together with SSL may expose extra vulnerabilities: <ulink url="https://en.wikipedia.org/wiki/CRIME">CRIME</ulink;. </para> </listitem> diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c index 8be1ce18de..f3293faf01 100644 --- a/src/backend/libpq/pqcomm.c +++ b/src/backend/libpq/pqcomm.c @@ -1073,7 +1073,7 @@ pq_recvbuf(bool nowait) /* Can fill buffer from PqRecvLength and upwards */ for (;;) { - /* If streaming compression is enabled then use correspondent compression read function. */ + /* If streaming compression is enabled then use corresponding compression read function. */ r = PqStream ? zpq_read(PqStream, PqRecvBuffer + PqRecvLength, PQ_RECV_BUFFER_SIZE - PqRecvLength) diff --git a/src/common/zpq_stream.c b/src/common/zpq_stream.c index 459566228a..e044e83e7c 100644 --- a/src/common/zpq_stream.c +++ b/src/common/zpq_stream.c @@ -9,15 +9,15 @@ typedef struct { /* - * Returns name of compression algorithm. + * Name of compression algorithm. */ char const* (*name)(void); /* - * Create compression stream with using rx/tx function for fetching/sending compressed data. + * Create compression stream with rx/tx function for reading/sending compressed data. * level: compression level * tx_func: function for writing compressed data in underlying stream - * rx_func: function for receiving compressed data from underlying stream + * rx_func: function for reading compressed data from underlying stream * arg: context passed to the function * rx_data: received data (compressed data already fetched from input stream) * rx_data_size: size of data fetched from input stream @@ -27,14 +27,14 @@ typedef struct /* * Read up to "size" raw (decompressed) bytes. * Returns number of decompressed bytes or error code. - * Error code is either ZPQ_DECOMPRESS_ERROR either error code returned by the rx function. + * Error code is either ZPQ_DECOMPRESS_ERROR or error code returned by the rx function. */ ssize_t (*read)(ZpqStream *zs, void *buf, size_t size); /* * Write up to "size" raw (decompressed) bytes. * Returns number of written raw bytes or error code returned by tx function. - * In the last case amount of written raw bytes is stored in *processed. + * In the last case number of bytes written is stored in *processed. */ ssize_t (*write)(ZpqStream *zs, void const *buf, size_t size, size_t *processed); @@ -49,12 +49,12 @@ typedef struct char const* (*error)(ZpqStream *zs); /* - * Returns amount of data in internal tx decompression buffer. + * Return amount of data in internal tx decompression buffer. */ size_t (*buffered_tx)(ZpqStream *zs); /* - * Returns amount of data in internal rx compression buffer. + * Return amount of data in internal rx compression buffer. */ size_t (*buffered_rx)(ZpqStream *zs); } ZpqAlgorithm; @@ -265,7 +265,7 @@ zstd_name(void) #define ZLIB_BUFFER_SIZE 8192 /* We have to flush stream after each protocol command * and command is mostly limited by record length, - * which in turn usually less than page size (except TOAST) + * which in turn is usually less than page size (except TOAST) */ typedef struct ZlibStream diff --git a/src/include/common/zpq_stream.h b/src/include/common/zpq_stream.h index 27aef0aab9..991e2c0165 100644 --- a/src/include/common/zpq_stream.h +++ b/src/include/common/zpq_stream.h @@ -1,6 +1,6 @@ /* * zpq_stream.h - * Streaiming compression for libpq + * Streaming compression for libpq */ #ifndef ZPQ_STREAM_H diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index eac6ef66cc..679f8cde9f 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -3269,7 +3269,7 @@ keep_going: /* We will come back to here until there is { appendPQExpBuffer(&conn->errorMessage, libpq_gettext( - "server is not supported requested compression algorithms %s\n"), + "server does not support requested compression algorithms %s\n"), conn->compression); goto error_return; } diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c index 095866bba0..5b879569b4 100644 --- a/src/interfaces/libpq/fe-protocol3.c +++ b/src/interfaces/libpq/fe-protocol3.c @@ -2135,15 +2135,15 @@ pqBuildStartupPacket3(PGconn *conn, int *packetlen, } /* - * Build comma-separated list of compression algorithms suggested by client to the server. - * It can be either explicitly specified by user in connection string, either - * include all algorithms supported by clit library. - * This functions returns true if compression string is successfully parsed and - * stores comma-separated list of algorithms in *client_compressors. - * If compression is disabled, then NULL is assigned to *client_compressors. - * Also it creates array of compressor descriptors, each element of which corresponds - * the correspondent algorithm name in *client_compressors list. This array is stored in PGconn - * and is used during handshake when compassion acknowledgment response is received from the server. + * Build comma-separated list of compression algorithms requested by client. + * It can be either explicitly specified by user in connection string, or + * include all algorithms supported by client library. + * This function returns true if the compression string is successfully parsed and + * stores a comma-separated list of algorithms in *client_compressors. + * If compression is disabled, then NULL is assigned to *client_compressors. + * Also it creates an array of compressor descriptors, each element of which corresponds to + * the corresponding algorithm name in *client_compressors list. This array is stored in PGconn + * and is used during handshake when a compression acknowledgment response is received from the server. */ static bool build_compressors_list(PGconn *conn, char** client_compressors, bool build_descriptors) @@ -2170,7 +2170,7 @@ build_compressors_list(PGconn *conn, char** client_compressors, bool build_descr if (n_supported_algorithms == 0) { - *client_compressors = NULL; /* no compressors are avaialable */ + *client_compressors = NULL; /* no compressors are available */ conn->compressors = NULL; return true; } @@ -2204,7 +2204,7 @@ build_compressors_list(PGconn *conn, char** client_compressors, bool build_descr } else { - /* List of compresison algorithms separated by commas */ + /* List of compression algorithms separated by commas */ char *src, *dst; int n_suggested_algorithms = 0; char* suggested_algorithms = strdup(value); @@ -2232,7 +2232,7 @@ build_compressors_list(PGconn *conn, char** client_compressors, bool build_descr if (sscanf(col+1, "%d", &compression_level) != 1 && !build_descriptors) { fprintf(stderr, - libpq_gettext("WARNING: invlaid compression level %s in compression option '%s'\n"), + libpq_gettext("WARNING: invalid compression level %s in compression option '%s'\n"), col+1, value); return false; } @@ -2262,7 +2262,7 @@ build_compressors_list(PGconn *conn, char** client_compressors, bool build_descr { if (!build_descriptors) fprintf(stderr, - libpq_gettext("WARNING: none of specified algirthms %s is supported by client\n"), + libpq_gettext("WARNING: none of the specified algorithms are supported by client: %s\n"), value); else { -- 2.17.0 --yudcn1FV7Hsu/q59-- ^ permalink raw reply [nested|flat] 2+ messages in thread
* [PATCH v2 5/7] Row pattern recognition patch (docs). @ 2023-06-26 08:05 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Tatsuo Ishii @ 2023-06-26 08:05 UTC (permalink / raw) --- doc/src/sgml/advanced.sgml | 52 +++++++++++++++++++++++++++ doc/src/sgml/func.sgml | 69 ++++++++++++++++++++++++++++++++++++ doc/src/sgml/ref/select.sgml | 29 +++++++++++++-- 3 files changed, 148 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 755c9f1485..e9bbd5bc7c 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -537,6 +537,58 @@ WHERE pos < 3; <literal>rank</literal> less than 3. </para> + <para> + Window function <function>rpr</function> can be used with row pattern + common syntax to perform row pattern recognition in a query. Row pattern + common syntax includes two sub clauses. <literal>DEFINE</literal> defines + definition variables along with an expression. The expression must be a + logical expression, which means it must + return <literal>TRUE</literal>, <literal>FALSE</literal> + or <literal>NULL</literal>. Moreover if the expression comprises a column + reference, it must be the argument of <function>rpr</function>. An + example of <literal>DEFINE</literal> is as follows. + +<programlisting> +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +</programlisting> + + Note that <function>PREV</function> returns price column in the previous + row if it's called in a context of row pattern recognition. So in the + second line means the definition variable "UP" is <literal>TRUE</literal> + when price column in the current row is greater than the price column in + the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when + price column in the current row is lower than the price column in the + previous row. + </para> + <para> + Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be + used. <literal>PATTERN</literal> defines a sequence of rows that satisfies + certain conditions. For example following <literal>PATTERN</literal> + defines that a row starts with the condition "LOWPRICE", then one or more + rows satisfy "UP" and finally one or more rows satisfy "DOWN". If a + sequence of rows found, rpr returns the column at the starting row. + Example of a <literal>SELECT</literal> using the <literal>DEFINE</literal> + and <literal>PATTERN</literal> clause is as follows. + +<programlisting> +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +</programlisting> + </para> + <para> When a query involves multiple window functions, it is possible to write out each one with a separate <literal>OVER</literal> clause, but this is diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 5a47ce4343..8069c58ca5 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -21648,6 +21648,22 @@ SELECT count(*) FROM sometable; returns <literal>NULL</literal> if there is no such row. </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>rpr</primary> + </indexterm> + <function>rpr</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Perform row pattern recognition using column specified + by <parameter>value</parameter> and returns the value of the column if + current row is the first matching row; + returns <literal>NULL</literal> otherwise. + </para></entry> + </row> </tbody> </tgroup> </table> @@ -21687,6 +21703,59 @@ SELECT count(*) FROM sometable; Other frame specifications can be used to obtain other effects. </para> + <para> + Row pattern recognition navigation functions are listed in + <xref linkend="functions-rpr-navigation-table"/>. These functions + can be used to describe DEFINE clause of Row pattern recognition. + </para> + + <table id="functions-rpr-navigation-table"> + <title>Row Pattern Navigation Functions</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="func_table_entry"><para role="func_signature"> + Function + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>prev</primary> + </indexterm> + <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the previous row; + returns NULL if there is no previous row in the window frame. + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>next</primary> + </indexterm> + <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the next row; + returns NULL if there is no next row in the window frame. + </para></entry> + </row> + + </tbody> + </tgroup> + </table> + <note> <para> The SQL standard defines a <literal>RESPECT NULLS</literal> or diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index 0ee0cc7e64..16478a3950 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -966,8 +966,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl The <replaceable class="parameter">frame_clause</replaceable> can be one of <synopsis> -{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] -{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] +{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] +{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] </synopsis> where <replaceable>frame_start</replaceable> @@ -1074,6 +1074,31 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. </para> + <para> + The + optional <replaceable class="parameter">row_pattern_common_syntax</replaceable> + defines the <firstterm>row pattern recognition condition</firstterm> for + this + window. <replaceable class="parameter">row_pattern_common_syntax</replaceable> + includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST + ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls + how to proceed to next row position after a match + found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the + default) next row position is next to the last row of previous match. On + the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next + row position is always next to the last row of previous + match. <literal>DEFINE</literal> defines definition variables along with a + boolean expression. <literal>PATTERN</literal> defines a sequence of rows + that satisfies certain conditions using variables defined + in <literal>DEFINE</literal> clause. + +<synopsis> +[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ] +PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...] +DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...] +</synopsis> + </para> + <para> The purpose of a <literal>WINDOW</literal> clause is to specify the behavior of <firstterm>window functions</firstterm> appearing in the query's -- 2.25.1 ----Next_Part(Mon_Jun_26_17_45_07_2023_724)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v2-0006-Row-pattern-recognition-patch-tests.patch" ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2023-06-26 08:05 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-01-09 15:49 [PATCH] fixes to docs and comments Justin Pryzby <[email protected]> 2023-06-26 08:05 [PATCH v2 5/7] Row pattern recognition patch (docs). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox