public inbox for [email protected]help / color / mirror / Atom feed
Re: PATCH: Using BRIN indexes for sorted output 4+ messages / 4 participants [nested] [flat]
* Re: PATCH: Using BRIN indexes for sorted output @ 2023-02-24 16:04 Tomas Vondra <[email protected]> 2023-02-24 16:11 ` Re: PATCH: Using BRIN indexes for sorted output Matthias van de Meent <[email protected]> 2023-03-01 18:33 ` Re: PATCH: Using BRIN indexes for sorted output Alvaro Herrera <[email protected]> 0 siblings, 2 replies; 4+ messages in thread From: Tomas Vondra @ 2023-02-24 16:04 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: Matthias van de Meent <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Greg Stark <[email protected]>; Zhihong Yu <[email protected]>; PostgreSQL Hackers <[email protected]> On 2/24/23 16:14, Alvaro Herrera wrote: > On 2023-Feb-24, Tomas Vondra wrote: > >> I guess the easiest fix would be to do the arithmetic in 64 bits. That'd >> eliminate the overflow. > > Yeah, that might be easy to set up. We then don't have to worry about > it until BlockNumber is enlarged to 64 bits ... but by that time surely > we can just grow it again to a 128 bit loop variable. > >> Alternatively, we could do something like >> >> prevHeapBlk = 0; >> for (heapBlk = 0; (heapBlk < nblocks) && (prevHeapBlk <= heapBlk); >> heapBlk += pagesPerRange) >> { >> ... >> prevHeapBlk = heapBlk; >> } > > I think a formulation of this kind has the benefit that it works after > BlockNumber is enlarged to 64 bits, and doesn't have to be changed ever > again (assuming it is correct). > Did anyone even propose doing that? I suspect this is unlikely to be the only place that'd might be broken by that. > ... if pagesPerRange is not a whole divisor of MaxBlockNumber, I think > this will neglect the last range in the table. > Why would it? Let's say BlockNumber is uint8, i.e. 255 max. And there are 10 pages per range. That's 25 "full" ranges, and the last range being just 5 pages. So we get into prevHeapBlk = 240 heapBlk = 250 and we read the last 5 pages. And then we update prevHeapBlk = 250 heapBlk = (250 + 10) % 255 = 5 and we don't do that loop. Or did I get this wrong, somehow? regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: PATCH: Using BRIN indexes for sorted output 2023-02-24 16:04 Re: PATCH: Using BRIN indexes for sorted output Tomas Vondra <[email protected]> @ 2023-02-24 16:11 ` Matthias van de Meent <[email protected]> 1 sibling, 0 replies; 4+ messages in thread From: Matthias van de Meent @ 2023-02-24 16:11 UTC (permalink / raw) To: Tomas Vondra <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Greg Stark <[email protected]>; Zhihong Yu <[email protected]>; PostgreSQL Hackers <[email protected]> On Fri, 24 Feb 2023 at 17:04, Tomas Vondra <[email protected]> wrote: > > On 2/24/23 16:14, Alvaro Herrera wrote: > > ... if pagesPerRange is not a whole divisor of MaxBlockNumber, I think > > this will neglect the last range in the table. > > > > Why would it? Let's say BlockNumber is uint8, i.e. 255 max. And there > are 10 pages per range. That's 25 "full" ranges, and the last range > being just 5 pages. So we get into > > prevHeapBlk = 240 > heapBlk = 250 > > and we read the last 5 pages. And then we update > > prevHeapBlk = 250 > heapBlk = (250 + 10) % 255 = 5 > > and we don't do that loop. Or did I get this wrong, somehow? The result is off-by-one due to (u)int8 overflows being mod-256, but apart from that your result is accurate. The condition only stops the loop when we wrap around or when we go past the last block, but no earlier than that. Kind regards, Matthias van de Meent ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: PATCH: Using BRIN indexes for sorted output 2023-02-24 16:04 Re: PATCH: Using BRIN indexes for sorted output Tomas Vondra <[email protected]> @ 2023-03-01 18:33 ` Alvaro Herrera <[email protected]> 1 sibling, 0 replies; 4+ messages in thread From: Alvaro Herrera @ 2023-03-01 18:33 UTC (permalink / raw) To: Tomas Vondra <[email protected]>; +Cc: Matthias van de Meent <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Greg Stark <[email protected]>; Zhihong Yu <[email protected]>; PostgreSQL Hackers <[email protected]> On 2023-Feb-24, Tomas Vondra wrote: > On 2/24/23 16:14, Alvaro Herrera wrote: > > I think a formulation of this kind has the benefit that it works after > > BlockNumber is enlarged to 64 bits, and doesn't have to be changed ever > > again (assuming it is correct). > > Did anyone even propose doing that? I suspect this is unlikely to be the > only place that'd might be broken by that. True about other places also needing fixes, and no I haven't see anyone; but while 32 TB does seem very far away to us now, it might be not *that* far away. So I think doing it the other way is better. > > ... if pagesPerRange is not a whole divisor of MaxBlockNumber, I think > > this will neglect the last range in the table. > > Why would it? Let's say BlockNumber is uint8, i.e. 255 max. And there > are 10 pages per range. That's 25 "full" ranges, and the last range > being just 5 pages. So we get into > > prevHeapBlk = 240 > heapBlk = 250 > > and we read the last 5 pages. And then we update > > prevHeapBlk = 250 > heapBlk = (250 + 10) % 255 = 5 > > and we don't do that loop. Or did I get this wrong, somehow? I stand corrected. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ ^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v5 5/7] Row pattern recognition patch (docs). @ 2023-09-02 06:32 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Tatsuo Ishii @ 2023-09-02 06:32 UTC (permalink / raw) --- doc/src/sgml/advanced.sgml | 52 ++++++++++++++++++++++++++++++++++ doc/src/sgml/func.sgml | 54 ++++++++++++++++++++++++++++++++++++ doc/src/sgml/ref/select.sgml | 38 +++++++++++++++++++++++-- 3 files changed, 142 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 755c9f1485..eda3612822 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -537,6 +537,58 @@ WHERE pos < 3; <literal>rank</literal> less than 3. </para> + <para> + Row pattern common syntax can be used with row pattern common syntax to + perform row pattern recognition in a query. Row pattern common syntax + includes two sub clauses. <literal>DEFINE</literal> defines definition + variables along with an expression. The expression must be a logical + expression, which means it must + return <literal>TRUE</literal>, <literal>FALSE</literal> + or <literal>NULL</literal>. Moreover if the expression comprises a column + reference, it must be the argument of <function>rpr</function>. An example + of <literal>DEFINE</literal> is as follows. + +<programlisting> +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +</programlisting> + + Note that <function>PREV</function> returns price column in the previous + row if it's called in a context of row pattern recognition. So in the + second line means the definition variable "UP" is <literal>TRUE</literal> + when price column in the current row is greater than the price column in + the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when + price column in the current row is lower than the price column in the + previous row. + </para> + <para> + Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be + used. <literal>PATTERN</literal> defines a sequence of rows that satisfies + certain conditions. For example following <literal>PATTERN</literal> + defines that a row starts with the condition "LOWPRICE", then one or more + rows satisfy "UP" and finally one or more rows satisfy "DOWN". If a + sequence of rows found, rpr returns the column at the starting row. + Example of a <literal>SELECT</literal> using the <literal>DEFINE</literal> + and <literal>PATTERN</literal> clause is as follows. + +<programlisting> +SELECT company, tdate, price, max(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +</programlisting> + </para> + <para> When a query involves multiple window functions, it is possible to write out each one with a separate <literal>OVER</literal> clause, but this is diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7a0d4b9134..b7bfc9271e 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -21772,6 +21772,7 @@ SELECT count(*) FROM sometable; returns <literal>NULL</literal> if there is no such row. </para></entry> </row> + </tbody> </tgroup> </table> @@ -21811,6 +21812,59 @@ SELECT count(*) FROM sometable; Other frame specifications can be used to obtain other effects. </para> + <para> + Row pattern recognition navigation functions are listed in + <xref linkend="functions-rpr-navigation-table"/>. These functions + can be used to describe DEFINE clause of Row pattern recognition. + </para> + + <table id="functions-rpr-navigation-table"> + <title>Row Pattern Navigation Functions</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="func_table_entry"><para role="func_signature"> + Function + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>prev</primary> + </indexterm> + <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the previous row; + returns NULL if there is no previous row in the window frame. + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>next</primary> + </indexterm> + <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the next row; + returns NULL if there is no next row in the window frame. + </para></entry> + </row> + + </tbody> + </tgroup> + </table> + <note> <para> The SQL standard defines a <literal>RESPECT NULLS</literal> or diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index 0ee0cc7e64..8d3becd57a 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -966,8 +966,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl The <replaceable class="parameter">frame_clause</replaceable> can be one of <synopsis> -{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] -{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] +{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] +{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] </synopsis> where <replaceable>frame_start</replaceable> @@ -1074,6 +1074,40 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. </para> + <para> + The + optional <replaceable class="parameter">row_pattern_common_syntax</replaceable> + defines the <firstterm>row pattern recognition condition</firstterm> for + this + window. <replaceable class="parameter">row_pattern_common_syntax</replaceable> + includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST + ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls + how to proceed to next row position after a match + found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the + default) next row position is next to the last row of previous match. On + the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next + row position is always next to the last row of previous + match. <literal>DEFINE</literal> defines definition variables along with a + boolean expression. <literal>PATTERN</literal> defines a sequence of rows + that satisfies certain conditions using variables defined + in <literal>DEFINE</literal> clause. If the variable is not defined in + the <literal>DEFINE</literal> clause, it is implicitly assumed + following is defined in the <literal>DEFINE</literal> clause. + +<synopsis> +<literal>variable_name</literal> AS TRUE +</synopsis> + + Note that the maximu number of variables defined + in <literal>DEFINE</literal> clause is 26. + +<synopsis> +[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ] +PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...] +DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...] +</synopsis> + </para> + <para> The purpose of a <literal>WINDOW</literal> clause is to specify the behavior of <firstterm>window functions</firstterm> appearing in the query's -- 2.25.1 ----Next_Part(Sat_Sep__2_15_52_35_2023_273)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v5-0006-Row-pattern-recognition-patch-tests.patch" ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2023-09-02 06:32 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-02-24 16:04 Re: PATCH: Using BRIN indexes for sorted output Tomas Vondra <[email protected]> 2023-02-24 16:11 ` Matthias van de Meent <[email protected]> 2023-03-01 18:33 ` Alvaro Herrera <[email protected]> 2023-09-02 06:32 [PATCH v5 5/7] Row pattern recognition patch (docs). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox