public inbox for [email protected]help / color / mirror / Atom feed
Re: remaining sql/json patches 4+ messages / 3 participants [nested] [flat]
* Re: remaining sql/json patches @ 2023-10-06 09:23 Amit Langote <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Amit Langote @ 2023-10-06 09:23 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Erik Rijkers <[email protected]>; pgsql-hackers; jian he <[email protected]> On Wed, Oct 4, 2023 at 10:26 PM Amit Langote <[email protected]> wrote: > On Tue, Oct 3, 2023 at 10:11 PM Amit Langote <[email protected]> wrote: > > On Mon, Oct 2, 2023 at 2:26 PM Amit Langote <[email protected]> wrote: > > > On Mon, Oct 2, 2023 at 1:24 PM Amit Langote <[email protected]> wrote: > > > > Pushed this 30 min ago (no email on -committers yet!) and am looking > > > > at the following llvm crash reported by buildfarm animal pogona [1]: > > > > This seems to me to be complaining about the following addition: > > > > > > > > + { > > > > + Oid ioparam = op->d.iocoerce.typioparam; > > > > + LLVMValueRef v_params[6]; > > > > + LLVMValueRef v_success; > > > > + > > > > + v_params[0] = l_ptr_const(op->d.iocoerce.finfo_in, > > > > + l_ptr(StructFmgrInfo)); > > > > + v_params[1] = v_output; > > > > + v_params[2] = l_oid_const(lc, ioparam); > > > > + v_params[3] = l_int32_const(lc, -1); > > > > + v_params[4] = l_ptr_const(op->d.iocoerce.escontext, > > > > + > > > > l_ptr(StructErrorSaveContext)); > > > > > > > > - LLVMBuildStore(b, v_retval, v_resvaluep); > > > > + /* > > > > + * InputFunctionCallSafe() will write directly into > > > > + * *op->resvalue. > > > > + */ > > > > + v_params[5] = v_resvaluep; > > > > + > > > > + v_success = LLVMBuildCall(b, llvm_pg_func(mod, > > > > "InputFunctionCallSafe"), > > > > + v_params, lengthof(v_params), > > > > + "funccall_iocoerce_in_safe"); > > > > + > > > > + /* > > > > + * Return null if InputFunctionCallSafe() encountered > > > > + * an error. > > > > + */ > > > > + v_resnullp = LLVMBuildICmp(b, LLVMIntEQ, v_success, > > > > + l_sbool_const(0), ""); > > > > + } > > > > ...I haven't yet pinpointed down > which of the LLVM's asserts it is, nor have I been able to walk > through LLVM source code using gdb to figure what the new code is > doing wrong. Maybe I'm still missing a trick or two... I finally managed to analyze the crash by getting the correct LLVM build. So the following bits are the culprits: 1. v_output needed to be converted from being reference to a Datum to be reference to char * as follows before passing to InputFunctionCallSafe(): - v_params[1] = v_output; + v_params[1] = LLVMBuildIntToPtr(b, v_output, + l_ptr(LLVMInt8TypeInContext(lc)), + ""); 2. Assignment of op->d.iocoerce.escontext needed to be changed like this: v_params[4] = l_ptr_const(op->d.iocoerce.escontext, - l_ptr(StructErrorSaveContext)); + l_ptr(StructNode)); 3. v_success needed to be "zero-extended" to match in type with whatever s_bool_const() produces, as follows: + v_success = LLVMBuildZExt(b, v_success, TypeStorageBool, ""); v_resnullp = LLVMBuildICmp(b, LLVMIntEQ, v_success, l_sbool_const(0), ""); No more crashes with the above fixes. Attached shows the delta against the patch I reverted. I'll push the fixed up version on Monday. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] iocoerce-llvm-fixes.diff (3.0K, ../../CA+HiwqGGJtcVP=EGP0hH1Q6EqJE99KnOQvibC18QnepsWVu9fQ@mail.gmail.com/2-iocoerce-llvm-fixes.diff) download | inline diff: diff --git a/src/backend/jit/llvm/llvmjit.c b/src/backend/jit/llvm/llvmjit.c index 125e1e73ae..5a9be73957 100644 --- a/src/backend/jit/llvm/llvmjit.c +++ b/src/backend/jit/llvm/llvmjit.c @@ -77,7 +77,7 @@ LLVMTypeRef StructExprState; LLVMTypeRef StructAggState; LLVMTypeRef StructAggStatePerGroupData; LLVMTypeRef StructAggStatePerTransData; -LLVMTypeRef StructErrorSaveContext; +LLVMTypeRef StructNode; LLVMValueRef AttributeTemplate; @@ -1130,7 +1130,7 @@ llvm_create_types(void) StructAggState = llvm_pg_var_type("StructAggState"); StructAggStatePerGroupData = llvm_pg_var_type("StructAggStatePerGroupData"); StructAggStatePerTransData = llvm_pg_var_type("StructAggStatePerTransData"); - StructErrorSaveContext = llvm_pg_var_type("StructErrorSaveContext"); + StructNode = llvm_pg_var_type("StructNode"); AttributeTemplate = LLVMGetNamedFunction(llvm_types_module, "AttributeTemplate"); } diff --git a/src/backend/jit/llvm/llvmjit_expr.c b/src/backend/jit/llvm/llvmjit_expr.c index 7d44a4c9f4..aa0e09bd7a 100644 --- a/src/backend/jit/llvm/llvmjit_expr.c +++ b/src/backend/jit/llvm/llvmjit_expr.c @@ -1358,11 +1358,13 @@ llvm_compile_expr(ExprState *state) v_params[0] = l_ptr_const(op->d.iocoerce.finfo_in, l_ptr(StructFmgrInfo)); - v_params[1] = v_output; + v_params[1] = LLVMBuildIntToPtr(b, v_output, + l_ptr(LLVMInt8TypeInContext(lc)), + ""); v_params[2] = l_oid_const(lc, ioparam); v_params[3] = l_int32_const(lc, -1); v_params[4] = l_ptr_const(op->d.iocoerce.escontext, - l_ptr(StructErrorSaveContext)); + l_ptr(StructNode)); /* * InputFunctionCallSafe() will write directly into @@ -1378,6 +1380,7 @@ llvm_compile_expr(ExprState *state) * Return null if InputFunctionCallSafe() encountered * an error. */ + v_success = LLVMBuildZExt(b, v_success, TypeStorageBool, ""); v_resnullp = LLVMBuildICmp(b, LLVMIntEQ, v_success, l_sbool_const(0), ""); } diff --git a/src/backend/jit/llvm/llvmjit_types.c b/src/backend/jit/llvm/llvmjit_types.c index e1e9625038..78f1f14ba0 100644 --- a/src/backend/jit/llvm/llvmjit_types.c +++ b/src/backend/jit/llvm/llvmjit_types.c @@ -67,7 +67,7 @@ TupleTableSlot StructTupleTableSlot; HeapTupleTableSlot StructHeapTupleTableSlot; MinimalTupleTableSlot StructMinimalTupleTableSlot; TupleDescData StructTupleDescData; -ErrorSaveContext StructErrorSaveContext; +Node StructNode; /* diff --git a/src/include/jit/llvmjit.h b/src/include/jit/llvmjit.h index 5b7681eba9..e4ff3be566 100644 --- a/src/include/jit/llvmjit.h +++ b/src/include/jit/llvmjit.h @@ -83,7 +83,7 @@ extern PGDLLIMPORT LLVMTypeRef StructExprState; extern PGDLLIMPORT LLVMTypeRef StructAggState; extern PGDLLIMPORT LLVMTypeRef StructAggStatePerTransData; extern PGDLLIMPORT LLVMTypeRef StructAggStatePerGroupData; -extern PGDLLIMPORT LLVMTypeRef StructErrorSaveContext; +extern PGDLLIMPORT LLVMTypeRef StructNode; extern PGDLLIMPORT LLVMValueRef AttributeTemplate; ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: remaining sql/json patches @ 2023-10-06 10:01 Alvaro Herrera <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Alvaro Herrera @ 2023-10-06 10:01 UTC (permalink / raw) To: Amit Langote <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Erik Rijkers <[email protected]>; pgsql-hackers; jian he <[email protected]> On 2023-Oct-06, Amit Langote wrote: > 2. Assignment of op->d.iocoerce.escontext needed to be changed like this: > > v_params[4] = l_ptr_const(op->d.iocoerce.escontext, > - > l_ptr(StructErrorSaveContext)); > + l_ptr(StructNode)); Oh, so you had to go back to using StructNode in order to get this fixed? That's weird. Is it just because InputFunctionCallSafe is defined to take fmNodePtr? (I still fail to see that a pointer to ErrorSaveContext would differ in any material way from a pointer to Node). Another think I thought was weird is that it would only crash in LLVM5 debug and not the other LLVM-enabled animals, but looking closer at the buildfarm results, I think that may have been only because you reverted too quickly, and phycodorus and petalura didn't actually run with 7fbc75b26ed8 before you reverted it. Dragonet did make a run with it, but it's marked as "LLVM optimized" instead of "LLVM debug". I suppose that must be making a difference. -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/ "World domination is proceeding according to plan" (Andrew Morton) ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: remaining sql/json patches @ 2023-10-06 11:27 Amit Langote <[email protected]> parent: Alvaro Herrera <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Amit Langote @ 2023-10-06 11:27 UTC (permalink / raw) To: Alvaro Herrera <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Erik Rijkers <[email protected]>; pgsql-hackers; jian he <[email protected]> On Fri, Oct 6, 2023 at 19:01 Alvaro Herrera <[email protected]> wrote: > On 2023-Oct-06, Amit Langote wrote: > > > 2. Assignment of op->d.iocoerce.escontext needed to be changed like this: > > > > v_params[4] = > l_ptr_const(op->d.iocoerce.escontext, > > - > > l_ptr(StructErrorSaveContext)); > > + l_ptr(StructNode)); > > Oh, so you had to go back to using StructNode in order to get this > fixed? That's weird. Is it just because InputFunctionCallSafe is > defined to take fmNodePtr? (I still fail to see that a pointer to > ErrorSaveContext would differ in any material way from a pointer to > Node). The difference matters to LLVM’s type system, which considers Node to be a type with 1 sub-type (struct member) and ErrorSaveContext with 4 sub-types. It doesn’t seem to understand that both share the first member. Another think I thought was weird is that it would only crash in LLVM5 > debug and not the other LLVM-enabled animals, but looking closer at the > buildfarm results, I think that may have been only because you reverted > too quickly, and phycodorus and petalura didn't actually run with > 7fbc75b26ed8 before you reverted it. Dragonet did make a run with it, > but it's marked as "LLVM optimized" instead of "LLVM debug". I suppose > that must be making a difference. AFAICS, only assert-enabled LLVM builds crash. > ^ permalink raw reply [nested|flat] 4+ messages in thread
* [PATCH v10 5/7] Row pattern recognition patch (docs). @ 2023-10-22 02:22 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Tatsuo Ishii @ 2023-10-22 02:22 UTC (permalink / raw) --- doc/src/sgml/advanced.sgml | 80 ++++++++++++++++++++++++++++++++++++ doc/src/sgml/func.sgml | 54 ++++++++++++++++++++++++ doc/src/sgml/ref/select.sgml | 38 ++++++++++++++++- 3 files changed, 170 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 755c9f1485..cf18dd887e 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -537,6 +537,86 @@ WHERE pos < 3; <literal>rank</literal> less than 3. </para> + <para> + Row pattern common syntax can be used to perform row pattern recognition + in a query. Row pattern common syntax includes two sub + clauses: <literal>DEFINE</literal> + and <literal>PATTERN</literal>. <literal>DEFINE</literal> defines + definition variables along with an expression. The expression must be a + logical expression, which means it must + return <literal>TRUE</literal>, <literal>FALSE</literal> + or <literal>NULL</literal>. The expression may comprise column references + and functions. Window functions, aggregate functions and subqueries are + not allowed. An example of <literal>DEFINE</literal> is as follows. + +<programlisting> +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +</programlisting> + + Note that <function>PREV</function> returns the price column in the + previous row if it's called in a context of row pattern recognition. So in + the second line the definition variable "UP" is <literal>TRUE</literal> + when the price column in the current row is greater than the price column + in the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when + the price column in the current row is lower than the price column in the + previous row. + </para> + <para> + Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be + used. <literal>PATTERN</literal> defines a sequence of rows that satisfies + certain conditions. For example following <literal>PATTERN</literal> + defines that a row starts with the condition "LOWPRICE", then one or more + rows satisfy "UP" and finally one or more rows satisfy "DOWN". Note that + "+" means one or more matches. Also you can use "*", which means zero or + more matches. If a sequence of rows which satisfies the PATTERN is found, + in the starting row of the sequence of rows all window functions and + aggregates are shown in the target list. Note that aggregations only look + into the matched rows, rather than whole frame. In the second or + subsequent rows all window functions and aggregates are NULL. For rows + that do not match the PATTERN, all window functions and aggregates are + shown AS NULL too, except count which shows 0. This is because the + unmatched rows are in an empty frame. Example of + a <literal>SELECT</literal> using the <literal>DEFINE</literal> + and <literal>PATTERN</literal> clause is as follows. + +<programlisting> +SELECT company, tdate, price, + first_value(price) OVER w, + max(price) OVER w, + count(price) OVER w +FROM stock, + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +</programlisting> +<screen> + company | tdate | price | first_value | max | count +----------+------------+-------+-------------+-----+------- + company1 | 2023-07-01 | 100 | 100 | 200 | 4 + company1 | 2023-07-02 | 200 | | | + company1 | 2023-07-03 | 150 | | | + company1 | 2023-07-04 | 140 | | | + company1 | 2023-07-05 | 150 | | | 0 + company1 | 2023-07-06 | 90 | 90 | 130 | 4 + company1 | 2023-07-07 | 110 | | | + company1 | 2023-07-08 | 130 | | | + company1 | 2023-07-09 | 120 | | | + company1 | 2023-07-10 | 130 | | | 0 +</screen> + </para> + <para> When a query involves multiple window functions, it is possible to write out each one with a separate <literal>OVER</literal> clause, but this is diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7c3e940afe..1d835af15a 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -21878,6 +21878,7 @@ SELECT count(*) FROM sometable; returns <literal>NULL</literal> if there is no such row. </para></entry> </row> + </tbody> </tgroup> </table> @@ -21917,6 +21918,59 @@ SELECT count(*) FROM sometable; Other frame specifications can be used to obtain other effects. </para> + <para> + Row pattern recognition navigation functions are listed in + <xref linkend="functions-rpr-navigation-table"/>. These functions + can be used to describe DEFINE clause of Row pattern recognition. + </para> + + <table id="functions-rpr-navigation-table"> + <title>Row Pattern Navigation Functions</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="func_table_entry"><para role="func_signature"> + Function + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>prev</primary> + </indexterm> + <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the previous row; + returns NULL if there is no previous row in the window frame. + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>next</primary> + </indexterm> + <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the next row; + returns NULL if there is no next row in the window frame. + </para></entry> + </row> + + </tbody> + </tgroup> + </table> + <note> <para> The SQL standard defines a <literal>RESPECT NULLS</literal> or diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index 42d78913cf..522ad9dd70 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -969,8 +969,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl The <replaceable class="parameter">frame_clause</replaceable> can be one of <synopsis> -{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] -{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] +{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] +{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] </synopsis> where <replaceable>frame_start</replaceable> @@ -1077,6 +1077,40 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. </para> + <para> + The + optional <replaceable class="parameter">row_pattern_common_syntax</replaceable> + defines the <firstterm>row pattern recognition condition</firstterm> for + this + window. <replaceable class="parameter">row_pattern_common_syntax</replaceable> + includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST + ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls + how to proceed to next row position after a match + found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the + default) next row position is next to the last row of previous match. On + the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next + row position is always next to the last row of previous + match. <literal>DEFINE</literal> defines definition variables along with a + boolean expression. <literal>PATTERN</literal> defines a sequence of rows + that satisfies certain conditions using variables defined + in <literal>DEFINE</literal> clause. If the variable is not defined in + the <literal>DEFINE</literal> clause, it is implicitly assumed + following is defined in the <literal>DEFINE</literal> clause. + +<synopsis> +<literal>variable_name</literal> AS TRUE +</synopsis> + + Note that the maximu number of variables defined + in <literal>DEFINE</literal> clause is 26. + +<synopsis> +[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ] +PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...] +DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...] +</synopsis> + </para> + <para> The purpose of a <literal>WINDOW</literal> clause is to specify the behavior of <firstterm>window functions</firstterm> appearing in the query's -- 2.25.1 ----Next_Part(Sun_Oct_22_11_39_20_2023_140)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v10-0006-Row-pattern-recognition-patch-tests.patch" ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2023-10-22 02:22 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-10-06 09:23 Re: remaining sql/json patches Amit Langote <[email protected]> 2023-10-06 10:01 ` Alvaro Herrera <[email protected]> 2023-10-06 11:27 ` Amit Langote <[email protected]> 2023-10-22 02:22 [PATCH v10 5/7] Row pattern recognition patch (docs). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox