From: torikoshia <torikoshia@oss.nttdata.com>
To: Fujii Masao <masao.fujii@oss.nttdata.com>
To: zhjwpku@gmail.com
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: Add new COPY option REJECT_LIMIT
Date: Mon, 22 Jul 2024 21:37:03 +0900
Message-ID: <f9feda0ecd4e2e09620a6f9ca8f4aac1@oss.nttdata.com> (raw)
In-Reply-To: <54bea7dd-94b1-41f1-a4e9-c5c53cc82ce7@oss.nttdata.com>
References: <63f99327aa6b404cc951217fa3e61fe4@oss.nttdata.com>
<fab7018f-6e00-4be4-bd1e-4a013096f6c9@oss.nttdata.com>
<de0551d0f9d9e9072e324e51ed5c426d@oss.nttdata.com>
<6e3dcdc6-8e18-4716-a23b-3bee316aeaa7@oss.nttdata.com>
<979c6d306343f3d665d14eab2cd1b7bc@oss.nttdata.com>
<920a29a36032befbeb355cbdfbbe0f63@oss.nttdata.com>
<a873643c-e7d1-456c-b16b-6162ab698036@oss.nttdata.com>
<54bea7dd-94b1-41f1-a4e9-c5c53cc82ce7@oss.nttdata.com>
On Fri, Jul 19, 2024 at 11:48 PM Junwang Zhao <zhjwpku@gmail.com> wrote:
Thanks for the comment.
> In patch 0002, the ratio is calculated by the already skipped/processed
> rows, but what if a user wants to copy 1000 rows, and he/she can
> tolerate
> 10 error rows, so he/she might set *reject_limit 0.01*, but one bad row
> in the
> first 100 rows will fail the entire command, this might surprise the
> user.
Since the ratio is calculated after all data is processed, the case "one
bad row in the first 100 rows will fail the entire command" doesn't
happen:
=# \! wc -l 1000rows-with-10err.data
1000 1000rows-with-10err.data
=# COPY t1 from '1000rows-with-10err.data' with (log_verbosity
verbose, reject_limit 0.01);
NOTICE: skipping row due to data type incompatibility at line 10 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 11 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 12 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 13 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 14 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 15 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 16 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 17 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 18 for
column i: "a"
NOTICE: skipping row due to data type incompatibility at line 19 for
column i: "a"
NOTICE: 10 rows were skipped due to data type incompatibility
COPY 990
On 2024-07-20 02:08, Fujii Masao wrote:
> On 2024/07/19 22:03, Fujii Masao wrote:
>>
>>
>> On 2024/07/17 22:21, torikoshia wrote:
>>> On 2024-07-03 02:07, Fujii Masao wrote:
>>>> However, if we support REJECT_LIMIT, I'm not sure if the ON_ERROR
>>>> option is still necessary.
>>>
>>> I remembered another reason for the necessity of ON_ERROR.
>>>
>>> ON_ERROR defines how to behave when encountering an error and it just
>>> accepts 'ignore' and 'stop' currently, but is expected to support
>>> other options such as saving details of errors to a table[1].
>>
>> Wouldn't it be better to separate the option specifying where
>> error details are output from the ON_ERROR option
>> (which determines behavior when encountering errors)?
>> "table" seems valid for both ON_ERROR=ignore and ON_ERROR=stop.
>
> I still find it odd to accept "table" as a value for ON_ERROR. However,
> "set_to_null" or "replace-column" proposed in [1] seem valid for
> ON_ERROR. So, I'm okay with keeping the ON_ERROR option.
Agreed.
>> On my second thought, whatever value ON_ERROR is specified(e.g.
>> ignore, stop, table), it seems fine to use REJECT_LIMIT.
>> I feel REJECT_LIMIT has both "ignore" and "stop" characteristics,
>> meaning it ignores errors until it reaches REJECT_LIMIT and stops when
>> it exceeds the REJECT_LIMIT.
>
> ON_ERROR specifies how to handle errors, and "stop" means to fail
> the command. So, if ON_ERROR=stop, REJECT_LIMIT should have no effect,
> and the command should fail immediately upon encountering an error.
>
> As in your original proposal, I now think REJECT_LIMIT should only
> apply when ON_ERROR=ignore. The command would ignore errors and
> continue processing, but if the number of errors exceeds REJECT_LIMIT,
> the command should fail. Thought?
Makes sense.
Updated the patch.
> BTW if "set_to_null" is supported someday, REJECT_LIMIT can also
> apply. The command would cinsert NULL into the target table upon
> encountering errors and continue, but fail if the number of errors
> exceed REJECT_LIMIT.
Agreed.
--
Regards,
--
Atsushi Torikoshi
NTT DATA Group Corporation
Attachments:download all (2 files, .tar.gz)v3-0001-Add-new-COPY-option-REJECT_LIMIT-number.patch (text/x-diff, 10.0K) download
From 594e578bf1633b8a24f6378436dc349664de6ba4 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Date: Mon, 22 Jul 2024 21:19:38 +0900
Subject: [PATCH v3] Add new COPY option REJECT_LIMIT number
---
doc/src/sgml/ref/copy.sgml | 22 +++++++++++++
src/backend/commands/copy.c | 49 +++++++++++++++++++++++++++++
src/backend/commands/copyfrom.c | 6 ++++
src/include/commands/copy.h | 10 ++++++
src/test/regress/expected/copy2.out | 14 +++++++++
src/test/regress/sql/copy2.sql | 30 ++++++++++++++++++
6 files changed, 131 insertions(+)
diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgmlindex 1518af8a04..beb455372d 100644--- a/doc/src/sgml/ref/copy.sgml+++ b/doc/src/sgml/ref/copy.sgml@@ -44,6 +44,7 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable
FORCE_NOT_NULL { ( <replaceable class="parameter">column_name</replaceable> [, ...] ) | * }
FORCE_NULL { ( <replaceable class="parameter">column_name</replaceable> [, ...] ) | * }
ON_ERROR <replaceable class="parameter">error_action</replaceable>
+ REJECT_LIMIT { <replaceable class="parameter">integer</replaceable> | INFINITY }
ENCODING '<replaceable class="parameter">encoding_name</replaceable>'
LOG_VERBOSITY <replaceable class="parameter">verbosity</replaceable>
</synopsis>
@@ -411,6 +412,27 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable
</listitem>
</varlistentry>
+ <varlistentry>+ <term><literal>REJECT_LIMIT</literal></term>+ <listitem>+ <para>+ When a positive integer value is specified, <command>COPY</command> limits+ the maximum tolerable number of errors while converting a column's input+ value into its data type.+ If input data caused more errors than the specified value, entire+ <command>COPY</command> fails.+ Otherwise, <command>COPY</command> discards the input row and continues+ with the next one.+ This option must be used with <literal>ON_ERROR</literal> to be set to+ other than <literal>stop</literal>.+ </para>+ <para>+ When specified <literal>INFINITY</literal>, <command>COPY</command> ignores all+ the errors. This is a synonym for <literal>ON_ERROR</literal> <literal>ignore</literal>.+ </para>+ </listitem>+ </varlistentry>+
<varlistentry>
<term><literal>ENCODING</literal></term>
<listitem>
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.cindex df7a4a21c9..8fbab9336f 100644--- a/src/backend/commands/copy.c+++ b/src/backend/commands/copy.c@@ -415,6 +415,43 @@ defGetCopyOnErrorChoice(DefElem *def, ParseState *pstate, bool is_from)
return COPY_ON_ERROR_STOP; /* keep compiler quiet */
}
+/*+ * Extract a CopyRejectLimits values from a DefElem.+ */+static CopyRejectLimits+defGetCopyRejectLimitOptions(DefElem *def)+{+ CopyRejectLimits limits;+ int64 num_err;++ switch(nodeTag(def->arg))+ {+ case T_Integer:+ num_err = defGetInt64(def);+ if (num_err <= 0)+ ereport(ERROR,+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),+ errmsg("number for REJECT_LIMIT must be greater than zero")));+ break;+ case T_String:+ if (pg_strcasecmp(defGetString(def), "INFINITY") == 0)+ /* when set to 0, it is treated as no limit */+ num_err = 0;+ else+ ereport(ERROR,+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),+ errmsg("string for REJECT_LIMIT must be 'INFINITY'")));+ break;+ default:+ ereport(ERROR,+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),+ errmsg("value for REJECT_LIMIT must be positive integer or 'INFINITY'")));+ }+ limits.num_err = num_err;++ return limits;+}+
/*
* Extract a CopyLogVerbosityChoice value from a DefElem.
*/
@@ -466,6 +503,7 @@ ProcessCopyOptions(ParseState *pstate,
bool header_specified = false;
bool on_error_specified = false;
bool log_verbosity_specified = false;
+ bool reject_limit_specified = false;
ListCell *option;
/* Support external use for option sanity checking */
@@ -632,6 +670,17 @@ ProcessCopyOptions(ParseState *pstate,
log_verbosity_specified = true;
opts_out->log_verbosity = defGetCopyLogVerbosityChoice(defel, pstate);
}
+ else if (strcmp(defel->defname, "reject_limit") == 0)+ {+ if (reject_limit_specified)+ errorConflictingDefElem(defel, pstate);+ if (!opts_out->on_error)+ ereport(ERROR,+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),+ errmsg("REJECT_LIMIT requires ON_ERROR to be set to other than stop")));+ reject_limit_specified = true;+ opts_out->reject_limits = defGetCopyRejectLimitOptions(defel);+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.cindex ce4d62e707..65d950ef07 100644--- a/src/backend/commands/copyfrom.c+++ b/src/backend/commands/copyfrom.c@@ -1012,6 +1012,12 @@ CopyFrom(CopyFromState cstate)
pgstat_progress_update_param(PROGRESS_COPY_TUPLES_SKIPPED,
++skipped);
+ if (cstate->opts.reject_limits.num_err &&+ skipped > cstate->opts.reject_limits.num_err)+ ereport(ERROR,+ (errcode(ERRCODE_BAD_COPY_FILE_FORMAT),+ errmsg("exceeded the number specified by REJECT_LIMIT \"%lld\"",+ (long long) cstate->opts.reject_limits.num_err)));
continue;
}
diff --git a/src/include/commands/copy.h b/src/include/commands/copy.hindex 141fd48dc1..86ba1cbe16 100644--- a/src/include/commands/copy.h+++ b/src/include/commands/copy.h@@ -49,6 +49,15 @@ typedef enum CopyLogVerbosityChoice
COPY_LOG_VERBOSITY_VERBOSE, /* logs additional messages */
} CopyLogVerbosityChoice;
+/*+ * A struct to hold reject_limit options, in a parsed form.+ * More values to be added in another patch.+ */+typedef struct CopyRejectLimits+{+ int64 num_err; /* maximum tolerable number of errors */+} CopyRejectLimits;+
/*
* A struct to hold COPY options, in a parsed form. All of these are related
* to formatting, except for 'freeze', which doesn't really belong here, but
@@ -83,6 +92,7 @@ typedef struct CopyFormatOptions
bool convert_selectively; /* do selective binary conversion? */
CopyOnErrorChoice on_error; /* what to do when error happened */
CopyLogVerbosityChoice log_verbosity; /* verbosity of logged messages */
+ CopyRejectLimits reject_limits; /* thresholds of reject_limit */
List *convert_select; /* list of column names (can be NIL) */
} CopyFormatOptions;
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.outindex 931542f268..8ce12f5b6f 100644--- a/src/test/regress/expected/copy2.out+++ b/src/test/regress/expected/copy2.out@@ -85,6 +85,10 @@ COPY x from stdin (log_verbosity default, log_verbosity verbose);
ERROR: conflicting or redundant options
LINE 1: COPY x from stdin (log_verbosity default, log_verbosity verb...
^
+COPY x from stdin (on_error ignore, reject_limit 'INFINITY', reject_limit 3);+ERROR: conflicting or redundant options+LINE 1: ... stdin (on_error ignore, reject_limit 'INFINITY', reject_lim...+ ^
-- incorrect options
COPY x to stdin (format BINARY, delimiter ',');
ERROR: cannot specify DELIMITER in BINARY mode
@@ -116,6 +120,8 @@ COPY x to stdout (log_verbosity unsupported);
ERROR: COPY LOG_VERBOSITY "unsupported" not recognized
LINE 1: COPY x to stdout (log_verbosity unsupported);
^
+COPY x from stdin with (on_error ignore, reject_limit 0);+ERROR: number for REJECT_LIMIT must be greater than zero
-- too many columns in column list: should fail
COPY x (a, b, c, d, e, d, c) from stdin;
ERROR: column "d" specified more than once
@@ -789,6 +795,14 @@ CONTEXT: COPY check_ign_err, line 1: "1 {1}"
COPY check_ign_err FROM STDIN WITH (on_error ignore);
ERROR: extra data after last expected column
CONTEXT: COPY check_ign_err, line 1: "1 {1} 3 abc"
+-- tests for reject_limit option+COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 3);+ERROR: exceeded the number specified by REJECT_LIMIT "3"+CONTEXT: COPY check_ign_err, line 5, column n: ""+COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 4);+NOTICE: 4 rows were skipped due to data type incompatibility+COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 'INFINITY');+NOTICE: 4 rows were skipped due to data type incompatibility
-- clean up
DROP TABLE forcetest;
DROP TABLE vistest;
diff --git a/src/test/regress/sql/copy2.sql b/src/test/regress/sql/copy2.sqlindex 8b14962194..8cc82990f4 100644--- a/src/test/regress/sql/copy2.sql+++ b/src/test/regress/sql/copy2.sql@@ -68,6 +68,7 @@ COPY x from stdin (convert_selectively (a), convert_selectively (b));
COPY x from stdin (encoding 'sql_ascii', encoding 'sql_ascii');
COPY x from stdin (on_error ignore, on_error ignore);
COPY x from stdin (log_verbosity default, log_verbosity verbose);
+COPY x from stdin (on_error ignore, reject_limit 'INFINITY', reject_limit 3);
-- incorrect options
COPY x to stdin (format BINARY, delimiter ',');
@@ -82,6 +83,7 @@ COPY x to stdout (format TEXT, force_null(a));
COPY x to stdin (format CSV, force_null(a));
COPY x to stdin (format BINARY, on_error unsupported);
COPY x to stdout (log_verbosity unsupported);
+COPY x from stdin with (on_error ignore, reject_limit 0);
-- too many columns in column list: should fail
COPY x (a, b, c, d, e, d, c) from stdin;
@@ -557,6 +559,34 @@ COPY check_ign_err FROM STDIN WITH (on_error ignore);
1 {1} 3 abc
\.
+-- tests for reject_limit option+COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 3);+6 {6} 6+a {7} 7+8 {8} 8888888888+9 {a, 9} 9++10 {10} 10+\.++COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 4);+6 {6} 6+a {7} 7+8 {8} 8888888888+9 {a, 9} 9++10 {10} 10+\.++COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 'INFINITY');+6 {6} 6+a {7} 7+8 {8} 8888888888+9 {a, 9} 9++10 {10} 10+\.+
-- clean up
DROP TABLE forcetest;
DROP TABLE vistest;
--
2.39.2
v3-0002-Add-new-COPY-option-REJECT_LIMIT-ratio.patch (text/x-diff, 7.6K) download
From 87894c42dfa222a273170f4783ef8f3fae9537f5 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Date: Mon, 22 Jul 2024 21:23:42 +0900
Subject: [PATCH v3] Add new COPY option REJECT_LIMIT ratio
---
doc/src/sgml/ref/copy.sgml | 11 ++++++++++-
src/backend/commands/copy.c | 18 +++++++++++++-----
src/backend/commands/copyfrom.c | 10 ++++++++++
src/include/commands/copy.h | 1 +
src/test/regress/expected/copy2.out | 7 +++++++
src/test/regress/sql/copy2.sql | 19 +++++++++++++++++++
6 files changed, 60 insertions(+), 6 deletions(-)
diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgmlindex beb455372d..e2e0cc9665 100644--- a/doc/src/sgml/ref/copy.sgml+++ b/doc/src/sgml/ref/copy.sgml@@ -44,7 +44,7 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable
FORCE_NOT_NULL { ( <replaceable class="parameter">column_name</replaceable> [, ...] ) | * }
FORCE_NULL { ( <replaceable class="parameter">column_name</replaceable> [, ...] ) | * }
ON_ERROR <replaceable class="parameter">error_action</replaceable>
- REJECT_LIMIT { <replaceable class="parameter">integer</replaceable> | INFINITY }+ REJECT_LIMIT { <replaceable class="parameter">integer</replaceable> | <replaceable class="parameter">floating point</replaceable> | INFINITY }
ENCODING '<replaceable class="parameter">encoding_name</replaceable>'
LOG_VERBOSITY <replaceable class="parameter">verbosity</replaceable>
</synopsis>
@@ -426,6 +426,15 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable
This option must be used with <literal>ON_ERROR</literal> to be set to
other than <literal>stop</literal>.
</para>
+ <para>+ When a positive floating point value is specified, <command>COPY</command>+ limits the maximum ratio of errors while converting a column's input+ value into its data type.+ If input data caused an error ratio greater than the specified value,+ entire <command>COPY</command> fails.+ Otherwise, <command>COPY</command> discards only the input rows where+ errors occured and copies all the other rows.+ </para>
<para>
When specified <literal>INFINITY</literal>, <command>COPY</command> ignores all
the errors. This is a synonym for <literal>ON_ERROR</literal> <literal>ignore</literal>.
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.cindex 8fbab9336f..b3cc63d44f 100644--- a/src/backend/commands/copy.c+++ b/src/backend/commands/copy.c@@ -422,7 +422,8 @@ static CopyRejectLimits
defGetCopyRejectLimitOptions(DefElem *def)
{
CopyRejectLimits limits;
- int64 num_err;+ uint64 num_err = 0;+ double ratio_err = 0;
switch(nodeTag(def->arg))
{
@@ -433,14 +434,20 @@ defGetCopyRejectLimitOptions(DefElem *def)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("number for REJECT_LIMIT must be greater than zero")));
break;
+ case T_Float:+ ratio_err = defGetNumeric(def);+ if (ratio_err <= 0 || ratio_err >= 1)+ ereport(ERROR,+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),+ errmsg("float for REJECT_LIMIT must be greater than zero and smaller than 1")));+ break;
case T_String:
- if (pg_strcasecmp(defGetString(def), "INFINITY") == 0)- /* when set to 0, it is treated as no limit */- num_err = 0;- else+ if (pg_strcasecmp(defGetString(def), "INFINITY") != 0)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("string for REJECT_LIMIT must be 'INFINITY'")));
++ /* when set to 0, it is treated as no limit */
break;
default:
ereport(ERROR,
@@ -448,6 +455,7 @@ defGetCopyRejectLimitOptions(DefElem *def)
errmsg("value for REJECT_LIMIT must be positive integer or 'INFINITY'")));
}
limits.num_err = num_err;
+ limits.ratio_err = ratio_err;
return limits;
}
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.cindex 65d950ef07..69485a25a1 100644--- a/src/backend/commands/copyfrom.c+++ b/src/backend/commands/copyfrom.c@@ -646,6 +646,7 @@ CopyFrom(CopyFromState cstate)
int64 processed = 0;
int64 excluded = 0;
int64 skipped = 0;
+ double ratio_err = 0;
bool has_before_insert_row_trig;
bool has_instead_insert_row_trig;
bool leafpart_use_multi_insert = false;
@@ -1310,6 +1311,15 @@ CopyFrom(CopyFromState cstate)
CopyMultiInsertInfoFlush(&multiInsertInfo, NULL, &processed);
}
+ ratio_err = (double) skipped / (processed + skipped);+ if (cstate->opts.reject_limits.ratio_err &&+ cstate->opts.reject_limits.ratio_err < ratio_err)+ ereport(ERROR,+ (errcode(ERRCODE_BAD_COPY_FILE_FORMAT),+ errmsg("exceeded the ratio specified by REJECT_LIMIT \"%f\", the error ratio was: \"%f\"",+ cstate->opts.reject_limits.ratio_err,+ ratio_err)));+
/* Done, clean up */
error_context_stack = errcallback.previous;
diff --git a/src/include/commands/copy.h b/src/include/commands/copy.hindex 86ba1cbe16..7ea88da894 100644--- a/src/include/commands/copy.h+++ b/src/include/commands/copy.h@@ -56,6 +56,7 @@ typedef enum CopyLogVerbosityChoice
typedef struct CopyRejectLimits
{
int64 num_err; /* maximum tolerable number of errors */
+ double ratio_err; /* maximum tolerable ratio of errors */
} CopyRejectLimits;
/*
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.outindex 8ce12f5b6f..173bd7170e 100644--- a/src/test/regress/expected/copy2.out+++ b/src/test/regress/expected/copy2.out@@ -122,6 +122,8 @@ LINE 1: COPY x to stdout (log_verbosity unsupported);
^
COPY x from stdin with (on_error ignore, reject_limit 0);
ERROR: number for REJECT_LIMIT must be greater than zero
+COPY x from stdin with (on_error ignore, reject_limit 1.1);+ERROR: float for REJECT_LIMIT must be greater than zero and smaller than 1
-- too many columns in column list: should fail
COPY x (a, b, c, d, e, d, c) from stdin;
ERROR: column "d" specified more than once
@@ -803,6 +805,11 @@ COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 4);
NOTICE: 4 rows were skipped due to data type incompatibility
COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 'INFINITY');
NOTICE: 4 rows were skipped due to data type incompatibility
+COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 0.6);+ERROR: exceeded the ratio specified by REJECT_LIMIT "0.600000", the error ratio was: "0.666667"+CONTEXT: COPY check_ign_err, line 7: ""+COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 0.7);+NOTICE: 4 rows were skipped due to data type incompatibility
-- clean up
DROP TABLE forcetest;
DROP TABLE vistest;
diff --git a/src/test/regress/sql/copy2.sql b/src/test/regress/sql/copy2.sqlindex 8cc82990f4..1b90ba0b35 100644--- a/src/test/regress/sql/copy2.sql+++ b/src/test/regress/sql/copy2.sql@@ -84,6 +84,7 @@ COPY x to stdin (format CSV, force_null(a));
COPY x to stdin (format BINARY, on_error unsupported);
COPY x to stdout (log_verbosity unsupported);
COPY x from stdin with (on_error ignore, reject_limit 0);
+COPY x from stdin with (on_error ignore, reject_limit 1.1);
-- too many columns in column list: should fail
COPY x (a, b, c, d, e, d, c) from stdin;
@@ -587,6 +588,24 @@ a {7} 7
10 {10} 10
\.
+COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 0.6);+6 {6} 6+a {7} 7+8 {8} 8888888888+9 {a, 9} 9++10 {10} 10+\.++COPY check_ign_err FROM STDIN WITH (on_error ignore, reject_limit 0.7);+6 {6} 6+a {7} 7+8 {8} 8888888888+9 {a, 9} 9++10 {10} 10+\.+
-- clean up
DROP TABLE forcetest;
DROP TABLE vistest;
--
2.39.2
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: torikoshia@oss.nttdata.com, masao.fujii@oss.nttdata.com, zhjwpku@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Add new COPY option REJECT_LIMIT
In-Reply-To: <f9feda0ecd4e2e09620a6f9ca8f4aac1@oss.nttdata.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox