From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: torikoshia <torikoshia@oss.nttdata.com>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: Add new COPY option REJECT_LIMIT
Date: Sat, 20 Jul 2024 02:08:39 +0900
Message-ID: <54bea7dd-94b1-41f1-a4e9-c5c53cc82ce7@oss.nttdata.com> (raw)
In-Reply-To: <a873643c-e7d1-456c-b16b-6162ab698036@oss.nttdata.com>
References: <63f99327aa6b404cc951217fa3e61fe4@oss.nttdata.com>
<fab7018f-6e00-4be4-bd1e-4a013096f6c9@oss.nttdata.com>
<de0551d0f9d9e9072e324e51ed5c426d@oss.nttdata.com>
<6e3dcdc6-8e18-4716-a23b-3bee316aeaa7@oss.nttdata.com>
<979c6d306343f3d665d14eab2cd1b7bc@oss.nttdata.com>
<920a29a36032befbeb355cbdfbbe0f63@oss.nttdata.com>
<a873643c-e7d1-456c-b16b-6162ab698036@oss.nttdata.com>
On 2024/07/19 22:03, Fujii Masao wrote:
>
>
> On 2024/07/17 22:21, torikoshia wrote:
>> On 2024-07-03 02:07, Fujii Masao wrote:
>>> However, if we support REJECT_LIMIT, I'm not sure if the ON_ERROR option is still necessary.
>>
>> I remembered another reason for the necessity of ON_ERROR.
>>
>> ON_ERROR defines how to behave when encountering an error and it just accepts 'ignore' and 'stop' currently, but is expected to support other options such as saving details of errors to a table[1].
>
> Wouldn't it be better to separate the option specifying where
> error details are output from the ON_ERROR option
> (which determines behavior when encountering errors)?
> "table" seems valid for both ON_ERROR=ignore and ON_ERROR=stop.
I still find it odd to accept "table" as a value for ON_ERROR. However,
"set_to_null" or "replace-column" proposed in [1] seem valid for
ON_ERROR. So, I'm okay with keeping the ON_ERROR option.
> On my second thought, whatever value ON_ERROR is specified(e.g. ignore, stop, table), it seems fine to use REJECT_LIMIT.
> I feel REJECT_LIMIT has both "ignore" and "stop" characteristics, meaning it ignores errors until it reaches REJECT_LIMIT and stops when it exceeds the REJECT_LIMIT.
ON_ERROR specifies how to handle errors, and "stop" means to fail
the command. So, if ON_ERROR=stop, REJECT_LIMIT should have no effect,
and the command should fail immediately upon encountering an error.
As in your original proposal, I now think REJECT_LIMIT should only
apply when ON_ERROR=ignore. The command would ignore errors and
continue processing, but if the number of errors exceeds REJECT_LIMIT,
the command should fail. Thought?
BTW if "set_to_null" is supported someday, REJECT_LIMIT can also
apply. The command would cinsert NULL into the target table upon
encountering errors and continue, but fail if the number of errors
exceed REJECT_LIMIT.
Regards,
[1] https://www.postgresql.org/message-id/flat/CAKFQuwawy1e6YR4S=j+y7pXqg_Dw1WBVrgvf=BP3d1_aSfe_+Q@mail....
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: masao.fujii@oss.nttdata.com, torikoshia@oss.nttdata.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Add new COPY option REJECT_LIMIT
In-Reply-To: <54bea7dd-94b1-41f1-a4e9-c5c53cc82ce7@oss.nttdata.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox