From: torikoshia <torikoshia@oss.nttdata.com>
To: Kirill Reshke <reshkekirill@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Add reject_limit option to file_fdw
Date: Wed, 13 Nov 2024 21:47:21 +0900
Message-ID: <153fd507364f195f036695c3bb48cf44@oss.nttdata.com> (raw)
In-Reply-To: <CALdSSPhxp71=tcVY=W9-jKv-XV0=LSrTkYqZ6NwBiM8_C91S6w@mail.gmail.com>
References: <bab68a9fc502b12693f0755b6f35f327@oss.nttdata.com>
<fd14c3a0-a036-4bce-b5f8-4af83259eae1@oss.nttdata.com>
<54747389a562688eab7077b144daff6a@oss.nttdata.com>
<2b3ad6d4-e05f-4188-8cfe-1fd9d949a9c9@oss.nttdata.com>
<192ede1e4c2b34b853e5ac46f8dee50b@oss.nttdata.com>
<d7a03664-bef5-49b0-998f-3ea5356b53c9@oss.nttdata.com>
<f02ec7b6c348b9fbeeb69657733da9ba@oss.nttdata.com>
<CALdSSPhxp71=tcVY=W9-jKv-XV0=LSrTkYqZ6NwBiM8_C91S6w@mail.gmail.com>
On 2024-11-12 15:23, Kirill Reshke wrote:
Thanks for your review!
> On Tue, 12 Nov 2024 at 06:17, torikoshia <torikoshia@oss.nttdata.com>
> wrote:
>>
>> On 2024-11-12 01:49, Fujii Masao wrote:
>> > On 2024/11/11 21:45, torikoshia wrote:
>> >>> Thanks for adding the comment. It clearly states that REJECT_LIMIT
>> >>> can be
>> >>> a single-quoted string. However, it might also be helpful to mention
>> >>> that
>> >>> it can be provided as an int64 in the COPY command option. How about
>> >>> updating it like this?
>> >>>
>> >>> ------------------------------------
>> >>> REJECT_LIMIT can be specified in two ways: as an int64 for the COPY
>> >>> command
>> >>> option or as a single-quoted string for the foreign table option
>> >>> using
>> >>> file_fdw. Therefore this function needs to handle both formats.
>> >>> ------------------------------------
>> >>
>> >> Thanks! it seems better.
>> >>
>> >>
>> >> Attached v3 patch.
>> >
>> > Thanks for updating the patch! It looks like you forgot to attach it,
>> > though.
>>
>> Oops, thanks for pointing it out.
>> Here it is.
>>
>>
>> --
>> Regards,
>>
>> --
>> Atsushi Torikoshi
>> Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
>
> Hi!
>
> A little question from me.
>
> This is your doc for reject_limit:
>
> + <varlistentry>
> + <term><literal>reject_limit</literal></term>
> +
> + <listitem>
> + <para>
> + Specifies the maximum number of errors tolerated while
> converting a column's
> + input value to its data type, the same as
> <command>COPY</command>'s
> + <literal>REJECT_LIMIT</literal> option.
> + </para>
> + </listitem>
> + </varlistentry>
> +
>
> This is how it looks on the current HEAD for copy.
>
> <varlistentry>
> <term><literal>REJECT_LIMIT</literal></term>
> <listitem>
> <para>
> Specifies the maximum number of errors tolerated while converting
> a
> column's input value to its data type, when
> <literal>ON_ERROR</literal> is
> set to <literal>ignore</literal>.
> If the input causes more errors than the specified value, the
> <command>COPY</command>
> command fails, even with <literal>ON_ERROR</literal> set to
> <literal>ignore</literal>.
> This clause must be used with
> <literal>ON_ERROR</literal>=<literal>ignore</literal>
> and <replaceable class="parameter">maxerror</replaceable> must
> be positive <type>bigint</type>.
> If not specified,
> <literal>ON_ERROR</literal>=<literal>ignore</literal>
> allows an unlimited number of errors, meaning
> <command>COPY</command> will
> skip all erroneous data.
> </para>
> </listitem>
> </varlistentry>
>
> There is a difference. Should we add REJECT_LIMIT vs ON_ERROR
> clarification for file_fdw too? or maybe we put a reference for COPY
> doc.
As you may know, some options for file_fdw are the same as those for
COPY. While the manual provides detailed descriptions of these options
in the COPY section, the explanations in file_fdw are shorter and less
detailed.
I intended to follow this approach, but do you think it should be
changed?
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: torikoshia@oss.nttdata.com, reshkekirill@gmail.com, masao.fujii@oss.nttdata.com
Subject: Re: Add reject_limit option to file_fdw
In-Reply-To: <153fd507364f195f036695c3bb48cf44@oss.nttdata.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox