MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Wed, 13 Nov 2024 21:47:21 +0900
From: torikoshia <torikoshia@oss.nttdata.com>
To: Kirill Reshke <reshkekirill@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>, Pgsql Hackers
 <pgsql-hackers@postgresql.org>
Subject: Re: Add reject_limit option to file_fdw
In-Reply-To: 
 <CALdSSPhxp71=tcVY=W9-jKv-XV0=LSrTkYqZ6NwBiM8_C91S6w@mail.gmail.com>
References: <bab68a9fc502b12693f0755b6f35f327@oss.nttdata.com>
 <fd14c3a0-a036-4bce-b5f8-4af83259eae1@oss.nttdata.com>
 <54747389a562688eab7077b144daff6a@oss.nttdata.com>
 <2b3ad6d4-e05f-4188-8cfe-1fd9d949a9c9@oss.nttdata.com>
 <192ede1e4c2b34b853e5ac46f8dee50b@oss.nttdata.com>
 <d7a03664-bef5-49b0-998f-3ea5356b53c9@oss.nttdata.com>
 <f02ec7b6c348b9fbeeb69657733da9ba@oss.nttdata.com>
 <CALdSSPhxp71=tcVY=W9-jKv-XV0=LSrTkYqZ6NwBiM8_C91S6w@mail.gmail.com>
User-Agent: Roundcube Webmail/1.4.11
Message-ID: <153fd507364f195f036695c3bb48cf44@oss.nttdata.com>
Archived-At: 
 <https://www.postgresql.org/message-id/153fd507364f195f036695c3bb48cf44%40oss.nttdata.com>
Precedence: bulk

On 2024-11-12 15:23, Kirill Reshke wrote:

Thanks for your review!

> On Tue, 12 Nov 2024 at 06:17, torikoshia <torikoshia@oss.nttdata.com> 
> wrote:
>> 
>> On 2024-11-12 01:49, Fujii Masao wrote:
>> > On 2024/11/11 21:45, torikoshia wrote:
>> >>> Thanks for adding the comment. It clearly states that REJECT_LIMIT
>> >>> can be
>> >>> a single-quoted string. However, it might also be helpful to mention
>> >>> that
>> >>> it can be provided as an int64 in the COPY command option. How about
>> >>> updating it like this?
>> >>>
>> >>> ------------------------------------
>> >>> REJECT_LIMIT can be specified in two ways: as an int64 for the COPY
>> >>> command
>> >>> option or as a single-quoted string for the foreign table option
>> >>> using
>> >>> file_fdw. Therefore this function needs to handle both formats.
>> >>> ------------------------------------
>> >>
>> >> Thanks! it seems better.
>> >>
>> >>
>> >> Attached v3 patch.
>> >
>> > Thanks for updating the patch! It looks like you forgot to attach it,
>> > though.
>> 
>> Oops, thanks for pointing it out.
>> Here it is.
>> 
>> 
>> --
>> Regards,
>> 
>> --
>> Atsushi Torikoshi
>> Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
> 
> Hi!
> 
> A little question from me.
> 
> This is your doc for reject_limit:
> 
> +  <varlistentry>
> +   <term><literal>reject_limit</literal></term>
> +
> +   <listitem>
> +    <para>
> +     Specifies the maximum number of errors tolerated while
> converting a column's
> +     input value to its data type, the same as 
> <command>COPY</command>'s
> +    <literal>REJECT_LIMIT</literal> option.
> +    </para>
> +   </listitem>
> +  </varlistentry>
> +
> 
> This is how it looks on the current HEAD for copy.
> 
> <varlistentry>
>     <term><literal>REJECT_LIMIT</literal></term>
>     <listitem>
>      <para>
>       Specifies the maximum number of errors tolerated while converting 
> a
>       column's input value to its data type, when 
> <literal>ON_ERROR</literal> is
>       set to <literal>ignore</literal>.
>       If the input causes more errors than the specified value, the
> <command>COPY</command>
>       command fails, even with <literal>ON_ERROR</literal> set to
> <literal>ignore</literal>.
>       This clause must be used with
> <literal>ON_ERROR</literal>=<literal>ignore</literal>
>       and <replaceable class="parameter">maxerror</replaceable> must
> be positive <type>bigint</type>.
>       If not specified, 
> <literal>ON_ERROR</literal>=<literal>ignore</literal>
>       allows an unlimited number of errors, meaning 
> <command>COPY</command> will
>       skip all erroneous data.
>      </para>
>     </listitem>
>    </varlistentry>
> 
> There is a difference. Should we add REJECT_LIMIT vs ON_ERROR
> clarification for file_fdw too? or maybe we put a reference for COPY
> doc.

As you may know, some options for file_fdw are the same as those for 
COPY. While the manual provides detailed descriptions of these options 
in the COPY section, the explanations in file_fdw are shorter and less 
detailed.

I intended to follow this approach, but do you think it should be 
changed?

-- 
Regards,

--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.