Content-Type: multipart/alternative;
 boundary="------------6gTkJ3RQv3nBOATcg0ulvwd0"
Message-ID: <5c7e1ac8-5083-4c08-af19-cb9ade2f16ce@dunslane.net>
Date: Fri, 28 Feb 2025 14:46:58 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: pgsql: Refactor COPY FROM to use format callback functions.
To: Masahiko Sawada <msawada@postgresql.org>,
 pgsql-committers@lists.postgresql.org
References: <E1to58i-000AeZ-03@gemulon.postgresql.org>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAmwQTAQgARQIbAwIX
 gAIZAQULCQgHAgMiAgEGFQoJCAsCBBYCAwECHgcWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUC
 ZFlxxwUJGVGAbgAKCRCZ+n/LWfw7gXikB/9ZdcUy6CTBFIIuL/bVsc1eLEW/gJBjJBF6HxNY
 xgEkAgXAp4Lg4A5U+QB9GouFr7+GYxF0BU4hzoGhNPUWltxnHdMWP8nC/38LAqgMi8L/bbsm
 HW5YPBdWYaAZAPJQVfOAgjTbRUb26KSprpyrrJKW0ZmrZfjhNPcQ72jpWzoPLQqx2X6B0fru
 1jq+cBh8lb6r1mJTim1T3JIn+F/v5VpdQS+EL8xqsHkfzKjIPsW3CIXpkypSk6saA55Rkkbl
 26AW8ftPVB0Q6Lnn6FLt9CP0MGNixBQ55yq8r1K+nCBvCCjvQjM8RDm0UUum0WNl+ifQgTLO
 E8TWEnwVtkBf+3QWzsBNBE7KWFkBCADRnOM0FCzsYW6jtncg+dWIagjUZpvaClmqn/sJluLa
 Q3v1VXMQJzYs3eC1gh386W+XBwLRpDj3jzH81lX+p73Re3d3oJW7X+ffsxuzu5ZVdMUkqBYo
 nkAbKxr6gyJ12F/+JkUVzLcoTN+d/7YsQvUVi7NaKH8mJgjz112O4fUe3p9wfAaFa0RXHc5S
 GPzRTYRRlv/XZBIho4J2tkZOnteZJZ+GbxQVlINt6fd8P6al3MWOvpP/ExJPguEfjOsO6Njy
 xjo3WfpD4lHMOR/Oc3/8mScEF84rF2jXbsFgelWnbPWAvXY+pD0dXOFRkagGmC/viwBDqq5b
 5tk76kKmUbZxABEBAAHCwHwEGAEIACYCGwwWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUCZFlx
 5wUJGVGAjgAKCRCZ+n/LWfw7gf+iB/4g8CPY5jihf5r/8EsoIGe2H+dpVmpPF8YGBzTIvCz/
 fQoOq8AX/pE76QEuFnFZWfjw+wgBXgCVmkox2Eflkk6z4ND3pcwGZ6CfCxTQCDk/dij+2DQ4
 6bmDCy/sBgcbz9mTpoLC11HLoPae6YN9nBNQRZDcEFEu54OaVOqlIdbA6m+POIBCXZdHOFc0
 WoDTgxHRzC1jgQNidyd6tKqcsVJs0dzF0oKTmFFmUAqTdJO12LBuNA1rlqrR3EtpYk8B/wtS
 5dIMD7Q8hwQpL+4C6GNpb6ZKnPkLi47pDOLhz2qBrqN+rqUEsT3YnExYpzj5yOBi+FlmV1Hw
 49QYe1sn2ZPs
In-Reply-To: <E1to58i-000AeZ-03@gemulon.postgresql.org>
Archived-At: 
 <https://www.postgresql.org/message-id/5c7e1ac8-5083-4c08-af19-cb9ade2f16ce%40dunslane.net>
Precedence: bulk

This is a multi-part message in MIME format.
--------------6gTkJ3RQv3nBOATcg0ulvwd0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 2025-02-28 Fr 1:31 PM, Masahiko Sawada wrote:
> Refactor COPY FROM to use format callback functions.
>
> This commit introduces a new CopyFromRoutine struct, which is a set of
> callback routines to read tuples in a specific format. It also makes
> COPY FROM with the existing formats (text, CSV, and binary) utilize
> these format callbacks.
>
> This change is a preliminary step towards making the COPY FROM command
> extensible in terms of input formats.
>
> Similar to 2e4127b6d2d, this refactoring contributes to a performance
> improvement by reducing the number of "if" branches that need to be
> checked on a per-row basis when sending field representations in text
> or CSV mode. The performance benchmark results showed ~5% performance
> gain in text or CSV mode.
>
> Author: Sutou Kouhei<kou@clear-code.com>
> Reviewed-by: Masahiko Sawada<sawada.mshk@gmail.com>
> Reviewed-by: Michael Paquier<michael@paquier.xyz>
> Reviewed-by: Andres Freund<andres@anarazel.de>
> Reviewed-by: Tomas Vondra<tomas.vondra@enterprisedb.com>
> Reviewed-by: Junwang Zhao<zhjwpku@gmail.com>
> Discussion:https://postgr.es/m/20231204.153548.2126325458835528809.kou@clear-code.com


This patch has completely broken the file_textarray fdw, which uses 
NextCopyFromRawFields(). Removing that from API is not a good thing.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

--------------6gTkJ3RQv3nBOATcg0ulvwd0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 2025-02-28 Fr 1:31 PM, Masahiko
      Sawada wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:E1to58i-000AeZ-03@gemulon.postgresql.org">
      <pre wrap="" class="moz-quote-pre">Refactor COPY FROM to use format callback functions.

This commit introduces a new CopyFromRoutine struct, which is a set of
callback routines to read tuples in a specific format. It also makes
COPY FROM with the existing formats (text, CSV, and binary) utilize
these format callbacks.

This change is a preliminary step towards making the COPY FROM command
extensible in terms of input formats.

Similar to 2e4127b6d2d, this refactoring contributes to a performance
improvement by reducing the number of "if" branches that need to be
checked on a per-row basis when sending field representations in text
or CSV mode. The performance benchmark results showed ~5% performance
gain in text or CSV mode.

Author: Sutou Kouhei <a class="moz-txt-link-rfc2396E" href="mailto:kou@clear-code.com">&lt;kou@clear-code.com&gt;</a>
Reviewed-by: Masahiko Sawada <a class="moz-txt-link-rfc2396E" href="mailto:sawada.mshk@gmail.com">&lt;sawada.mshk@gmail.com&gt;</a>
Reviewed-by: Michael Paquier <a class="moz-txt-link-rfc2396E" href="mailto:michael@paquier.xyz">&lt;michael@paquier.xyz&gt;</a>
Reviewed-by: Andres Freund <a class="moz-txt-link-rfc2396E" href="mailto:andres@anarazel.de">&lt;andres@anarazel.de&gt;</a>
Reviewed-by: Tomas Vondra <a class="moz-txt-link-rfc2396E" href="mailto:tomas.vondra@enterprisedb.com">&lt;tomas.vondra@enterprisedb.com&gt;</a>
Reviewed-by: Junwang Zhao <a class="moz-txt-link-rfc2396E" href="mailto:zhjwpku@gmail.com">&lt;zhjwpku@gmail.com&gt;</a>
Discussion: <a class="moz-txt-link-freetext" href="https://postgr.es/m/20231204.153548.2126325458835528809.kou@clear-code.com">https://postgr.es/m/20231204.153548.2126325458835528809.kou@clear-code.com</a>
</pre>
    </blockquote>
    <p><br>
    </p>
    <p><br>
    </p>
    <p>This patch has completely broken the file_textarray fdw, which
      uses NextCopyFromRawFields(). Removing that from API is not a good
      thing.</p>
    <p><br>
    </p>
    <p>cheers</p>
    <p><br>
    </p>
    <p>andrew<br>
    </p>
    <p><span style="white-space: pre-wrap">
</span></p>
    <pre class="moz-signature"
    signature-switch-id="d0437855-2267-4610-80b3-83167ec45b0b" cols="72">--
Andrew Dunstan
EDB: <a class="moz-txt-link-freetext" href="https://www.enterprisedb.com">https://www.enterprisedb.com</a>
</pre>
  </body>
</html>

--------------6gTkJ3RQv3nBOATcg0ulvwd0--