From: Tom Lane <tgl@sss.pgh.pa.us>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: psql \copy from sends a lot of packets
Date: Sat, 06 Feb 2021 17:23:33 -0500
Message-ID: <334619.1612650213@sss.pgh.pa.us> (raw)
In-Reply-To: <40b2cec0-d0fb-3191-2ae1-9a3fe16a7e48@iki.fi>
References: <40b2cec0-d0fb-3191-2ae1-9a3fe16a7e48@iki.fi>
Heikki Linnakangas <hlinnaka@iki.fi> writes:
> I just noticed that if you load a file using psql:
> it sends every line as a separate FE/BE protocol CopyData packet.
> ...
> I'll add this to the next commitfest. There's similar inefficiency in
> the server side in COPY TO, but I'll leave that for another patch.
The FE/BE protocol documentation is pretty explicit about this:
Copy-in mode (data transfer to the server) is initiated when the
backend executes a COPY FROM STDIN SQL statement. The backend sends a
CopyInResponse message to the frontend. The frontend should then send
zero or more CopyData messages, forming a stream of input data. (The
message boundaries are not required to have anything to do with row
boundaries, although that is often a reasonable choice.)
...
Copy-out mode (data transfer from the server) is initiated when the
backend executes a COPY TO STDOUT SQL statement. The backend sends a
CopyOutResponse message to the frontend, followed by zero or more
CopyData messages (always one per row), followed by CopyDone.
So while changing psql isn't so much a problem, changing the server
is a wire protocol break. Maybe we should do it anyway, but I'm
not sure.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us, hlinnaka@iki.fi
Subject: Re: psql \copy from sends a lot of packets
In-Reply-To: <334619.1612650213@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox