Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
From: "Daniel Verite" <daniel@manitou-mail.org>
Subject: Re: Emitting JSON to file using COPY TO
To: "Andrew Dunstan" <andrew@dunslane.net>
Cc: Joe Conway <mail@joeconway.com>,
 Davin Shearer <davin@apache.org>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
In-Reply-To: <315b81d4-4b67-7828-0355-3808cd14acd1@dunslane.net>
Date: Wed, 06 Dec 2023 19:59:11 +0100
Message-Id: <b3a73eca-ec3b-4b64-b6c2-82418bb16701@manitou-mail.org>
Archived-At: <https://www.postgresql.org/message-id/b3a73eca-ec3b-4b64-b6c2-82418bb16701%40manitou-mail.org>
Precedence: bulk

	Andrew Dunstan wrote:

> IMNSHO, we should produce either a single JSON=20
> document (the ARRAY case) or a series of JSON documents, one per row=20
> (the LINES case).

"COPY Operations" in the doc says:

" The backend sends a CopyOutResponse message to the frontend, followed
   by zero or more CopyData messages (always one per row), followed by
   CopyDone".

In the ARRAY case, the first messages with the copyjsontest
regression test look like this (tshark output):

PostgreSQL
    Type: CopyOut response
    Length: 13
    Format: Text (0)
    Columns: 3
	Format: Text (0)
PostgreSQL
    Type: Copy data
    Length: 6
    Copy data: 5b0a
PostgreSQL
    Type: Copy data
    Length: 76
    Copy data:
207b226964223a312c226631223a226c696e652077697468205c2220696e2069743a2031=E2=
=80=A6

The first Copy data message with contents "5b0a" does not qualify
as a row of data with 3 columns as advertised in the CopyOut
message. Isn't that a problem?

At least the json non-ARRAY case ("json lines") doesn't have
this issue, since every CopyData message corresponds effectively
to a row in the table.


[1] https://www.postgresql.org/docs/current/protocol-flow.html#PROTOCOL-COPY


Best regards,
--=20
Daniel V=C3=A9rit=C3=A9
https://postgresql.verite.pro/
Twitter: @DanielVerite