From: Euler Taveira <euler@eulerto.com>
To: Daniel Verite <daniel@manitou-mail.org>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Joe Conway <mail@joeconway.com>
Cc: Davin Shearer <davin@apache.org>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Emitting JSON to file using COPY TO
Date: Wed, 06 Dec 2023 23:42:06 -0300
Message-ID: <6ab82c2f-c19f-4cd6-a2a9-ddf9832e6209@app.fastmail.com> (raw)
In-Reply-To: <b3a73eca-ec3b-4b64-b6c2-82418bb16701@manitou-mail.org>
References: <b3a73eca-ec3b-4b64-b6c2-82418bb16701@manitou-mail.org>
On Wed, Dec 6, 2023, at 3:59 PM, Daniel Verite wrote:
> The first Copy data message with contents "5b0a" does not qualify
> as a row of data with 3 columns as advertised in the CopyOut
> message. Isn't that a problem?
>
> At least the json non-ARRAY case ("json lines") doesn't have
> this issue, since every CopyData message corresponds effectively
> to a row in the table.
Moreover, if your interface wants to process the COPY data stream while
receiving it, you cannot provide "json array" format because each row (plus all
of the received ones) is not a valid JSON. Hence, a JSON parser cannot be
executed until you receive the whole data set. (wal2json format 1 has this
disadvantage. Format 2 was born to provide a better alternative -- each row is
a valid JSON.) I'm not saying that "json array" is not useful but that for
large data sets, it is less useful.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: euler@eulerto.com, daniel@manitou-mail.org, andrew@dunslane.net, mail@joeconway.com, davin@apache.org, pgsql-hackers@postgresql.org
Subject: Re: Emitting JSON to file using COPY TO
In-Reply-To: <6ab82c2f-c19f-4cd6-a2a9-ddf9832e6209@app.fastmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox