pg.ddx.io  pgsql-general@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Andrew Dunstan <andrew@dunslane.net>
To: Joe Conway <mail@joeconway.com>
To: Davin Shearer <davin@apache.org>
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: Emitting JSON to file using COPY TO
Date: Sun, 3 Dec 2023 14:52:29 -0500
Message-ID: <6603ea04-8ec3-67b2-e9f6-e00fd0e46858@dunslane.net> (raw)
In-Reply-To: <a70d33fb-76f8-40a8-9a5f-7e03b120e661@joeconway.com>
References: <CALvfUkBxTYy5uWPFVwpk_7ii2zgT07t3d-yR_cy4sfrrLU=kcg@mail.gmail.com>
	<fc1e39f8-274f-4be6-8a70-9fb0d9fe84a2@aklaver.com>
	<CAFCRh-_LhP5Ln4-beVz=1pQ9VycU3hAxnJHXhPHZRmX0BSCVEQ@mail.gmail.com>
	<CAFCRh--Vhx0105S7SWAYHGcVMTSRtbcoArDUGSbYSWRUGunW_g@mail.gmail.com>
	<CAKFQuwZb3zaSqkOvGXRRLZEW61az+4xyHZ7tBPR91JfrcdCKvA@mail.gmail.com>
	<3853387.1701096982@sss.pgh.pa.us>
	<CAFCRh-_GdiUvjd5z5FfvTfhruOnYqBu163XU47zZE8RNATCJGQ@mail.gmail.com>
	<3a98decf-3fe3-4b49-9b68-fda01338872c@sedlakovi.org>
	<CALvfUkCq3pvkmniWEKZwkbA6hNNvkANF3OamMfhC170RgY+HJg@mail.gmail.com>
	<24e3ee88-ec1e-421b-89ae-8a47ee0d2df1@joeconway.com>
	<b22367ef-6ac1-e1c0-da05-0e760e1b7e29@dunslane.net>
	<8c88da85-c197-4765-96f8-a9a1c78305a6@joeconway.com>
	<a70d33fb-76f8-40a8-9a5f-7e03b120e661@joeconway.com>


On 2023-12-03 Su 14:24, Joe Conway wrote:
> On 12/3/23 11:03, Joe Conway wrote:
>> On 12/3/23 10:10, Andrew Dunstan wrote:
>>> I  realize this is just a POC, but I'd prefer to see 
>>> composite_to_json()
>>> not exposed. You could use the already public datum_to_json() instead,
>>> passing JSONTYPE_COMPOSITE and F_RECORD_OUT as the second and third
>>> arguments.
>>
>> Ok, thanks, will do
>
> Just FYI, this change does loose some performance in my not massively 
> scientific A/B/A test:
>
> 8<---------------------------
> -- with datum_to_json()
> test=# \timing
> Timing is on.
> test=# copy foo to '/tmp/buf' (format json, force_array);
> COPY 10000000
> Time: 37196.898 ms (00:37.197)
> Time: 37408.161 ms (00:37.408)
> Time: 38393.309 ms (00:38.393)
> Time: 36855.438 ms (00:36.855)
> Time: 37806.280 ms (00:37.806)
>
> Avg = 37532
>
> -- original patch
> test=# \timing
> Timing is on.
> test=# copy foo to '/tmp/buf' (format json, force_array);
> COPY 10000000
> Time: 37426.207 ms (00:37.426)
> Time: 36068.187 ms (00:36.068)
> Time: 38285.252 ms (00:38.285)
> Time: 36971.042 ms (00:36.971)
> Time: 35690.822 ms (00:35.691)
>
> Avg = 36888
>
> -- with datum_to_json()
> test=# \timing
> Timing is on.
> test=# copy foo to '/tmp/buf' (format json, force_array);
> COPY 10000000
> Time: 39083.467 ms (00:39.083)
> Time: 37249.326 ms (00:37.249)
> Time: 38529.721 ms (00:38.530)
> Time: 38704.920 ms (00:38.705)
> Time: 39001.326 ms (00:39.001)
>
> Avg = 38513
> 8<---------------------------
>
> That is somewhere in the 3% range.


I assume it's because datum_to_json() constructs a text value from which 
you then need to extract the cstring, whereas composite_to_json(), just 
gives you back the stringinfo. I guess that's a good enough reason to go 
with exposing composite_to_json().


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com






view thread (163+ messages)  latest in thread

Message-ID: <6603ea04-8ec3-67b2-e9f6-e00fd0e46858@dunslane.net>
Permalink:  ../6603ea04-8ec3-67b2-e9f6-e00fd0e46858@dunslane.net/
Also on:    postgresql.org/message-id/6603ea04-8ec3-67b2-e9f6-e00fd0e46858@dunslane.net

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-general@postgresql.org
  Cc: andrew@dunslane.net, mail@joeconway.com, davin@apache.org, pgsql-hackers@postgreSQL.org
  Subject: Re: Emitting JSON to file using COPY TO
  In-Reply-To: <6603ea04-8ec3-67b2-e9f6-e00fd0e46858@dunslane.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox