pg.ddx.io  pgsql-general@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Nathan Bossart <nathandbossart@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Joe Conway <mail@joeconway.com>
Cc: Davin Shearer <davin@apache.org>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Emitting JSON to file using COPY TO
Date: Wed, 6 Dec 2023 10:33:49 -0600
Message-ID: <20231206163349.GA2873889@nathanxps13> (raw)
In-Reply-To: <1110690.1701880139@sss.pgh.pa.us>
References: <2554e520-e103-8978-dcb5-807dfeb77402@dunslane.net>
	<cb1f118c-4698-4451-9ba9-dd8595f06c37@joeconway.com>
	<926ff917-8371-40ec-b5e6-ab7b0e09bdc5@joeconway.com>
	<315b81d4-4b67-7828-0355-3808cd14acd1@dunslane.net>
	<7a60faf6-e7f1-419d-aee6-10a78ea2fe81@joeconway.com>
	<2e7ff718-895d-83fc-46f7-be25e23b23b4@dunslane.net>
	<!bbf922c5-f571-483e-a2c6-2730b02183fe@joeconway.com>
	<1104915.1701877459@sss.pgh.pa.us>
	<19a5f9d8-bd1f-9!e51-0f5b-510c1189a8a7@dunslane.net>
	<1110690.1701880139@sss.pgh.pa.us>

On Wed, Dec 06, 2023 at 11:28:59AM -0500, Tom Lane wrote:
> It might be acceptable to plan on improving the performance later,
> depending on just how bad it is now.

On 10M rows with 11 integers each, I'm seeing the following:

	(format text)
	Time: 10056.311 ms (00:10.056)
	Time: 8789.331 ms (00:08.789)
	Time: 8755.070 ms (00:08.755)

	(format csv)
	Time: 12295.480 ms (00:12.295)
	Time: 12311.059 ms (00:12.311)
	Time: 12305.469 ms (00:12.305)

	(format json)
	Time: 24568.621 ms (00:24.569)
	Time: 23756.234 ms (00:23.756)
	Time: 24265.730 ms (00:24.266)

'perf top' tends to look a bit like this:

  13.31%  postgres                      [.] appendStringInfoString
   7.57%  postgres                      [.] datum_to_json_internal
   6.82%  postgres                      [.] SearchCatCache1
   5.35%  [kernel]                      [k] intel_gpio_irq
   3.57%  postgres                      [.] composite_to_json
   3.31%  postgres                      [.] IsValidJsonNumber

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com





view thread (163+ messages)  latest in thread

Message-ID: <20231206163349.GA2873889@nathanxps13>
Permalink:  ../20231206163349.GA2873889@nathanxps13/
Also on:    postgresql.org/message-id/20231206163349.GA2873889@nathanxps13

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-general@postgresql.org
  Cc: nathandbossart@gmail.com, tgl@sss.pgh.pa.us, andrew@dunslane.net, mail@joeconway.com, davin@apache.org, pgsql-hackers@postgresql.org
  Subject: Re: Emitting JSON to file using COPY TO
  In-Reply-To: <20231206163349.GA2873889@nathanxps13>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox