public inbox for [email protected]  

help / color / mirror / Atom feed

From: Jeroen Vermeulen <[email protected]>
To: Bharath Rupireddy <[email protected]>
Cc: [email protected]
Subject: Re: libpq: PQgetCopyData() and allocation overhead
Date: Wed, 1 Mar 2023 15:23:45 +0100
Message-ID: <CA+zULE5BoZtkYMf9yPGH+BnPEPspXENVCZd_b4s=ayewU6DHZg@mail.gmail.com> (raw)
In-Reply-To: <CA+zULE6aB9Y_CgT_rmGFPpXUuEANmVXi7_RbrZLH7EPhzy3-OA@mail.gmail.com>
References: <CA+zULE5AYd+78w7Uu7jN=5N3w-9vsKuK2WrpjkKwS65hyhWfbw@mail.gmail.com>
	<CALj2ACXiC3Wy7o=81j8jpY=nQq3Hs4y+06gXDkOBrmoO+JDrrw@mail.gmail.com>
	<CA+zULE6bV9yg9+=u0KAhjz-C_YpDr6bh5YknkffgqDHiKWugOg@mail.gmail.com>
	<CA+zULE56RADYr4R=ss=bvrzA2r6PyzWE98Gkkwr_7nOzKsSd-A@mail.gmail.com>
	<CALj2ACVaBEbFnBwJ44Uuy7xwr50M+wwTJoEz0C9dFOc=iZeTDw@mail.gmail.com>
	<CA+zULE6aB9Y_CgT_rmGFPpXUuEANmVXi7_RbrZLH7EPhzy3-OA@mail.gmail.com>

Update: in the latest iteration, I have an alternative that reduces CPU
time by more than half, compared to PQgetCopyData().

And the code is simpler, too, both in the client and in libpq itself.  The
one downside is that it breaks with libpq's existing API style.

PR for easy discussion: https://github.com/jtv/postgres/pull/1


Jeroen

On Mon, 27 Feb 2023 at 17:08, Jeroen Vermeulen <[email protected]> wrote:

> Done.  Thanks for looking!
>
> Jelte Fennema pointed out that I should probably be using PQExpBuffer for
> this.  I'll look into that later; this is a proof of concept, not a
> production-ready API proposal.
>
>
> Jeroen
>
> On Mon, 27 Feb 2023 at 14:48, Bharath Rupireddy <
> [email protected]> wrote:
>
>> On Fri, Feb 10, 2023 at 5:49 PM Jeroen Vermeulen <[email protected]>
>> wrote:
>> >
>> > OK, I've updated the PR with a benchmark (in the main directory).
>> >
>> > On this benchmark I'm seeing about a 24% reduction in "user" CPU time,
>> and a 8% reduction in "system" CPU time.  (Almost no reduction in
>> wall-clock time.)
>>
>> I can help run some logical replication performance benchmarks
>> tomorrow. Would you mind cleaning the PR and providing the changes
>> (there are multiple commits in the PR) as a single patch here for the
>> sake of ease of review and test?
>>
>> --
>> Bharath Rupireddy
>> PostgreSQL Contributors Team
>> RDS Open Source Databases
>> Amazon Web Services: https://aws.amazon.com
>>
>

view thread (6+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: libpq: PQgetCopyData() and allocation overhead
  In-Reply-To: <CA+zULE5BoZtkYMf9yPGH+BnPEPspXENVCZd_b4s=ayewU6DHZg@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox