agora inbox for pgsql-interfaces@postgresql.org
help / color / mirror / Atom feedFrom: Pavel Golub <pavel@microolap.com>
To: James Duong <jamesd@simba.com>
Cc: pgsql-interfaces@postgresql.org <pgsql-interfaces@postgresql.org>
Subject: Re: libpq single-row mode performance
Date: Thu, 3 Jul 2014 15:27:49 +0300
Message-ID: <13210629610.20140703152749@gf.microolap.com> (raw)
In-Reply-To: <59b2f266559a4a46835d2add16d80c39@DM2PR04MB589.namprd04.prod.outlook.com>
References: <59b2f266559a4a46835d2add16d80c39@DM2PR04MB589.namprd04.prod.outlook.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-interfaces>
Hello, James.
You wrote:
JD> Hi,
JD>
JD> I’m writing an app on top of libpq. To avoid running out of
JD> memory I’m using the single-row mode, but am finding the overhead in using this quite significant.
JD>
JD> What I see is that ~60% of my application’s run time when
JD> retrieving data is spent in calls to either PQgetResult() and
JD> PQclear(). I’ve added a switch in my app to turn off single-row
JD> mode and the performance roughly doubles.
JD>
JD> Would it be possible to optimize the single-row mode? For
JD> example, add an API call PQnextRow(PGConn*), which is only usable
JD> in single-row mode and will just update the existing result with
JD> the contents of the next row? This would let us avoid to overhead of:
JD> 1. Malloc’ing a PGResult and initializing its defaults.
JD> 2. Copying the column metadata to the new results.
JD> 3. Possibly we can avoid malloc’ing cell data if the next
JD> row has cells the same size or smaller than a previous row, though
JD> I’m not sure of the internals here.
JD> 4. Release memory with PQclear().
JD>
JD> James Duong | Senior Computer Scientist | Simba Technologies Inc.
JD> Tel +1.604.633.0008 ext. 120 | Fax +1.604.633.0004 | jamesd@simba.com
JD>
JD> 938 West 8th Avenue | Vancouver, BC | Canada | V5Z 1E5
JD> The Data Access and Analytics Experts | www.simba.com
JD>
JD>
JD>
JD> This email message is for the sole use of the intended
JD> recipient(s) and may contain confidential and privileged
JD> information. Any unauthorized review, use, disclosure, or
JD> distribution is prohibited. If you are not the intended
JD> recipient, please contact the sender by reply email and destroy
JD> all copies of the original message. Thank you.
JD>
Good idea. +1 from me. Should we send this message to pg-hackers as
well?
--
With best wishes,
Pavel mailto:pavel@gf.microolap.com
--
Sent via pgsql-interfaces mailing list (pgsql-interfaces@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-interfaces
view thread (2+ messages)
Message-ID: <13210629610.20140703152749@gf.microolap.com>
Permalink: ../13210629610.20140703152749@gf.microolap.com/
Also on: postgresql.org/message-id/13210629610.20140703152749@gf.microolap.com
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-interfaces@postgresql.org
Cc: pavel@microolap.com, jamesd@simba.com
Subject: Re: libpq single-row mode performance
In-Reply-To: <13210629610.20140703152749@gf.microolap.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox