From: Tom Lane <tgl@sss.pgh.pa.us>
To: Cory Nemelka <cnemelka@gmail.com>
Cc: Aldo Sarmiento <aldo@bigpurpledot.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: Processing very large TEXT columns (300MB+) using C/libpq
Date: Fri, 20 Oct 2017 12:55:22 -0400
Message-ID: <4750.1508518522@sss.pgh.pa.us> (raw)
In-Reply-To: <CAMe5Gn1ksuVLRcR6DPp+nzsEsNZVVRHjMUoM-9t_JBAW2Xg4HA@mail.gmail.com>
References: <CAMe5Gn1OMeOutXDLzGOHHO28r8ZYy5URdsuTxP2R14oTMeZ8pw@mail.gmail.com>
<CAHX=r6wgC=WvEtgRYbhvNoK80-fqS9QBaJBwqyyU_xv9NpAfuw@mail.gmail.com>
<CAMe5Gn1ksuVLRcR6DPp+nzsEsNZVVRHjMUoM-9t_JBAW2Xg4HA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-admin>
Cory Nemelka <cnemelka@gmail.com> writes:
> Yes, but I should be able to read them much faster. The psql client can
> display an 11MB column in a little over a minute, while in C using libpg
> library, it takes over an hour.
Well, of course psql relies on libpq, so it seems unlikely that libpq
itself is where the time is going. Have you tried applying a profiler?
"perf" or "oprofile" or similar tool ought to pinpoint the culprit
pretty easily.
regards, tom lane
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: tgl@sss.pgh.pa.us, cnemelka@gmail.com, aldo@bigpurpledot.com
Subject: Re: Processing very large TEXT columns (300MB+) using C/libpq
In-Reply-To: <4750.1508518522@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox