From: Jasen Betts <jasen@xnet.co.nz>
To: pgsql-php@postgresql.org
Subject: Re: large resultset
Date: 15 Jun 2010 12:09:25 GMT
Message-ID: <hv7qhl$ogq$1@reversiblemaps.ath.cx> (raw)
References: <AANLkTinXSz6PsoTyPhryAHsN3V-M0v-m3sTCLTJSJttO@mail.gmail.com>
<AANLkTimP7qSASY3dRA09p6e35CzRLo8pOyZ6J5_5Bisx@mail.gmail.com>
<8f468dc246fb6075f2a1f46b09f71a6d@xs4all.nl>
On 2010-06-15, vinny <vinny@xs4all.nl> wrote:
>
> On Tue, 15 Jun 2010 10:33:05 +0100, Thom Brown <thombrown@gmail.com>
> wrote:
>> On 15 June 2010 09:31, AI Rumman wrote:
>> How to return large resutlset (almost 2 millions record) in php?
>>
>> Presumably this is so people can download results rather than display
>> them in a browser?
>>
>> Here's a basic a raw version of what you can do:
>>
>> $db = pg_connect($connection_string);
>>
>> $results = pg_query($db, $query);
>>
>> while ($result = pg_fetch_array($results)
>> {
>> echo implode("|", $result) . "n";
>> }
>>
>> pg_close($db);
>>
>> Obviously that wouldn't do by itself, but it's quite simple to loop over
>> a result set.
>>
>> Regards
>>
>> Thom
>>
>
> If that's the case surely you'd use some SQL to merge the data into one
> long string and return it in a single record.
> Looping over two million results is going to take a while,
yeah, so is transmitting the bytes.
> not to mention quite a bit of memory.
yeah, libpq is going to want to load the result-set into memory.
you'll probably hit PHPs memory limit and it'll be game over.
The only pure php solution is to use a cursor. (seing as pg_get_line
is not avaulable)
OTOH you can do copy like this...
$c=escapeshellarg($connstr);
$q=escapeshellarg($query);
system ("psql $c -c 'copy ( '$q' ) to stdout with csv header'");
needs *nix based server and pgversion >= 8.3 (I think).
You don't get to choose the format beyond what pg supports, but I
think you can tweak encoding on the command-line if needed.
OTOH you could use ICONV for that.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-php@postgresql.org
Cc: jasen@xnet.co.nz
Subject: Re: large resultset
In-Reply-To: <hv7qhl$ogq$1@reversiblemaps.ath.cx>
References: <AANLkTinXSz6PsoTyPhryAHsN3V-M0v-m3sTCLTJSJttO@mail.gmail.com> <AANLkTimP7qSASY3dRA09p6e35CzRLo8pOyZ6J5_5Bisx@mail.gmail.com> <8f468dc246fb6075f2a1f46b09f71a6d@xs4all.nl> <hv7qhl$ogq$1@reversiblemaps.ath.cx>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox