pg.ddx.io  pgsql-php@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: vinny <vinny@xs4all.nl>
To: Marco Dieckhoff <marco.dieckhoff@googlemail.com>
Cc: pgsql-php@postgresql.org
Subject: Re: large resultset
Date: Mon, 21 Jun 2010 21:52:14 +0200
Message-ID: <1277149934.9667.21.camel@bigboy> (raw)
In-Reply-To: <4C174CC5.5080708@googlemail.com>
References: <AANLkTinXSz6PsoTyPhryAHsN3V-M0v-m3sTCLTJSJttO@mail.gmail.com>
	<AANLkTimP7qSASY3dRA09p6e35CzRLo8pOyZ6J5_5Bisx@mail.gmail.com>
	<8f468dc246fb6075f2a1f46b09f71a6d@xs4all.nl>
	<4C174CC5.5080708@googlemail.com>

On Tue, 2010-06-15 at 11:49 +0200, Marco Dieckhoff wrote:
> Am 15.06.2010 11:40, schrieb vinny:
> > On Tue, 15 Jun 2010 10:33:05 +0100, Thom Brown<thombrown@gmail.com>
> > wrote:
> >    
> >> On 15 June 2010 09:31, AI Rumman  wrote:
> >>   How to return large resutlset (almost 2 millions record) in php?
> >>
> >>
> >> Obviously that wouldn't do by itself, but it's quite simple to loop over
> >> a result set.
> >>      
> > If that's the case surely you'd use some SQL to merge the data into one
> > long string and return it in a single record.
> > Looping over two million results is going to take a while, not to mention
> > quite a bit of memory.
> > v.
> >    
> 
> huh?
> 
> Returning one large record containg the information of two million 
> records will almost certainly fill up your memory.
> First the memory of the sql server, in order to process the combination 
> of two million records, and if that's even successfull, the memory of 
> the server processing php.

True, I was a little optimistic there :-)

> Looping may take "a little" longer, but as only one row at a time is 
> fetched, it should be the least possible memory use.

As far as I know, PHP always downloads the entire queryresult at once
and makes it available locally. The database may have less trouble
sending two million seperate records but PHP will still need enough
memory to hold the entire resultset at once.

If you'd really want to reduce memoryusage you'd have to execute several
queries that fetch a few thousand rows at a time so you really only deal
with a small portion of the records both on the dbserver and in PHP.




view thread (14+ messages)

Message-ID: <1277149934.9667.21.camel@bigboy>
Permalink:  ../1277149934.9667.21.camel@bigboy/
Also on:    postgresql.org/message-id/1277149934.9667.21.camel@bigboy

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-php@postgresql.org
  Cc: vinny@xs4all.nl, marco.dieckhoff@googlemail.com
  Subject: Re: large resultset
  In-Reply-To: <1277149934.9667.21.camel@bigboy>
  References: <AANLkTinXSz6PsoTyPhryAHsN3V-M0v-m3sTCLTJSJttO@mail.gmail.com> <AANLkTimP7qSASY3dRA09p6e35CzRLo8pOyZ6J5_5Bisx@mail.gmail.com> <8f468dc246fb6075f2a1f46b09f71a6d@xs4all.nl> <4C174CC5.5080708@googlemail.com> <1277149934.9667.21.camel@bigboy>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox