pg.ddx.io pgsql-interfaces@postgresql.org mailing list archive
help / color / mirror / Atom feedHandling Blobs with libpq
6+ messages / 6 participants
[nested] [flat]
* Handling Blobs with libpq
@ 2003-03-20 09:48 Stéphane Pinel <spinel@exenevex.com>
0 siblings, 0 replies; 6+ messages in thread
From: Stéphane Pinel @ 2003-03-20 09:48 UTC (permalink / raw)
To: pgsql-interfaces
What is the best approach to handle blobs with libpq ?
I've read in documentation that there are 2 ways:
1- Bytea
2- Larges objects (lo_import)
My main need is storing pictures (like Jpegs etc). Doc says that (2) is
more
appropriate but I need to work with blobs in memory, not from files.
Thanks for your help.
Stephane
^ permalink raw reply [nested|flat] 6+ messages in thread
* Handling Blobs with libpq
@ 2003-03-20 09:56 Stéphane Pinel <spinel@noos.fr>
2003-03-20 15:11 ` Re: Handling Blobs with libpq Jeroen T. Vermeulen <jtv@xs4all.nl>
2003-03-20 21:44 ` Re: Handling Blobs with libpq Nigel J. Andrews <nandrews@investsystems.co.uk>
2003-03-21 22:42 ` Re: Handling Blobs with libpq Daniel Bruce Lynes <dlynes@shaw.ca>
0 siblings, 3 replies; 6+ messages in thread
From: Stéphane Pinel @ 2003-03-20 09:56 UTC (permalink / raw)
To: pgsql-interfaces
What is the best approach to handle blobs with libpq ?
I've read in documentation that there are 2 ways:
1- Bytea
2- Larges objects (lo_import)
My main need is storing pictures (like Jpegs etc) or small blobs. Doc
says that (2) is more
appropriate but I need to work with blobs in memory, not from files.
Thanks for your help.
------------------------------------------------------------------------
---
Stephane
------------------------------------------------------------------------
---
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Handling Blobs with libpq
2003-03-20 09:56 Handling Blobs with libpq Stéphane Pinel <spinel@noos.fr>
@ 2003-03-20 15:11 ` Jeroen T. Vermeulen <jtv@xs4all.nl>
2 siblings, 0 replies; 6+ messages in thread
From: Jeroen T. Vermeulen @ 2003-03-20 15:11 UTC (permalink / raw)
To: Stéphane Pinel <spinel@noos.fr>; +Cc: pgsql-interfaces
On Thu, Mar 20, 2003 at 10:56:44AM +0100, Stéphane Pinel wrote:
>
> 2- Larges objects (lo_import)
>
> My main need is storing pictures (like Jpegs etc) or small blobs. Doc
> says that (2) is more
> appropriate but I need to work with blobs in memory, not from files.
You don't _have_ to import them from files--lo_import is just a convenience
function for reading a file into memory and writing it to a large object.
Just use the other large object functions: lo_open(), lo_creat(),
lo_read(), lo_write(), lo_seek().
Jeroen
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Handling Blobs with libpq
2003-03-20 09:56 Handling Blobs with libpq Stéphane Pinel <spinel@noos.fr>
@ 2003-03-20 21:44 ` Nigel J. Andrews <nandrews@investsystems.co.uk>
2003-03-20 21:49 ` Re: Handling Blobs with libpq Joe Conway <mail@joeconway.com>
2 siblings, 1 reply; 6+ messages in thread
From: Nigel J. Andrews @ 2003-03-20 21:44 UTC (permalink / raw)
To: Stéphane Pinel <spinel@noos.fr>; +Cc: pgsql-interfaces
On Thu, 20 Mar 2003, Stéphane Pinel wrote:
> What is the best approach to handle blobs with libpq ?
>
> I've read in documentation that there are 2 ways:
>
> 1- Bytea
> 2- Larges objects (lo_import)
>
> My main need is storing pictures (like Jpegs etc) or small blobs. Doc
> says that (2) is more
> appropriate but I need to work with blobs in memory, not from files.
>
FWIW, from some tests I ran with PHP bytea is slower and can be several times
slower. Depending on the data the escaping can increase the data size for
transmission up to 4 times. (Can that be right? I'm sure I saw a factor of 4
but can't find the graph now.) That is obviously going to slow the query
transmission/result reception and also cause the backend some work. Indeed, my
tests showed simple retrieving of data files stored as bytea to be
slower than doing the same thing with lo_s with the degree of slow down
proportional to the increase in data size caused by the escaping.
I'm not sure how much of this is down to the escaping/unescaping operations
specifically but that is irrelevent because they're part of the operation to
get what's wanted and what lo_s deliver naturally.
Also, someone recently pointed out that even with bytea if the client encoding
is different when selecting than it was when storing there is still a chance
that your data will be come out different to how it should.
--
Nigel J. Andrews
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Handling Blobs with libpq
2003-03-20 09:56 Handling Blobs with libpq Stéphane Pinel <spinel@noos.fr>
2003-03-20 21:44 ` Re: Handling Blobs with libpq Nigel J. Andrews <nandrews@investsystems.co.uk>
@ 2003-03-20 21:49 ` Joe Conway <mail@joeconway.com>
0 siblings, 0 replies; 6+ messages in thread
From: Joe Conway @ 2003-03-20 21:49 UTC (permalink / raw)
To: Nigel J. Andrews <nandrews@investsystems.co.uk>; +Cc: Stéphane Pinel <spinel@noos.fr>; pgsql-interfaces
Nigel J. Andrews wrote:
> Also, someone recently pointed out that even with bytea if the client encoding
> is different when selecting than it was when storing there is still a chance
> that your data will be come out different to how it should.
>
If you are using libpq in your own C program, one way around these
issues (for retrieval anyway) is to use a binary cursor.
Joe
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Handling Blobs with libpq
2003-03-20 09:56 Handling Blobs with libpq Stéphane Pinel <spinel@noos.fr>
@ 2003-03-21 22:42 ` Daniel Bruce Lynes <dlynes@shaw.ca>
2 siblings, 0 replies; 6+ messages in thread
From: Daniel Bruce Lynes @ 2003-03-21 22:42 UTC (permalink / raw)
To: pgsql-interfaces
On Thursday 20 March 2003 01:56, Stéphane Pinel wrote:
> My main need is storing pictures (like Jpegs etc) or small blobs. Doc
> says that (2) is more
> appropriate but I need to work with blobs in memory, not from files.
If you need to work with it in memory, you're better off using the bytea (same
thing as a varbinary). LOBs are generally for when the data is so large that
the performance of your application would be comprimised if you allocated
memory for the contents. LOBs allow you to do streaming so that you don't
need to allocate memory. Only the lo_export/lo_import functions require a
file. All of the other LOB functions only require a stream that's chunked
into byte arrays.
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2003-03-21 22:42 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-03-20 09:48 Handling Blobs with libpq Stéphane Pinel <spinel@exenevex.com>
2003-03-20 09:56 Handling Blobs with libpq Stéphane Pinel <spinel@noos.fr>
2003-03-20 15:11 ` Jeroen T. Vermeulen <jtv@xs4all.nl>
2003-03-20 21:44 ` Nigel J. Andrews <nandrews@investsystems.co.uk>
2003-03-20 21:49 ` Joe Conway <mail@joeconway.com>
2003-03-21 22:42 ` Daniel Bruce Lynes <dlynes@shaw.ca>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox