pg.ddx.io pgsql-interfaces@postgresql.org mailing list archive
help / color / mirror / Atom feedproblems with large objects
13+ messages / 7 participants
[nested] [flat]
* problems with large objects
@ 1999-08-27 17:21 Wolfgang Hottgenroth <woho@ieee.org>
0 siblings, 0 replies; 13+ messages in thread
From: Wolfgang Hottgenroth @ 1999-08-27 17:21 UTC (permalink / raw)
To: pgsql-interfaces@hub.org
Hi.
I use the postgreSQL-database-system and esspecially I do some work with
the large-objects, which I use through the
Perl-library. Unfortunately since I switched from 6.4.2 to 6.5.1 my
tools
refused to work. lo_import didn't work anymore.
I traced it down through the libpq, where I found that a test with
lo_lseek in the lo_open-function (file: fe-lobj.c) fails. Going deeper
into the source and drawing a trace of debug-output across some files I
switched tracing from the libpq to the backend.
Here I found in be-fsstubs.c, that after the function lo_open sets a
cookie using the function newLOfd, just this cookie will be cleared by
lo_commit, which is called by CommitTransaction (in xact.c). The handle,
handed out to the application by the library-function lo_open is now not
longer valid and the next access through this handle fails.
I attach the test-application (in Perl, named db_test), the debug-output
of the application (and the outputs in the libpq) (db_test2.log), the
debug-output of postmaster
(post2.log) and the files I patched with my debug-output (you will find
them easily because in every line I changed there is the string 'WN').
Did anyone of you ever see this problem? Is there something wrong with
my
application (which works fine with 6.4.2)?
I hope someone of you find time to have a look on the files and also I
hope he or she can
help my. I want to use the postgreSQL on an Alpha, and 6.4.2 did not run
on the Alpha (o yes, when I remove all optimisation it does). (But don't
be confused: all the test I did runs on an
Intel-Machine.)
Thanks,
Wolfgang Hottgenroth
Attachments:
be-fsstubs.c (text/plain, 12.4K) download
/*-------------------------------------------------------------------------
*
* be-fsstubs.c
* support for filesystem operations on large objects
*
* Copyright (c) 1994, Regents of the University of California
*
*
* IDENTIFICATION
* $Header: /usr/local/cvsroot/pgsql/src/backend/libpq/be-fsstubs.c,v 1.35 1999/06/04 21:13:38 tgl Exp $
*
* NOTES
* This should be moved to a more appropriate place. It is here
* for lack of a better place.
*
* Builtin functions for open/close/read/write operations on large objects.
*
* These functions operate in a private GlobalMemoryContext, which means
* that large object descriptors hang around until we destroy the context.
* That happens in lo_commit(). It'd be possible to prolong the lifetime
* of the context so that LO FDs are good across transactions (for example,
* we could release the context only if we see that no FDs remain open).
* But we'd need additional state in order to do the right thing at the
* end of an aborted transaction. FDs opened during an aborted xact would
* still need to be closed, since they might not be pointing at valid
* relations at all. For now, we'll stick with the existing documented
* semantics of LO FDs: they're only good within a transaction.
*
*-------------------------------------------------------------------------
*/
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <string.h>
#include <postgres.h>
#include <lib/dllist.h>
#include <libpq/libpq.h>
#include <libpq/libpq-fs.h>
#include <nodes/nodes.h>
#include <utils/memutils.h>
#include <lib/fstack.h>
#include <utils/mcxt.h>
#include <catalog/pg_shadow.h> /* for superuser() */
#include <storage/fd.h> /* for O_ */
#include <storage/large_object.h>
#include <libpq/be-fsstubs.h>
/* [PA] is Pascal André <andre@via.ecp.fr> */
/*#define FSDB 1*/
#define MAX_LOBJ_FDS 256
#define BUFSIZE 1024
#define FNAME_BUFSIZE 8192
static LargeObjectDesc *cookies[MAX_LOBJ_FDS];
static GlobalMemory fscxt = NULL;
static int newLOfd(LargeObjectDesc *lobjCookie);
static void deleteLOfd(int fd);
/*****************************************************************************
* File Interfaces for Large Objects
*****************************************************************************/
int
lo_open(Oid lobjId, int mode)
{
LargeObjectDesc *lobjDesc;
int fd;
MemoryContext currentContext;
#if 1 //FSDB
elog(NOTICE, "LOopen(%u,%d)", lobjId, mode);
#endif
if (fscxt == NULL)
fscxt = CreateGlobalMemory("Filesystem");
currentContext = MemoryContextSwitchTo((MemoryContext) fscxt);
lobjDesc = inv_open(lobjId, mode);
elog(NOTICE, "WN: be-fsstubs: lo_open: lobjDesc: %x", lobjDesc);
if (lobjDesc == NULL)
{ /* lookup failed */
MemoryContextSwitchTo(currentContext);
#if 1 //FSDB
elog(NOTICE, "cannot open large object %u", lobjId);
#endif
return -1;
}
fd = newLOfd(lobjDesc);
elog(NOTICE, "WN: be-fsstubs: lo_open: fd: %x", fd);
/* switch context back to orig. */
MemoryContextSwitchTo(currentContext);
#if 1 //FSDB
if (fd < 0) /* newLOfd couldn't find a slot */
elog(NOTICE, "Out of space for large object FDs");
#endif
elog(NOTICE, "WN: be-fsstubs: lo_open: return: %x", fd);
return fd;
}
int
lo_close(int fd)
{
MemoryContext currentContext;
if (fd < 0 || fd >= MAX_LOBJ_FDS)
{
elog(ERROR, "lo_close: large obj descriptor (%d) out of range", fd);
return -2;
}
if (cookies[fd] == NULL)
{
elog(ERROR, "lo_close: invalid large obj descriptor (%d)", fd);
return -3;
}
#if 1 //FSDB
elog(NOTICE, "LOclose(%d)", fd);
#endif
Assert(fscxt != NULL);
currentContext = MemoryContextSwitchTo((MemoryContext) fscxt);
inv_close(cookies[fd]);
MemoryContextSwitchTo(currentContext);
deleteLOfd(fd);
return 0;
}
/*
* We assume the large object supports byte oriented reads and seeks so
* that our work is easier.
*/
int
lo_read(int fd, char *buf, int len)
{
MemoryContext currentContext;
int status;
if (fd < 0 || fd >= MAX_LOBJ_FDS)
{
elog(ERROR, "lo_read: large obj descriptor (%d) out of range", fd);
return -2;
}
if (cookies[fd] == NULL)
{
elog(ERROR, "lo_read: invalid large obj descriptor (%d)", fd);
return -3;
}
Assert(fscxt != NULL);
currentContext = MemoryContextSwitchTo((MemoryContext) fscxt);
status = inv_read(cookies[fd], buf, len);
MemoryContextSwitchTo(currentContext);
return (status);
}
int
lo_write(int fd, char *buf, int len)
{
MemoryContext currentContext;
int status;
if (fd < 0 || fd >= MAX_LOBJ_FDS)
{
elog(ERROR, "lo_write: large obj descriptor (%d) out of range", fd);
return -2;
}
if (cookies[fd] == NULL)
{
elog(ERROR, "lo_write: invalid large obj descriptor (%d)", fd);
return -3;
}
Assert(fscxt != NULL);
currentContext = MemoryContextSwitchTo((MemoryContext) fscxt);
status = inv_write(cookies[fd], buf, len);
MemoryContextSwitchTo(currentContext);
return (status);
}
int
lo_lseek(int fd, int offset, int whence)
{
MemoryContext currentContext;
int status;
elog(NOTICE, "WN: be-fsstubs: lo_lseek, fd: %d", fd);
if (fd < 0 || fd >= MAX_LOBJ_FDS)
{
elog(ERROR, "lo_lseek: large obj descriptor (%d) out of range", fd);
return -2;
}
if (cookies[fd] == NULL)
{
elog(NOTICE, "WN: be-fsstubs: lo_lseek: cookies[%u]=%x", fd, cookies[fd]);
elog(ERROR, "lo_lseek: invalid large obj descriptor (%d)", fd);
return -3;
}
Assert(fscxt != NULL);
currentContext = MemoryContextSwitchTo((MemoryContext) fscxt);
status = inv_seek(cookies[fd], offset, whence);
MemoryContextSwitchTo(currentContext);
return status;
}
Oid
lo_creat(int mode)
{
LargeObjectDesc *lobjDesc;
MemoryContext currentContext;
Oid lobjId;
if (fscxt == NULL)
fscxt = CreateGlobalMemory("Filesystem");
currentContext = MemoryContextSwitchTo((MemoryContext) fscxt);
lobjDesc = inv_create(mode);
if (lobjDesc == NULL)
{
MemoryContextSwitchTo(currentContext);
return InvalidOid;
}
lobjId = RelationGetRelid(lobjDesc->heap_r);
inv_close(lobjDesc);
/* switch context back to original memory context */
MemoryContextSwitchTo(currentContext);
return lobjId;
}
int
lo_tell(int fd)
{
if (fd < 0 || fd >= MAX_LOBJ_FDS)
{
elog(ERROR, "lo_tell: large object descriptor (%d) out of range", fd);
return -2;
}
if (cookies[fd] == NULL)
{
elog(ERROR, "lo_tell: invalid large object descriptor (%d)", fd);
return -3;
}
/*
* We assume we do not need to switch contexts for inv_tell.
* That is true for now, but is probably more than this module
* ought to assume...
*/
return inv_tell(cookies[fd]);
}
int
lo_unlink(Oid lobjId)
{
/*
* inv_destroy does not need a context switch, indeed it doesn't
* touch any LO-specific data structures at all. (Again, that's
* probably more than this module ought to be assuming.)
*
* XXX there ought to be some code to clean up any open LOs that
* reference the specified relation... as is, they remain "open".
*/
return inv_destroy(lobjId);
}
/*****************************************************************************
* Read/Write using varlena
*****************************************************************************/
struct varlena *
loread(int fd, int len)
{
struct varlena *retval;
int totalread = 0;
retval = (struct varlena *) palloc(VARHDRSZ + len);
totalread = lo_read(fd, VARDATA(retval), len);
VARSIZE(retval) = totalread + VARHDRSZ;
return retval;
}
int
lowrite(int fd, struct varlena * wbuf)
{
int totalwritten;
int bytestowrite;
bytestowrite = VARSIZE(wbuf) - VARHDRSZ;
totalwritten = lo_write(fd, VARDATA(wbuf), bytestowrite);
return totalwritten;
}
/*****************************************************************************
* Import/Export of Large Object
*****************************************************************************/
/*
* lo_import -
* imports a file as an (inversion) large object.
*/
Oid
lo_import(text *filename)
{
File fd;
int nbytes,
tmp;
char buf[BUFSIZE];
char fnamebuf[FNAME_BUFSIZE];
LargeObjectDesc *lobj;
Oid lobjOid;
#ifndef ALLOW_DANGEROUS_LO_FUNCTIONS
if (!superuser())
elog(ERROR, "You must have Postgres superuser privilege to use "
"server-side lo_import().\n\tAnyone can use the "
"client-side lo_import() provided by libpq.");
#endif
/*
* open the file to be read in
*/
nbytes = VARSIZE(filename) - VARHDRSZ + 1;
if (nbytes > FNAME_BUFSIZE)
nbytes = FNAME_BUFSIZE;
StrNCpy(fnamebuf, VARDATA(filename), nbytes);
#ifndef __CYGWIN32__
fd = PathNameOpenFile(fnamebuf, O_RDONLY, 0666);
#else
fd = PathNameOpenFile(fnamebuf, O_RDONLY | O_BINARY, 0666);
#endif
if (fd < 0)
{ /* error */
elog(ERROR, "lo_import: can't open unix file \"%s\": %m",
fnamebuf);
}
/*
* create an inversion "object"
*/
lobj = inv_create(INV_READ | INV_WRITE);
if (lobj == NULL)
{
elog(ERROR, "lo_import: can't create inv object for \"%s\"",
fnamebuf);
}
/*
* the oid for the large object is just the oid of the relation
* XInv??? which contains the data.
*/
lobjOid = RelationGetRelid(lobj->heap_r);
/*
* read in from the Unix file and write to the inversion file
*/
while ((nbytes = FileRead(fd, buf, BUFSIZE)) > 0)
{
tmp = inv_write(lobj, buf, nbytes);
if (tmp < nbytes)
elog(ERROR, "lo_import: error while reading \"%s\"",
fnamebuf);
}
FileClose(fd);
inv_close(lobj);
return lobjOid;
}
/*
* lo_export -
* exports an (inversion) large object.
*/
int4
lo_export(Oid lobjId, text *filename)
{
File fd;
int nbytes,
tmp;
char buf[BUFSIZE];
char fnamebuf[FNAME_BUFSIZE];
LargeObjectDesc *lobj;
mode_t oumask;
#ifndef ALLOW_DANGEROUS_LO_FUNCTIONS
if (!superuser())
elog(ERROR, "You must have Postgres superuser privilege to use "
"server-side lo_export().\n\tAnyone can use the "
"client-side lo_export() provided by libpq.");
#endif
/*
* open the inversion "object"
*/
lobj = inv_open(lobjId, INV_READ);
if (lobj == NULL)
elog(ERROR, "lo_export: can't open inv object %u", lobjId);
/*
* open the file to be written to
*
* Note: we reduce backend's normal 077 umask to the slightly
* friendlier 022. This code used to drop it all the way to 0,
* but creating world-writable export files doesn't seem wise.
*/
nbytes = VARSIZE(filename) - VARHDRSZ + 1;
if (nbytes > FNAME_BUFSIZE)
nbytes = FNAME_BUFSIZE;
StrNCpy(fnamebuf, VARDATA(filename), nbytes);
oumask = umask((mode_t) 0022);
#ifndef __CYGWIN32__
fd = PathNameOpenFile(fnamebuf, O_CREAT | O_WRONLY | O_TRUNC, 0666);
#else
fd = PathNameOpenFile(fnamebuf, O_CREAT | O_WRONLY | O_TRUNC | O_BINARY, 0666);
#endif
umask(oumask);
if (fd < 0)
{ /* error */
elog(ERROR, "lo_export: can't open unix file \"%s\": %m",
fnamebuf);
}
/*
* read in from the Unix file and write to the inversion file
*/
while ((nbytes = inv_read(lobj, buf, BUFSIZE)) > 0)
{
tmp = FileWrite(fd, buf, nbytes);
if (tmp < nbytes)
elog(ERROR, "lo_export: error while writing \"%s\"",
fnamebuf);
}
inv_close(lobj);
FileClose(fd);
return 1;
}
/*
* lo_commit -
* prepares large objects for transaction commit [PA, 7/17/98]
*/
void
lo_commit(bool isCommit)
{
int i;
MemoryContext currentContext;
elog(NOTICE, "WN: be-fsstubs: lo_commit, isCommit: %u", isCommit);
if (fscxt == NULL)
return; /* no LO operations in this xact */
currentContext = MemoryContextSwitchTo((MemoryContext) fscxt);
/* Clean out still-open index scans (not necessary if aborting)
* and clear cookies array so that LO fds are no longer good.
*/
for (i = 0; i < MAX_LOBJ_FDS; i++)
{
if (cookies[i] != NULL)
{
if (isCommit)
inv_cleanindex(cookies[i]);
elog(NOTICE, "WN: be-fsstubs: cookies[%u]=NULL!", i);
cookies[i] = NULL;
}
}
MemoryContextSwitchTo(currentContext);
/* Release the LO memory context to prevent permanent memory leaks. */
GlobalMemoryDestroy(fscxt);
fscxt = NULL;
}
/*****************************************************************************
* Support routines for this file
*****************************************************************************/
static int
newLOfd(LargeObjectDesc *lobjCookie)
{
int i;
elog(NOTICE, "WN: be-fsstubs: newLOfd: MAX_LOBJ_FDS: %d, lobjCookie: %x", MAX_LOBJ_FDS, lobjCookie);
for (i = 0; i < MAX_LOBJ_FDS; i++)
{
elog(NOTICE, "WN: be-fsstubs: newLOfd: i: %d, cookies: %x", i, cookies[i]);
if (cookies[i] == NULL)
{
elog(NOTICE, "WN: be-fsstubs: newLOfd: cookie found!");
cookies[i] = lobjCookie;
elog(NOTICE, "WN: be-fsstubs: newLOfd: cookies[%u]=%x",i,cookies[i]);
return i;
}
}
return -1;
}
static void
deleteLOfd(int fd)
{
cookies[fd] = NULL;
}
db_test (text/plain, 521B) download
#!/usr/bin/perl -w
use Pg;
my $conn;
#DB_Open(\$conn, "dbname=adrema");
#$conn = Pg::connectdb("dbname=adrema");
$conn = PQsetdb('','','','','adrema');
my $filename = 'test.dat';
my $oid;
#$oid = $conn->lo_import($filename);
#$oid = PQlo_import($conn, $filename);
$oid = PQlo_creat($conn, PGRES_INV_WRITE);
print "db_test: lo_creat: oid: $oid\n";
#print $conn->errorMessage;
my $fd;
$fd = PQlo_open($conn, $oid, PGRES_INV_WRITE);
print "db_test: lo_open: fd: $fd\n";
#print $conn->errorMessage;
PQfinish($conn);
db_test2.log (text/plain, 3.5K) download
NOTICE: WN: xact: StartTransaction
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 1
NOTICE: WN: xact: StartTransaction
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 1
NOTICE: WN: xact: StartTransaction
NOTICE: LOopen(19425,131072)
NOTICE: WN: be-fsstubs: lo_open: lobjDesc: 821fcd0
NOTICE: WN: be-fsstubs: newLOfd: MAX_LOBJ_FDS: 256, lobjCookie: 821fcd0
NOTICE: WN: be-fsstubs: newLOfd: i: 0, cookies: 0
NOTICE: WN: be-fsstubs: newLOfd: cookie found!
NOTICE: WN: be-fsstubs: newLOfd: cookies[0]=821fcd0
NOTICE: WN: be-fsstubs: lo_open: fd: 0
NOTICE: WN: be-fsstubs: lo_open: return: 0
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 1
NOTICE: WN: be-fsstubs: cookies[0]=NULL!
NOTICE: WN: xact: StartTransaction
NOTICE: WN: be-fsstubs: lo_lseek, fd: 0
NOTICE: WN: be-fsstubs: lo_lseek: cookies[0]=0
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 0
fe-exec, PQfn, A
fe-exec, PQfn, C: i:0
fe-exec, PQfn, E
fe-exec, PQfn, I
fe-exec, PQfn, J1:
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: V
fe-exec, PQfn, 0
fe-exec, PQfn, 1
fe-exec, PQfn, 2
fe-exec, PQfn, 3: 4
fe-exec, PQfn, 3b: result_buf: 19425
fe-exec, PQfn, 5
fe-exec, PQfn, I
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: Z
db_test: lo_creat: oid: 19425
fe-lobj, lo_open: 0
fe-lobj, lo_open: 1a
fe-exec, PQfn, A
fe-exec, PQfn, C: i:0
fe-exec, PQfn, E
fe-exec, PQfn, C: i:1
fe-exec, PQfn, E
fe-exec, PQfn, I
fe-exec, PQfn, J1: @
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: V
fe-exec, PQfn, 0
fe-exec, PQfn, 1
fe-exec, PQfn, 2
fe-exec, PQfn, 3: 4
fe-exec, PQfn, 3b: result_buf: 0
fe-exec, PQfn, 5
fe-exec, PQfn, I
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: Z
fe-lobj, lo_open: 2 (3b8,0)
fe-lobj, lo_open: 3
fe-exec, PQfn, A
fe-exec, PQfn, C: i:0
fe-exec, PQfn, E
fe-exec, PQfn, C: i:1
fe-exec, PQfn, E
fe-exec, PQfn, C: i:2
fe-exec, PQfn, E
fe-exec, PQfn, I
fe-exec, PQfn, J1: @
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: E
fe-exec, PQfn, I
fe-exec, PQfn, J1: E
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: N
fe-exec, PQfn, I
fe-exec, PQfn, J1: N
fe-exec, PQfn, I
fe-exec, PQfn, J
fe-exec, PQfn, ID: Z
fe-lobj, lo_open: 3a (0,ffffffff)
db_test: lo_open: fd: -1
fe-exec.c (text/plain, 49.3K) download
/*-------------------------------------------------------------------------
*
* fe-exec.c
* functions related to sending a query down to the backend
*
* Copyright (c) 1994, Regents of the University of California
*
*
* IDENTIFICATION
* $Header: /usr/local/cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v 1.81 1999/05/28 01:54:53 tgl Exp $
*
*-------------------------------------------------------------------------
*/
#include "libpq-fe.h"
#include "libpq-int.h"
#include "postgres.h"
#ifdef WIN32
#include "win32.h"
#else
#if !defined(NO_UNISTD_H)
#include <unistd.h>
#endif
#endif
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <ctype.h>
/* keep this in same order as ExecStatusType in libpq-fe.h */
const char *const pgresStatus[] = {
"PGRES_EMPTY_QUERY",
"PGRES_COMMAND_OK",
"PGRES_TUPLES_OK",
"PGRES_COPY_OUT",
"PGRES_COPY_IN",
"PGRES_BAD_RESPONSE",
"PGRES_NONFATAL_ERROR",
"PGRES_FATAL_ERROR"
};
#define DONOTICE(conn,message) \
((*(conn)->noticeHook) ((conn)->noticeArg, (message)))
static int addTuple(PGresult *res, PGresAttValue *tup);
static void parseInput(PGconn *conn);
static void handleSendFailure(PGconn *conn);
static int getRowDescriptions(PGconn *conn);
static int getAnotherTuple(PGconn *conn, int binary);
static int getNotify(PGconn *conn);
static int getNotice(PGconn *conn);
/* ----------------
* Space management for PGresult.
*
* Formerly, libpq did a separate malloc() for each field of each tuple
* returned by a query. This was remarkably expensive --- malloc/free
* consumed a sizable part of the application's runtime. And there is
* no real need to keep track of the fields separately, since they will
* all be freed together when the PGresult is released. So now, we grab
* large blocks of storage from malloc and allocate space for query data
* within these blocks, using a trivially simple allocator. This reduces
* the number of malloc/free calls dramatically, and it also avoids
* fragmentation of the malloc storage arena.
* The PGresult structure itself is still malloc'd separately. We could
* combine it with the first allocation block, but that would waste space
* for the common case that no extra storage is actually needed (that is,
* the SQL command did not return tuples).
* We also malloc the top-level array of tuple pointers separately, because
* we need to be able to enlarge it via realloc, and our trivial space
* allocator doesn't handle that effectively. (Too bad the FE/BE protocol
* doesn't tell us up front how many tuples will be returned.)
* All other subsidiary storage for a PGresult is kept in PGresult_data blocks
* of size PGRESULT_DATA_BLOCKSIZE. The overhead at the start of each block
* is just a link to the next one, if any. Free-space management info is
* kept in the owning PGresult.
* A query returning a small amount of data will thus require three malloc
* calls: one for the PGresult, one for the tuples pointer array, and one
* PGresult_data block.
* Only the most recently allocated PGresult_data block is a candidate to
* have more stuff added to it --- any extra space left over in older blocks
* is wasted. We could be smarter and search the whole chain, but the point
* here is to be simple and fast. Typical applications do not keep a PGresult
* around very long anyway, so some wasted space within one is not a problem.
*
* Tuning constants for the space allocator are:
* PGRESULT_DATA_BLOCKSIZE: size of a standard allocation block, in bytes
* PGRESULT_ALIGN_BOUNDARY: assumed alignment requirement for binary data
* PGRESULT_SEP_ALLOC_THRESHOLD: objects bigger than this are given separate
* blocks, instead of being crammed into a regular allocation block.
* Requirements for correct function are:
* PGRESULT_ALIGN_BOUNDARY must be a multiple of the alignment requirements
* of all machine data types. (Currently this is set from configure
* tests, so it should be OK automatically.)
* PGRESULT_SEP_ALLOC_THRESHOLD + PGRESULT_BLOCK_OVERHEAD <=
* PGRESULT_DATA_BLOCKSIZE
* pqResultAlloc assumes an object smaller than the threshold will fit
* in a new block.
* The amount of space wasted at the end of a block could be as much as
* PGRESULT_SEP_ALLOC_THRESHOLD, so it doesn't pay to make that too large.
* ----------------
*/
#ifdef MAX
#undef MAX
#endif
#define MAX(a,b) ((a) > (b) ? (a) : (b))
#define PGRESULT_DATA_BLOCKSIZE 2048
#define PGRESULT_ALIGN_BOUNDARY MAXIMUM_ALIGNOF /* from configure */
#define PGRESULT_BLOCK_OVERHEAD MAX(sizeof(PGresult_data), PGRESULT_ALIGN_BOUNDARY)
#define PGRESULT_SEP_ALLOC_THRESHOLD (PGRESULT_DATA_BLOCKSIZE / 2)
/*
* PQmakeEmptyPGresult
* returns a newly allocated, initialized PGresult with given status.
* If conn is not NULL and status indicates an error, the conn's
* errorMessage is copied.
*
* Note this is exported --- you wouldn't think an application would need
* to build its own PGresults, but this has proven useful in both libpgtcl
* and the Perl5 interface, so maybe it's not so unreasonable.
*/
PGresult *
PQmakeEmptyPGresult(PGconn *conn, ExecStatusType status)
{
PGresult *result;
result = (PGresult *) malloc(sizeof(PGresult));
result->conn = conn; /* might be NULL */
result->ntups = 0;
result->numAttributes = 0;
result->attDescs = NULL;
result->tuples = NULL;
result->tupArrSize = 0;
result->resultStatus = status;
result->cmdStatus[0] = '\0';
result->binary = 0;
result->errMsg = NULL;
result->null_field[0] = '\0';
result->curBlock = NULL;
result->curOffset = 0;
result->spaceLeft = 0;
if (conn) /* consider copying conn's errorMessage */
{
switch (status)
{
case PGRES_EMPTY_QUERY:
case PGRES_COMMAND_OK:
case PGRES_TUPLES_OK:
case PGRES_COPY_OUT:
case PGRES_COPY_IN:
/* non-error cases */
break;
default:
pqSetResultError(result, conn->errorMessage);
break;
}
}
return result;
}
/*
* pqResultAlloc -
* Allocate subsidiary storage for a PGresult.
*
* nBytes is the amount of space needed for the object.
* If isBinary is true, we assume that we need to align the object on
* a machine allocation boundary.
* If isBinary is false, we assume the object is a char string and can
* be allocated on any byte boundary.
*/
void *
pqResultAlloc(PGresult *res, int nBytes, int isBinary)
{
char *space;
PGresult_data *block;
if (!res)
return NULL;
if (nBytes <= 0)
return res->null_field;
/*
* If alignment is needed, round up the current position to an
* alignment boundary.
*/
if (isBinary)
{
int offset = res->curOffset % PGRESULT_ALIGN_BOUNDARY;
if (offset)
{
res->curOffset += PGRESULT_ALIGN_BOUNDARY - offset;
res->spaceLeft -= PGRESULT_ALIGN_BOUNDARY - offset;
}
}
/* If there's enough space in the current block, no problem. */
if (nBytes <= res->spaceLeft)
{
space = res->curBlock->space + res->curOffset;
res->curOffset += nBytes;
res->spaceLeft -= nBytes;
return space;
}
/*
* If the requested object is very large, give it its own block; this
* avoids wasting what might be most of the current block to start a
* new block. (We'd have to special-case requests bigger than the
* block size anyway.) The object is always given binary alignment in
* this case.
*/
if (nBytes >= PGRESULT_SEP_ALLOC_THRESHOLD)
{
block = (PGresult_data *) malloc(nBytes + PGRESULT_BLOCK_OVERHEAD);
if (!block)
return NULL;
space = block->space + PGRESULT_BLOCK_OVERHEAD;
if (res->curBlock)
{
/*
* Tuck special block below the active block, so that we don't
* have to waste the free space in the active block.
*/
block->next = res->curBlock->next;
res->curBlock->next = block;
}
else
{
/* Must set up the new block as the first active block. */
block->next = NULL;
res->curBlock = block;
res->spaceLeft = 0; /* be sure it's marked full */
}
return space;
}
/* Otherwise, start a new block. */
block = (PGresult_data *) malloc(PGRESULT_DATA_BLOCKSIZE);
if (!block)
return NULL;
block->next = res->curBlock;
res->curBlock = block;
if (isBinary)
{
/* object needs full alignment */
res->curOffset = PGRESULT_BLOCK_OVERHEAD;
res->spaceLeft = PGRESULT_DATA_BLOCKSIZE - PGRESULT_BLOCK_OVERHEAD;
}
else
{
/* we can cram it right after the overhead pointer */
res->curOffset = sizeof(PGresult_data);
res->spaceLeft = PGRESULT_DATA_BLOCKSIZE - sizeof(PGresult_data);
}
space = block->space + res->curOffset;
res->curOffset += nBytes;
res->spaceLeft -= nBytes;
return space;
}
/*
* pqResultStrdup -
* Like strdup, but the space is subsidiary PGresult space.
*/
char *
pqResultStrdup(PGresult *res, const char *str)
{
char *space = (char *) pqResultAlloc(res, strlen(str) + 1, FALSE);
if (space)
strcpy(space, str);
return space;
}
/*
* pqSetResultError -
* assign a new error message to a PGresult
*/
void
pqSetResultError(PGresult *res, const char *msg)
{
if (!res)
return;
if (msg && *msg)
res->errMsg = pqResultStrdup(res, msg);
else
res->errMsg = NULL;
}
/*
* PQclear -
* free's the memory associated with a PGresult
*/
void
PQclear(PGresult *res)
{
PGresult_data *block;
if (!res)
return;
/* Free all the subsidiary blocks */
while ((block = res->curBlock) != NULL)
{
res->curBlock = block->next;
free(block);
}
/* Free the top-level tuple pointer array */
if (res->tuples)
free(res->tuples);
/* Free the PGresult structure itself */
free(res);
}
/*
* Handy subroutine to deallocate any partially constructed async result.
*/
void
pqClearAsyncResult(PGconn *conn)
{
if (conn->result)
PQclear(conn->result);
conn->result = NULL;
conn->curTuple = NULL;
}
/*
* addTuple
* add a row pointer to the PGresult structure, growing it if necessary
* Returns TRUE if OK, FALSE if not enough memory to add the row
*/
static int
addTuple(PGresult *res, PGresAttValue *tup)
{
if (res->ntups >= res->tupArrSize)
{
/*
* Try to grow the array.
*
* We can use realloc because shallow copying of the structure is
* okay. Note that the first time through, res->tuples is NULL.
* While ANSI says that realloc() should act like malloc() in that
* case, some old C libraries (like SunOS 4.1.x) coredump instead.
* On failure realloc is supposed to return NULL without damaging
* the existing allocation. Note that the positions beyond
* res->ntups are garbage, not necessarily NULL.
*/
int newSize = (res->tupArrSize > 0) ? res->tupArrSize * 2 : 128;
PGresAttValue **newTuples;
if (res->tuples == NULL)
newTuples = (PGresAttValue **)
malloc(newSize * sizeof(PGresAttValue *));
else
newTuples = (PGresAttValue **)
realloc(res->tuples, newSize * sizeof(PGresAttValue *));
if (!newTuples)
return FALSE; /* malloc or realloc failed */
res->tupArrSize = newSize;
res->tuples = newTuples;
}
res->tuples[res->ntups] = tup;
res->ntups++;
return TRUE;
}
/*
* PQsendQuery
* Submit a query, but don't wait for it to finish
*
* Returns: 1 if successfully submitted
* 0 if error (conn->errorMessage is set)
*/
int
PQsendQuery(PGconn *conn, const char *query)
{
if (!conn)
return 0;
if (!query)
{
sprintf(conn->errorMessage, "PQsendQuery() -- query pointer is null.");
return 0;
}
/* check to see if the query string is too long */
if (strlen(query) > MAX_MESSAGE_LEN - 2)
{
sprintf(conn->errorMessage, "PQsendQuery() -- query is too long. "
"Maximum length is %d\n", MAX_MESSAGE_LEN - 2);
return 0;
}
/* Don't try to send if we know there's no live connection. */
if (conn->status != CONNECTION_OK)
{
sprintf(conn->errorMessage, "PQsendQuery() -- There is no connection "
"to the backend.\n");
return 0;
}
/* Can't send while already busy, either. */
if (conn->asyncStatus != PGASYNC_IDLE)
{
sprintf(conn->errorMessage,
"PQsendQuery() -- another query already in progress.");
return 0;
}
/* clear the error string */
conn->errorMessage[0] = '\0';
/* initialize async result-accumulation state */
conn->result = NULL;
conn->curTuple = NULL;
/* send the query to the backend; */
/* the frontend-backend protocol uses 'Q' to designate queries */
if (pqPutnchar("Q", 1, conn) ||
pqPuts(query, conn) ||
pqFlush(conn))
{
handleSendFailure(conn);
return 0;
}
/* OK, it's launched! */
conn->asyncStatus = PGASYNC_BUSY;
return 1;
}
/*
* handleSendFailure: try to clean up after failure to send command.
*
* Primarily, what we want to accomplish here is to process an async
* NOTICE message that the backend might have sent just before it died.
*
* NOTE: this routine should only be called in PGASYNC_IDLE state.
*/
static void
handleSendFailure(PGconn *conn)
{
/* Preserve the error message emitted by the failing output routine */
char * svErrMsg = strdup(conn->errorMessage);
/*
* Accept any available input data, ignoring errors. Note that if
* pqReadData decides the backend has closed the channel, it will
* close our side of the socket --- that's just what we want here.
*/
while (pqReadData(conn) > 0)
/* loop until no more data readable */ ;
/*
* Parse any available input messages. Since we are in PGASYNC_IDLE
* state, only NOTICE and NOTIFY messages will be eaten.
*/
parseInput(conn);
/* Restore error message generated by output routine, if any. */
if (*svErrMsg != '\0')
strcpy(conn->errorMessage, svErrMsg);
free(svErrMsg);
}
/*
* Consume any available input from the backend
* 0 return: some kind of trouble
* 1 return: no problem
*/
int
PQconsumeInput(PGconn *conn)
{
if (!conn)
return 0;
/*
* Load more data, if available. We do this no matter what state we
* are in, since we are probably getting called because the
* application wants to get rid of a read-select condition. Note that
* we will NOT block waiting for more input.
*/
if (pqReadData(conn) < 0)
return 0;
/* Parsing of the data waits till later. */
return 1;
}
/*
* parseInput: if appropriate, parse input data from backend
* until input is exhausted or a stopping state is reached.
* Note that this function will NOT attempt to read more data from the backend.
*/
static void
parseInput(PGconn *conn)
{
char id;
/*
* Loop to parse successive complete messages available in the buffer.
*/
for (;;)
{
/*
* Quit if in COPY_OUT state: we expect raw data from the server
* until PQendcopy is called. Don't try to parse it according to
* the normal protocol. (This is bogus. The data lines ought to
* be part of the protocol and have identifying leading
* characters.)
*/
if (conn->asyncStatus == PGASYNC_COPY_OUT)
return;
/*
* OK to try to read a message type code.
*/
conn->inCursor = conn->inStart;
if (pqGetc(&id, conn))
return;
/*
* NOTIFY and NOTICE messages can happen in any state besides COPY
* OUT; always process them right away.
*/
if (id == 'A')
{
if (getNotify(conn))
return;
}
else if (id == 'N')
{
if (getNotice(conn))
return;
}
else
{
/*
* Other messages should only be processed while in BUSY
* state. (In particular, in READY state we hold off further
* parsing until the application collects the current
* PGresult.) If the state is IDLE then we got trouble.
*/
if (conn->asyncStatus != PGASYNC_BUSY)
{
if (conn->asyncStatus == PGASYNC_IDLE)
{
sprintf(conn->errorMessage,
"Backend message type 0x%02x arrived while idle\n",
id);
DONOTICE(conn, conn->errorMessage);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
}
return;
}
switch (id)
{
case 'C': /* command complete */
if (conn->result == NULL)
conn->result = PQmakeEmptyPGresult(conn,
PGRES_COMMAND_OK);
if (pqGets(conn->result->cmdStatus, CMDSTATUS_LEN, conn))
return;
conn->asyncStatus = PGASYNC_READY;
break;
case 'E': /* error return */
if (pqGets(conn->errorMessage, ERROR_MSG_LENGTH, conn))
return;
/* delete any partially constructed result */
pqClearAsyncResult(conn);
/* and build an error result holding the error message */
conn->result = PQmakeEmptyPGresult(conn,
PGRES_FATAL_ERROR);
conn->asyncStatus = PGASYNC_READY;
break;
case 'Z': /* backend is ready for new query */
conn->asyncStatus = PGASYNC_IDLE;
break;
case 'I': /* empty query */
/* read and throw away the closing '\0' */
if (pqGetc(&id, conn))
return;
if (id != '\0')
{
sprintf(conn->errorMessage,
"unexpected character %c following 'I'\n", id);
DONOTICE(conn, conn->errorMessage);
}
if (conn->result == NULL)
conn->result = PQmakeEmptyPGresult(conn,
PGRES_EMPTY_QUERY);
conn->asyncStatus = PGASYNC_READY;
break;
case 'K': /* secret key data from the backend */
/*
* This is expected only during backend startup, but
* it's just as easy to handle it as part of the main
* loop. Save the data and continue processing.
*/
if (pqGetInt(&(conn->be_pid), 4, conn))
return;
if (pqGetInt(&(conn->be_key), 4, conn))
return;
break;
case 'P': /* synchronous (normal) portal */
if (pqGets(conn->errorMessage, ERROR_MSG_LENGTH, conn))
return;
/* We pretty much ignore this message type... */
break;
case 'T': /* row descriptions (start of query
* results) */
if (conn->result == NULL)
{
/* First 'T' in a query sequence */
if (getRowDescriptions(conn))
return;
}
else
{
/*
* A new 'T' message is treated as the start of
* another PGresult. (It is not clear that this
* is really possible with the current backend.)
* We stop parsing until the application accepts
* the current result.
*/
conn->asyncStatus = PGASYNC_READY;
return;
}
break;
case 'D': /* ASCII data tuple */
if (conn->result != NULL)
{
/* Read another tuple of a normal query response */
if (getAnotherTuple(conn, FALSE))
return;
}
else
{
sprintf(conn->errorMessage,
"Backend sent D message without prior T\n");
DONOTICE(conn, conn->errorMessage);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
return;
}
break;
case 'B': /* Binary data tuple */
if (conn->result != NULL)
{
/* Read another tuple of a normal query response */
if (getAnotherTuple(conn, TRUE))
return;
}
else
{
sprintf(conn->errorMessage,
"Backend sent B message without prior T\n");
DONOTICE(conn, conn->errorMessage);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
return;
}
break;
case 'G': /* Start Copy In */
conn->asyncStatus = PGASYNC_COPY_IN;
break;
case 'H': /* Start Copy Out */
conn->asyncStatus = PGASYNC_COPY_OUT;
break;
default:
sprintf(conn->errorMessage,
"unknown protocol character '%c' read from backend. "
"(The protocol character is the first character the "
"backend sends in response to a query it receives).\n",
id);
/* Discard the unexpected message; good idea?? */
conn->inStart = conn->inEnd;
/* delete any partially constructed result */
pqClearAsyncResult(conn);
/* and build an error result holding the error message */
conn->result = PQmakeEmptyPGresult(conn,
PGRES_FATAL_ERROR);
conn->asyncStatus = PGASYNC_READY;
return;
} /* switch on protocol character */
}
/* Successfully consumed this message */
conn->inStart = conn->inCursor;
}
}
/*
* parseInput subroutine to read a 'T' (row descriptions) message.
* We build a PGresult structure containing the attribute data.
* Returns: 0 if completed message, EOF if not enough data yet.
*
* Note that if we run out of data, we have to release the partially
* constructed PGresult, and rebuild it again next time. Fortunately,
* that shouldn't happen often, since 'T' messages usually fit in a packet.
*/
static int
getRowDescriptions(PGconn *conn)
{
PGresult *result;
int nfields;
int i;
result = PQmakeEmptyPGresult(conn, PGRES_TUPLES_OK);
/* parseInput already read the 'T' label. */
/* the next two bytes are the number of fields */
if (pqGetInt(&(result->numAttributes), 2, conn))
{
PQclear(result);
return EOF;
}
nfields = result->numAttributes;
/* allocate space for the attribute descriptors */
if (nfields > 0)
{
result->attDescs = (PGresAttDesc *)
pqResultAlloc(result, nfields * sizeof(PGresAttDesc), TRUE);
MemSet((char *) result->attDescs, 0, nfields * sizeof(PGresAttDesc));
}
/* get type info */
for (i = 0; i < nfields; i++)
{
char typName[MAX_MESSAGE_LEN];
int typid;
int typlen;
int atttypmod;
if (pqGets(typName, MAX_MESSAGE_LEN, conn) ||
pqGetInt(&typid, 4, conn) ||
pqGetInt(&typlen, 2, conn) ||
pqGetInt(&atttypmod, 4, conn))
{
PQclear(result);
return EOF;
}
/*
* Since pqGetInt treats 2-byte integers as unsigned, we need to
* coerce the special value "-1" to signed form. (-1 is sent for
* variable-length fields.) Formerly, libpq effectively did a
* sign-extension on the 2-byte value by storing it in a signed
* short. Now we only coerce the single value 65535 == -1; values
* 32768..65534 are taken as valid field lengths.
*/
if (typlen == 0xFFFF)
typlen = -1;
result->attDescs[i].name = pqResultStrdup(result, typName);
result->attDescs[i].typid = typid;
result->attDescs[i].typlen = typlen;
result->attDescs[i].atttypmod = atttypmod;
}
/* Success! */
conn->result = result;
return 0;
}
/*
* parseInput subroutine to read a 'B' or 'D' (row data) message.
* We add another tuple to the existing PGresult structure.
* Returns: 0 if completed message, EOF if error or not enough data yet.
*
* Note that if we run out of data, we have to suspend and reprocess
* the message after more data is received. We keep a partially constructed
* tuple in conn->curTuple, and avoid reallocating already-allocated storage.
*/
static int
getAnotherTuple(PGconn *conn, int binary)
{
PGresult *result = conn->result;
int nfields = result->numAttributes;
PGresAttValue *tup;
char bitmap[MAX_FIELDS]; /* the backend sends us a bitmap
* of which attributes are null */
int i;
int nbytes; /* the number of bytes in bitmap */
char bmap; /* One byte of the bitmap */
int bitmap_index; /* Its index */
int bitcnt; /* number of bits examined in current byte */
int vlen; /* length of the current field value */
result->binary = binary;
/* Allocate tuple space if first time for this data message */
if (conn->curTuple == NULL)
{
conn->curTuple = (PGresAttValue *)
pqResultAlloc(result, nfields * sizeof(PGresAttValue), TRUE);
if (conn->curTuple == NULL)
goto outOfMemory;
MemSet((char *) conn->curTuple, 0, nfields * sizeof(PGresAttValue));
}
tup = conn->curTuple;
/* Get the null-value bitmap */
nbytes = (nfields + BYTELEN - 1) / BYTELEN;
if (nbytes >= MAX_FIELDS)
{
/* Replace partially constructed result with an error result */
pqClearAsyncResult(conn);
sprintf(conn->errorMessage,
"getAnotherTuple() -- null-values bitmap is too large\n");
conn->result = PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
conn->asyncStatus = PGASYNC_READY;
/* Discard the broken message */
conn->inStart = conn->inEnd;
return EOF;
}
if (pqGetnchar(bitmap, nbytes, conn))
return EOF;
/* Scan the fields */
bitmap_index = 0;
bmap = bitmap[bitmap_index];
bitcnt = 0;
for (i = 0; i < nfields; i++)
{
if (!(bmap & 0200))
{
/* if the field value is absent, make it a null string */
tup[i].value = result->null_field;
tup[i].len = NULL_LEN;
}
else
{
/* get the value length (the first four bytes are for length) */
if (pqGetInt(&vlen, 4, conn))
return EOF;
if (binary == 0)
vlen = vlen - 4;
if (vlen < 0)
vlen = 0;
if (tup[i].value == NULL)
{
tup[i].value = (char *) pqResultAlloc(result, vlen + 1, binary);
if (tup[i].value == NULL)
goto outOfMemory;
}
tup[i].len = vlen;
/* read in the value */
if (vlen > 0)
if (pqGetnchar((char *) (tup[i].value), vlen, conn))
return EOF;
/* we have to terminate this ourselves */
tup[i].value[vlen] = '\0';
}
/* advance the bitmap stuff */
bitcnt++;
if (bitcnt == BYTELEN)
{
bitmap_index++;
bmap = bitmap[bitmap_index];
bitcnt = 0;
}
else
bmap <<= 1;
}
/* Success! Store the completed tuple in the result */
if (!addTuple(result, tup))
goto outOfMemory;
/* and reset for a new message */
conn->curTuple = NULL;
return 0;
outOfMemory:
/* Replace partially constructed result with an error result */
pqClearAsyncResult(conn);
sprintf(conn->errorMessage,
"getAnotherTuple() -- out of memory for result\n");
conn->result = PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
conn->asyncStatus = PGASYNC_READY;
/* Discard the failed message --- good idea? */
conn->inStart = conn->inEnd;
return EOF;
}
/*
* PQisBusy
* Return TRUE if PQgetResult would block waiting for input.
*/
int
PQisBusy(PGconn *conn)
{
if (!conn)
return FALSE;
/* Parse any available data, if our state permits. */
parseInput(conn);
/* PQgetResult will return immediately in all states except BUSY. */
return conn->asyncStatus == PGASYNC_BUSY;
}
/*
* PQgetResult
* Get the next PGresult produced by a query.
* Returns NULL if and only if no query work remains.
*/
PGresult *
PQgetResult(PGconn *conn)
{
PGresult *res;
if (!conn)
return NULL;
/* Parse any available data, if our state permits. */
parseInput(conn);
/* If not ready to return something, block until we are. */
while (conn->asyncStatus == PGASYNC_BUSY)
{
/* Wait for some more data, and load it. */
if (pqWait(TRUE, FALSE, conn) ||
pqReadData(conn) < 0)
{
pqClearAsyncResult(conn);
conn->asyncStatus = PGASYNC_IDLE;
/* conn->errorMessage has been set by pqWait or pqReadData. */
return PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
}
/* Parse it. */
parseInput(conn);
}
/* Return the appropriate thing. */
switch (conn->asyncStatus)
{
case PGASYNC_IDLE:
res = NULL; /* query is complete */
break;
case PGASYNC_READY:
/*
* conn->result is the PGresult to return. If it is NULL
* (which probably shouldn't happen) we assume there is an
* appropriate error message in conn->errorMessage.
*/
res = conn->result;
conn->result = NULL;/* handing over ownership to caller */
conn->curTuple = NULL; /* just in case */
if (!res)
res = PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
else
{
/*
* Make sure PQerrorMessage agrees with result; it could
* be that we have done other operations that changed
* errorMessage since the result's error message was
* saved.
*/
strcpy(conn->errorMessage, PQresultErrorMessage(res));
}
/* Set the state back to BUSY, allowing parsing to proceed. */
conn->asyncStatus = PGASYNC_BUSY;
break;
case PGASYNC_COPY_IN:
res = PQmakeEmptyPGresult(conn, PGRES_COPY_IN);
break;
case PGASYNC_COPY_OUT:
res = PQmakeEmptyPGresult(conn, PGRES_COPY_OUT);
break;
default:
sprintf(conn->errorMessage,
"PQgetResult: Unexpected asyncStatus %d\n",
(int) conn->asyncStatus);
res = PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
break;
}
return res;
}
/*
* PQexec
* send a query to the backend and package up the result in a PGresult
*
* If the query was not even sent, return NULL; conn->errorMessage is set to
* a relevant message.
* If the query was sent, a new PGresult is returned (which could indicate
* either success or failure).
* The user is responsible for freeing the PGresult via PQclear()
* when done with it.
*/
PGresult *
PQexec(PGconn *conn, const char *query)
{
PGresult *result;
PGresult *lastResult;
/*
* Silently discard any prior query result that application didn't
* eat. This is probably poor design, but it's here for backward
* compatibility.
*/
while ((result = PQgetResult(conn)) != NULL)
{
if (result->resultStatus == PGRES_COPY_IN ||
result->resultStatus == PGRES_COPY_OUT)
{
PQclear(result);
sprintf(conn->errorMessage,
"PQexec: you gotta get out of a COPY state yourself.\n");
return NULL;
}
PQclear(result);
}
/* OK to send the message */
if (!PQsendQuery(conn, query))
return NULL;
/*
* For backwards compatibility, return the last result if there are
* more than one. We have to stop if we see copy in/out, however. We
* will resume parsing when application calls PQendcopy.
*/
lastResult = NULL;
while ((result = PQgetResult(conn)) != NULL)
{
if (lastResult)
PQclear(lastResult);
lastResult = result;
if (result->resultStatus == PGRES_COPY_IN ||
result->resultStatus == PGRES_COPY_OUT)
break;
}
return lastResult;
}
/*
* Attempt to read a Notice response message.
* This is possible in several places, so we break it out as a subroutine.
* Entry: 'N' flag character has already been consumed.
* Exit: returns 0 if successfully consumed Notice message.
* returns EOF if not enough data.
*/
static int
getNotice(PGconn *conn)
{
if (pqGets(conn->errorMessage, ERROR_MSG_LENGTH, conn))
return EOF;
DONOTICE(conn, conn->errorMessage);
return 0;
}
/*
* Attempt to read a Notify response message.
* This is possible in several places, so we break it out as a subroutine.
* Entry: 'A' flag character has already been consumed.
* Exit: returns 0 if successfully consumed Notify message.
* returns EOF if not enough data.
*/
static int
getNotify(PGconn *conn)
{
PGnotify tempNotify;
PGnotify *newNotify;
if (pqGetInt(&(tempNotify.be_pid), 4, conn))
return EOF;
if (pqGets(tempNotify.relname, NAMEDATALEN, conn))
return EOF;
newNotify = (PGnotify *) malloc(sizeof(PGnotify));
memcpy(newNotify, &tempNotify, sizeof(PGnotify));
DLAddTail(conn->notifyList, DLNewElem(newNotify));
return 0;
}
/*
* PQnotifies
* returns a PGnotify* structure of the latest async notification
* that has not yet been handled
*
* returns NULL, if there is currently
* no unhandled async notification from the backend
*
* the CALLER is responsible for FREE'ing the structure returned
*/
PGnotify *
PQnotifies(PGconn *conn)
{
Dlelem *e;
PGnotify *event;
if (!conn)
return NULL;
/* Parse any available data to see if we can extract NOTIFY messages. */
parseInput(conn);
/* RemHead returns NULL if list is empty */
e = DLRemHead(conn->notifyList);
if (!e)
return NULL;
event = (PGnotify *) DLE_VAL(e);
DLFreeElem(e);
return event;
}
/*
* PQgetline - gets a newline-terminated string from the backend.
*
* Chiefly here so that applications can use "COPY <rel> to stdout"
* and read the output string. Returns a null-terminated string in s.
*
* PQgetline reads up to maxlen-1 characters (like fgets(3)) but strips
* the terminating \n (like gets(3)).
*
* CAUTION: the caller is responsible for detecting the end-of-copy signal
* (a line containing just "\.") when using this routine.
*
* RETURNS:
* EOF if it is detected or invalid arguments are given
* 0 if EOL is reached (i.e., \n has been read)
* (this is required for backward-compatibility -- this
* routine used to always return EOF or 0, assuming that
* the line ended within maxlen bytes.)
* 1 in other cases (i.e., the buffer was filled before \n is reached)
*/
int
PQgetline(PGconn *conn, char *s, int maxlen)
{
int result = 1; /* return value if buffer overflows */
if (!s || maxlen <= 0)
return EOF;
if (!conn || conn->sock < 0)
{
*s = '\0';
return EOF;
}
/*
* Since this is a purely synchronous routine, we don't bother to
* maintain conn->inCursor; there is no need to back up.
*/
while (maxlen > 1)
{
if (conn->inStart < conn->inEnd)
{
char c = conn->inBuffer[conn->inStart++];
if (c == '\n')
{
result = 0; /* success exit */
break;
}
*s++ = c;
maxlen--;
}
else
{
/* need to load more data */
if (pqWait(TRUE, FALSE, conn) ||
pqReadData(conn) < 0)
{
result = EOF;
break;
}
}
}
*s = '\0';
return result;
}
/*
* PQgetlineAsync - gets a newline-terminated string without blocking.
*
* This routine is for applications that want to do "COPY <rel> to stdout"
* asynchronously, that is without blocking. Having issued the COPY command
* and gotten a PGRES_COPY_OUT response, the app should call PQconsumeInput
* and this routine until the end-of-data signal is detected. Unlike
* PQgetline, this routine takes responsibility for detecting end-of-data.
*
* On each call, PQgetlineAsync will return data if a complete newline-
* terminated data line is available in libpq's input buffer, or if the
* incoming data line is too long to fit in the buffer offered by the caller.
* Otherwise, no data is returned until the rest of the line arrives.
*
* If -1 is returned, the end-of-data signal has been recognized (and removed
* from libpq's input buffer). The caller *must* next call PQendcopy and
* then return to normal processing.
*
* RETURNS:
* -1 if the end-of-copy-data marker has been recognized
* 0 if no data is available
* >0 the number of bytes returned.
* The data returned will not extend beyond a newline character. If possible
* a whole line will be returned at one time. But if the buffer offered by
* the caller is too small to hold a line sent by the backend, then a partial
* data line will be returned. This can be detected by testing whether the
* last returned byte is '\n' or not.
* The returned string is *not* null-terminated.
*/
int
PQgetlineAsync(PGconn *conn, char *buffer, int bufsize)
{
int avail;
if (!conn || conn->asyncStatus != PGASYNC_COPY_OUT)
return -1; /* we are not doing a copy... */
/*
* Move data from libpq's buffer to the caller's. We want to accept
* data only in units of whole lines, not partial lines. This ensures
* that we can recognize the terminator line "\\.\n". (Otherwise, if
* it happened to cross a packet/buffer boundary, we might hand the
* first one or two characters off to the caller, which we shouldn't.)
*/
conn->inCursor = conn->inStart;
avail = bufsize;
while (avail > 0 && conn->inCursor < conn->inEnd)
{
char c = conn->inBuffer[conn->inCursor++];
*buffer++ = c;
--avail;
if (c == '\n')
{
/* Got a complete line; mark the data removed from libpq */
conn->inStart = conn->inCursor;
/* Is it the endmarker line? */
if (bufsize - avail == 3 && buffer[-3] == '\\' && buffer[-2] == '.')
return -1;
/* No, return the data line to the caller */
return bufsize - avail;
}
}
/*
* We don't have a complete line. We'd prefer to leave it in libpq's
* buffer until the rest arrives, but there is a special case: what if
* the line is longer than the buffer the caller is offering us? In
* that case we'd better hand over a partial line, else we'd get into
* an infinite loop. Do this in a way that ensures we can't
* misrecognize a terminator line later: leave last 3 characters in
* libpq buffer.
*/
if (avail == 0 && bufsize > 3)
{
conn->inStart = conn->inCursor - 3;
return bufsize - 3;
}
return 0;
}
/*
* PQputline -- sends a string to the backend.
* Returns 0 if OK, EOF if not.
*
* Chiefly here so that applications can use "COPY <rel> from stdin".
*/
int
PQputline(PGconn *conn, const char *s)
{
if (!conn || conn->sock < 0)
return EOF;
return pqPutnchar(s, strlen(s), conn);
}
/*
* PQputnbytes -- like PQputline, but buffer need not be null-terminated.
* Returns 0 if OK, EOF if not.
*/
int
PQputnbytes(PGconn *conn, const char *buffer, int nbytes)
{
if (!conn || conn->sock < 0)
return EOF;
return pqPutnchar(buffer, nbytes, conn);
}
/*
* PQendcopy
* After completing the data transfer portion of a copy in/out,
* the application must call this routine to finish the command protocol.
*
* RETURNS:
* 0 on success
* 1 on failure
*/
int
PQendcopy(PGconn *conn)
{
PGresult *result;
if (!conn)
return 0;
if (conn->asyncStatus != PGASYNC_COPY_IN &&
conn->asyncStatus != PGASYNC_COPY_OUT)
{
sprintf(conn->errorMessage,
"PQendcopy() -- I don't think there's a copy in progress.");
return 1;
}
(void) pqFlush(conn); /* make sure no data is waiting to be sent */
/* Return to active duty */
conn->asyncStatus = PGASYNC_BUSY;
conn->errorMessage[0] = '\0';
/* Wait for the completion response */
result = PQgetResult(conn);
/* Expecting a successful result */
if (result && result->resultStatus == PGRES_COMMAND_OK)
{
PQclear(result);
return 0;
}
/*
* Trouble. The worst case is that we've lost sync with the backend
* entirely due to application screwup of the copy in/out protocol. To
* recover, reset the connection (talk about using a sledgehammer...)
*/
PQclear(result);
if (conn->errorMessage[0])
DONOTICE(conn, conn->errorMessage);
DONOTICE(conn, "PQendcopy: resetting connection\n");
PQreset(conn);
return 1;
}
/* ----------------
* PQfn - Send a function call to the POSTGRES backend.
*
* conn : backend connection
* fnid : function id
* result_buf : pointer to result buffer (&int if integer)
* result_len : length of return value.
* actual_result_len: actual length returned. (differs from result_len
* for varlena structures.)
* result_type : If the result is an integer, this must be 1,
* otherwise this should be 0
* args : pointer to an array of function arguments.
* (each has length, if integer, and value/pointer)
* nargs : # of arguments in args array.
*
* RETURNS
* PGresult with status = PGRES_COMMAND_OK if successful.
* *actual_result_len is > 0 if there is a return value, 0 if not.
* PGresult with status = PGRES_FATAL_ERROR if backend returns an error.
* NULL on communications failure. conn->errorMessage will be set.
* ----------------
*/
PGresult *
PQfn(PGconn *conn,
int fnid,
int *result_buf,
int *actual_result_len,
int result_is_int,
PQArgBlock *args,
int nargs)
{
bool needInput = false;
ExecStatusType status = PGRES_FATAL_ERROR;
char id;
int i;
*actual_result_len = 0;
printf("fe-exec, PQfn, A\n");
if (!conn)
return NULL;
if (conn->sock < 0 || conn->asyncStatus != PGASYNC_IDLE)
{
printf("fe-exec, PQfn, B\n");
sprintf(conn->errorMessage, "PQfn() -- connection in wrong state\n");
return NULL;
}
/* clear the error string */
conn->errorMessage[0] = '\0';
if (pqPuts("F ", conn) || /* function */
pqPutInt(fnid, 4, conn) || /* function id */
pqPutInt(nargs, 4, conn)) /* # of args */
{
printf("fe-exec, PQfn, C\n");
handleSendFailure(conn);
return NULL;
}
for (i = 0; i < nargs; ++i)
{ /* len.int4 + contents */
printf("fe-exec, PQfn, C: i:%d\n",i);
if (pqPutInt(args[i].len, 4, conn))
{
printf("fe-exec, PQfn, D\n");
handleSendFailure(conn);
return NULL;
}
if (args[i].isint)
{
printf("fe-exec, PQfn, E\n");
if (pqPutInt(args[i].u.integer, 4, conn))
{
printf("fe-exec, PQfn, F\n");
handleSendFailure(conn);
return NULL;
}
}
else
{
printf("fe-exec, PQfn, G\n");
if (pqPutnchar((char *) args[i].u.ptr, args[i].len, conn))
{
handleSendFailure(conn);
return NULL;
}
}
}
if (pqFlush(conn))
{
printf("fe-exec, PQfn, H\n");
handleSendFailure(conn);
return NULL;
}
for (;;)
{
printf("fe-exec, PQfn, I\n");
if (needInput)
{
printf("fe-exec, PQfn, J\n");
/* Wait for some data to arrive (or for the channel to close) */
if (pqWait(TRUE, FALSE, conn) ||
pqReadData(conn) < 0)
break;
}
/*
* Scan the message. If we run out of data, loop around to try
* again.
*/
conn->inCursor = conn->inStart;
needInput = true;
if (pqGetc(&id, conn))
{
printf("fe-exec, PQfn, J1: %c\n",id);
continue;
}
/*
* We should see V or E response to the command, but might get N
* and/or A notices first. We also need to swallow the final Z
* before returning.
*/
printf("fe-exec, PQfn, ID: %c\n", id);
switch (id)
{
case 'V': /* function result */
printf("fe-exec, PQfn, 0\n");
if (pqGetc(&id, conn))
continue;
printf("fe-exec, PQfn, 1\n");
if (id == 'G')
{
printf("fe-exec, PQfn, 2\n");
/* function returned nonempty value */
if (pqGetInt(actual_result_len, 4, conn))
continue;
if (result_is_int)
{
printf("fe-exec, PQfn, 3: %d\n",*actual_result_len);
if (pqGetInt(result_buf, 4, conn))
{
printf("fe-exec, PQfn, 3a: result_buf: %lu\n", *result_buf);
continue;
}
printf("fe-exec, PQfn, 3b: result_buf: %lu\n", *result_buf);
}
else
{
printf("fe-exec, PQfn, 4\n");
if (pqGetnchar((char *) result_buf,
*actual_result_len,
conn))
continue;
}
if (pqGetc(&id, conn)) /* get the last '0' */
{
printf("fe-exec, PQfn, 4a\n");
continue;
}
}
if (id == '0')
{
printf("fe-exec, PQfn, 5\n");
/* correctly finished function result message */
status = PGRES_COMMAND_OK;
}
else
{
printf("fe-exec, PQfn, 6\n");
/* The backend violates the protocol. */
sprintf(conn->errorMessage,
"FATAL: PQfn: protocol error: id=%x\n", id);
conn->inStart = conn->inCursor;
return PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
}
break;
case 'E': /* error return */
if (pqGets(conn->errorMessage, ERROR_MSG_LENGTH, conn))
continue;
status = PGRES_FATAL_ERROR;
break;
case 'A': /* notify message */
/* handle notify and go back to processing return values */
if (getNotify(conn))
continue;
break;
case 'N': /* notice */
/* handle notice and go back to processing return values */
if (getNotice(conn))
continue;
break;
case 'Z': /* backend is ready for new query */
/* consume the message and exit */
conn->inStart = conn->inCursor;
return PQmakeEmptyPGresult(conn, status);
default:
/* The backend violates the protocol. */
sprintf(conn->errorMessage,
"FATAL: PQfn: protocol error: id=%x\n", id);
conn->inStart = conn->inCursor;
return PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
}
/* Completed this message, keep going */
conn->inStart = conn->inCursor;
needInput = false;
}
/* we fall out of the loop only upon failing to read data */
return PQmakeEmptyPGresult(conn, PGRES_FATAL_ERROR);
}
/* ====== accessor funcs for PGresult ======== */
ExecStatusType
PQresultStatus(PGresult *res)
{
if (!res)
return PGRES_NONFATAL_ERROR;
return res->resultStatus;
}
const char *
PQresStatus(ExecStatusType status)
{
if (((int) status) < 0 ||
((int) status) >= (sizeof(pgresStatus) / sizeof(pgresStatus[0])))
return "Invalid ExecStatusType code";
return pgresStatus[status];
}
const char *
PQresultErrorMessage(PGresult *res)
{
if (!res || !res->errMsg)
return "";
return res->errMsg;
}
int
PQntuples(PGresult *res)
{
if (!res)
return 0;
return res->ntups;
}
int
PQnfields(PGresult *res)
{
if (!res)
return 0;
return res->numAttributes;
}
int
PQbinaryTuples(PGresult *res)
{
if (!res)
return 0;
return res->binary;
}
/*
* Helper routines to range-check field numbers and tuple numbers.
* Return TRUE if OK, FALSE if not
*/
static int
check_field_number(const char *routineName, PGresult *res, int field_num)
{
if (!res)
return FALSE; /* no way to display error message... */
if (field_num < 0 || field_num >= res->numAttributes)
{
if (res->conn)
{
sprintf(res->conn->errorMessage,
"%s: ERROR! field number %d is out of range 0..%d\n",
routineName, field_num, res->numAttributes - 1);
DONOTICE(res->conn, res->conn->errorMessage);
}
return FALSE;
}
return TRUE;
}
static int
check_tuple_field_number(const char *routineName, PGresult *res,
int tup_num, int field_num)
{
if (!res)
return FALSE; /* no way to display error message... */
if (tup_num < 0 || tup_num >= res->ntups)
{
if (res->conn)
{
sprintf(res->conn->errorMessage,
"%s: ERROR! tuple number %d is out of range 0..%d\n",
routineName, tup_num, res->ntups - 1);
DONOTICE(res->conn, res->conn->errorMessage);
}
return FALSE;
}
if (field_num < 0 || field_num >= res->numAttributes)
{
if (res->conn)
{
sprintf(res->conn->errorMessage,
"%s: ERROR! field number %d is out of range 0..%d\n",
routineName, field_num, res->numAttributes - 1);
DONOTICE(res->conn, res->conn->errorMessage);
}
return FALSE;
}
return TRUE;
}
/*
returns NULL if the field_num is invalid
*/
char *
PQfname(PGresult *res, int field_num)
{
if (!check_field_number("PQfname", res, field_num))
return NULL;
if (res->attDescs)
return res->attDescs[field_num].name;
else
return NULL;
}
/*
returns -1 on a bad field name
*/
int
PQfnumber(PGresult *res, const char *field_name)
{
int i;
char *field_case;
if (!res)
return -1;
if (field_name == NULL ||
field_name[0] == '\0' ||
res->attDescs == NULL)
return -1;
field_case = strdup(field_name);
if (*field_case == '"')
{
strcpy(field_case, field_case + 1);
*(field_case + strlen(field_case) - 1) = '\0';
}
else
for (i = 0; field_case[i]; i++)
if (isascii((unsigned char) field_case[i]) &&
isupper(field_case[i]))
field_case[i] = tolower(field_case[i]);
for (i = 0; i < res->numAttributes; i++)
{
if (strcmp(field_case, res->attDescs[i].name) == 0)
{
free(field_case);
return i;
}
}
free(field_case);
return -1;
}
Oid
PQftype(PGresult *res, int field_num)
{
if (!check_field_number("PQftype", res, field_num))
return InvalidOid;
if (res->attDescs)
return res->attDescs[field_num].typid;
else
return InvalidOid;
}
int
PQfsize(PGresult *res, int field_num)
{
if (!check_field_number("PQfsize", res, field_num))
return 0;
if (res->attDescs)
return res->attDescs[field_num].typlen;
else
return 0;
}
int
PQfmod(PGresult *res, int field_num)
{
if (!check_field_number("PQfmod", res, field_num))
return 0;
if (res->attDescs)
return res->attDescs[field_num].atttypmod;
else
return 0;
}
char *
PQcmdStatus(PGresult *res)
{
if (!res)
return NULL;
return res->cmdStatus;
}
/*
PQoidStatus -
if the last command was an INSERT, return the oid string
if not, return ""
*/
const char *
PQoidStatus(PGresult *res)
{
char *p,
*e,
*scan;
int slen,
olen;
if (!res)
return "";
if (strncmp(res->cmdStatus, "INSERT ", 7) != 0)
return "";
/*----------
* The cmdStatus string looks like
* INSERT oid count\0
* In order to be able to return an ordinary C string without
* damaging the result for PQcmdStatus or PQcmdTuples, we copy
* the oid part of the string to just after the null, so that
* cmdStatus looks like
* INSERT oid count\0oid\0
* ^ our return value points here
* Pretty klugy eh? This routine should've just returned an Oid value.
*----------
*/
slen = strlen(res->cmdStatus);
p = res->cmdStatus + 7; /* where oid is now */
e = res->cmdStatus + slen + 1; /* where to put the oid string */
for (scan = p; *scan && *scan != ' ';)
scan++;
olen = scan - p;
if (slen + olen + 2 > sizeof(res->cmdStatus))
return ""; /* something very wrong if it doesn't fit */
strncpy(e, p, olen);
e[olen] = '\0';
return e;
}
/*
PQcmdTuples -
if the last command was an INSERT/UPDATE/DELETE, return number
of inserted/affected tuples, if not, return ""
*/
const char *
PQcmdTuples(PGresult *res)
{
if (!res)
return "";
if (strncmp(res->cmdStatus, "INSERT", 6) == 0 ||
strncmp(res->cmdStatus, "DELETE", 6) == 0 ||
strncmp(res->cmdStatus, "UPDATE", 6) == 0)
{
char *p = res->cmdStatus + 6;
if (*p == 0)
{
if (res->conn)
{
sprintf(res->conn->errorMessage,
"PQcmdTuples (%s) -- bad input from server\n",
res->cmdStatus);
DONOTICE(res->conn, res->conn->errorMessage);
}
return "";
}
p++;
if (*(res->cmdStatus) != 'I') /* UPDATE/DELETE */
return p;
while (*p != ' ' && *p)
p++; /* INSERT: skip oid */
if (*p == 0)
{
if (res->conn)
{
sprintf(res->conn->errorMessage,
"PQcmdTuples (INSERT) -- there's no # of tuples\n");
DONOTICE(res->conn, res->conn->errorMessage);
}
return "";
}
p++;
return p;
}
return "";
}
/*
PQgetvalue:
return the value of field 'field_num' of row 'tup_num'
If res is binary, then the value returned is NOT a null-terminated
ASCII string, but the binary representation in the server's native
format.
if res is not binary, a null-terminated ASCII string is returned.
*/
char *
PQgetvalue(PGresult *res, int tup_num, int field_num)
{
if (!check_tuple_field_number("PQgetvalue", res, tup_num, field_num))
return NULL;
return res->tuples[tup_num][field_num].value;
}
/* PQgetlength:
returns the length of a field value in bytes. If res is binary,
i.e. a result of a binary portal, then the length returned does
NOT include the size field of the varlena. (The data returned
by PQgetvalue doesn't either.)
*/
int
PQgetlength(PGresult *res, int tup_num, int field_num)
{
if (!check_tuple_field_number("PQgetlength", res, tup_num, field_num))
return 0;
if (res->tuples[tup_num][field_num].len != NULL_LEN)
return res->tuples[tup_num][field_num].len;
else
return 0;
}
/* PQgetisnull:
returns the null status of a field value.
*/
int
PQgetisnull(PGresult *res, int tup_num, int field_num)
{
if (!check_tuple_field_number("PQgetisnull", res, tup_num, field_num))
return 1; /* pretend it is null */
if (res->tuples[tup_num][field_num].len == NULL_LEN)
return 1;
else
return 0;
}
fe-lobj.c (text/plain, 12.7K) download
/*-------------------------------------------------------------------------
*
* fe-lobj.c
* Front-end large object interface
*
* Copyright (c) 1994, Regents of the University of California
*
*
* IDENTIFICATION
* $Header: /usr/local/cvsroot/pgsql/src/interfaces/libpq/fe-lobj.c,v 1.20 1999/05/10 00:46:26 momjian Exp $
*
*-------------------------------------------------------------------------
*/
#include "libpq-fe.h"
#include "libpq-int.h"
#include "postgres.h"
#ifdef WIN32
#include "win32.h"
#include <io.h>
#else
#if !defined(NO_UNISTD_H)
#include <unistd.h>
#endif
#endif
#include <string.h>
#include <fcntl.h>
#include <sys/stat.h>
#include "libpq/libpq-fs.h" /* must come after sys/stat.h */
#define LO_BUFSIZE 1024
static int lo_initialize(PGconn *conn);
/*
* lo_open
* opens an existing large object
*
* returns the file descriptor for use in later lo_* calls
* return -1 upon failure.
*/
int
lo_open(PGconn *conn, Oid lobjId, int mode)
{
int fd;
int result_len;
PQArgBlock argv[2];
PGresult *res;
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = lobjId;
argv[1].isint = 1;
argv[1].len = 4;
argv[1].u.integer = mode;
printf("fe-lobj, lo_open: 0\n");
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
printf("fe-lobj, lo_open: 1\n");
if (lo_initialize(conn) < 0)
return -1;
}
printf("fe-lobj, lo_open: 1a\n");
res = PQfn(conn, conn->lobjfuncs->fn_lo_open, &fd, &result_len, 1, argv, 2);
printf("fe-lobj, lo_open: 2 (%x,%x)\n",conn->lobjfuncs->fn_lo_open,fd);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
int WN_TEMP;
printf("fe-lobj, lo_open: 3\n");
PQclear(res);
/* have to do this to reset offset in shared fd cache */
/* but only if fd is valid */
if (fd >= 0 && (WN_TEMP = lo_lseek(conn, fd, 0L, SEEK_SET)) < 0)
{
printf("fe-lobj, lo_open: 3a (%x,%x)\n",fd,WN_TEMP);
return -1;
}
printf("fe-lobj, lo_open: 3b\n");
return fd;
}
else
{
printf("fe-lobj, lo_open: 4\n");
PQclear(res);
return -1;
}
}
/*
* lo_close
* closes an existing large object
*
* returns 0 upon success
* returns -1 upon failure.
*/
int
lo_close(PGconn *conn, int fd)
{
PQArgBlock argv[1];
PGresult *res;
int retval;
int result_len;
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
if (lo_initialize(conn) < 0)
return -1;
}
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = fd;
res = PQfn(conn, conn->lobjfuncs->fn_lo_close,
&retval, &result_len, 1, argv, 1);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
PQclear(res);
return retval;
}
else
{
PQclear(res);
return -1;
}
}
/*
* lo_read
* read len bytes of the large object into buf
*
* returns the length of bytes read.
* the CALLER must have allocated enough space to hold the result returned
*/
int
lo_read(PGconn *conn, int fd, char *buf, int len)
{
PQArgBlock argv[2];
PGresult *res;
int result_len;
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
if (lo_initialize(conn) < 0)
return -1;
}
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = fd;
argv[1].isint = 1;
argv[1].len = 4;
argv[1].u.integer = len;
res = PQfn(conn, conn->lobjfuncs->fn_lo_read,
(int *) buf, &result_len, 0, argv, 2);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
PQclear(res);
return result_len;
}
else
{
PQclear(res);
return -1;
}
}
/*
* lo_write
* write len bytes of buf into the large object fd
*
*/
int
lo_write(PGconn *conn, int fd, char *buf, int len)
{
PQArgBlock argv[2];
PGresult *res;
int result_len;
int retval;
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
if (lo_initialize(conn) < 0)
return -1;
}
if (len <= 0)
return 0;
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = fd;
argv[1].isint = 0;
argv[1].len = len;
argv[1].u.ptr = (int *) buf;
res = PQfn(conn, conn->lobjfuncs->fn_lo_write,
&retval, &result_len, 1, argv, 2);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
PQclear(res);
return retval;
}
else
{
PQclear(res);
return -1;
}
}
/*
* lo_lseek
* change the current read or write location on a large object
* currently, only L_SET is a legal value for whence
*
*/
int
lo_lseek(PGconn *conn, int fd, int offset, int whence)
{
PQArgBlock argv[3];
PGresult *res;
int retval;
int result_len;
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
if (lo_initialize(conn) < 0)
return -1;
}
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = fd;
argv[1].isint = 1;
argv[1].len = 4;
argv[1].u.integer = offset;
argv[2].isint = 1;
argv[2].len = 4;
argv[2].u.integer = whence;
res = PQfn(conn, conn->lobjfuncs->fn_lo_lseek,
&retval, &result_len, 1, argv, 3);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
PQclear(res);
return retval;
}
else
{
PQclear(res);
return -1;
}
}
/*
* lo_creat
* create a new large object
* the mode is a bitmask describing different attributes of the new object
*
* returns the oid of the large object created or
* InvalidOid upon failure
*/
Oid
lo_creat(PGconn *conn, int mode)
{
PQArgBlock argv[1];
PGresult *res;
int retval;
int result_len;
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
if (lo_initialize(conn) < 0)
return -1;
}
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = mode;
res = PQfn(conn, conn->lobjfuncs->fn_lo_creat,
&retval, &result_len, 1, argv, 1);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
PQclear(res);
return (Oid) retval;
}
else
{
PQclear(res);
return InvalidOid;
}
}
/*
* lo_tell
* returns the current seek location of the large object
*
*/
int
lo_tell(PGconn *conn, int fd)
{
int retval;
PQArgBlock argv[1];
PGresult *res;
int result_len;
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
if (lo_initialize(conn) < 0)
return -1;
}
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = fd;
res = PQfn(conn, conn->lobjfuncs->fn_lo_tell,
&retval, &result_len, 1, argv, 1);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
PQclear(res);
return retval;
}
else
{
PQclear(res);
return -1;
}
}
/*
* lo_unlink
* delete a file
*
*/
int
lo_unlink(PGconn *conn, Oid lobjId)
{
PQArgBlock argv[1];
PGresult *res;
int result_len;
int retval;
if (conn->lobjfuncs == (PGlobjfuncs *) NULL)
{
if (lo_initialize(conn) < 0)
return -1;
}
argv[0].isint = 1;
argv[0].len = 4;
argv[0].u.integer = lobjId;
res = PQfn(conn, conn->lobjfuncs->fn_lo_unlink,
&retval, &result_len, 1, argv, 1);
if (PQresultStatus(res) == PGRES_COMMAND_OK)
{
PQclear(res);
return retval;
}
else
{
PQclear(res);
return -1;
}
}
/*
* lo_import -
* imports a file as an (inversion) large object.
* returns the oid of that object upon success,
* returns InvalidOid upon failure
*
*/
Oid
lo_import(PGconn *conn, char *filename)
{
int fd;
int nbytes,
tmp;
char buf[LO_BUFSIZE];
Oid lobjOid;
int lobj;
/*
* open the file to be read in
*/
#ifndef __CYGWIN32__
fd = open(filename, O_RDONLY, 0666);
#else
fd = open(filename, O_RDONLY | O_BINARY, 0666);
#endif
if (fd < 0)
{ /* error */
sprintf(conn->errorMessage,
"lo_import: can't open unix file\"%s\"\n", filename);
return InvalidOid;
}
/*
* create an inversion "object"
*/
lobjOid = lo_creat(conn, INV_READ | INV_WRITE);
if (lobjOid == InvalidOid)
{
sprintf(conn->errorMessage,
"lo_import: can't create inv object for \"%s\"", filename);
return InvalidOid;
}
lobj = lo_open(conn, lobjOid, INV_WRITE);
if (lobj == -1)
{
sprintf(conn->errorMessage,
"lo_import: could not open inv object oid %u", lobjOid);
return InvalidOid;
}
/*
* read in from the Unix file and write to the inversion file
*/
while ((nbytes = read(fd, buf, LO_BUFSIZE)) > 0)
{
tmp = lo_write(conn, lobj, buf, nbytes);
if (tmp < nbytes)
{
sprintf(conn->errorMessage,
"lo_import: error while reading \"%s\"", filename);
return InvalidOid;
}
}
(void) close(fd);
(void) lo_close(conn, lobj);
return lobjOid;
}
/*
* lo_export -
* exports an (inversion) large object.
* returns -1 upon failure, 1 otherwise
*/
int
lo_export(PGconn *conn, Oid lobjId, char *filename)
{
int fd;
int nbytes,
tmp;
char buf[LO_BUFSIZE];
int lobj;
/*
* create an inversion "object"
*/
lobj = lo_open(conn, lobjId, INV_READ);
if (lobj == -1)
{
sprintf(conn->errorMessage,
"lo_export: can't open inv object %u", lobjId);
return -1;
}
/*
* open the file to be written to
*/
#ifndef __CYGWIN32__
fd = open(filename, O_CREAT | O_WRONLY | O_TRUNC, 0666);
#else
fd = open(filename, O_CREAT | O_WRONLY | O_TRUNC | O_BINARY, 0666);
#endif
if (fd < 0)
{ /* error */
sprintf(conn->errorMessage,
"lo_export: can't open unix file\"%s\"", filename);
return 0;
}
/*
* read in from the Unix file and write to the inversion file
*/
while ((nbytes = lo_read(conn, lobj, buf, LO_BUFSIZE)) > 0)
{
tmp = write(fd, buf, nbytes);
if (tmp < nbytes)
{
sprintf(conn->errorMessage,
"lo_export: error while writing \"%s\"",
filename);
return -1;
}
}
(void) lo_close(conn, lobj);
(void) close(fd);
return 1;
}
/* ----------------
* lo_initialize
*
* Initialize the large object interface for an existing connection.
* We ask the backend about the functions OID's in pg_proc for all
* functions that are required for large object operations.
* ----------------
*/
static int
lo_initialize(PGconn *conn)
{
PGresult *res;
PGlobjfuncs *lobjfuncs;
int n;
char *fname;
Oid foid;
/* ----------------
* Allocate the structure to hold the functions OID's
* ----------------
*/
lobjfuncs = (PGlobjfuncs *) malloc(sizeof(PGlobjfuncs));
if (lobjfuncs == (PGlobjfuncs *) NULL)
{
strcpy(conn->errorMessage,
"FATAL: malloc() failed in lo_initialize()\n");
return -1;
}
MemSet((char *) lobjfuncs, 0, sizeof(PGlobjfuncs));
/* ----------------
* Execute the query to get all the functions at once
* ----------------
*/
res = PQexec(conn, "select proname, oid from pg_proc \
where proname = 'lo_open' \
or proname = 'lo_close' \
or proname = 'lo_creat' \
or proname = 'lo_unlink' \
or proname = 'lo_lseek' \
or proname = 'lo_tell' \
or proname = 'loread' \
or proname = 'lowrite'");
if (res == (PGresult *) NULL)
{
free(lobjfuncs);
return -1;
}
if (res->resultStatus != PGRES_TUPLES_OK)
{
free(lobjfuncs);
PQclear(res);
strcpy(conn->errorMessage,
"ERROR: SELECT didn't return data in lo_initialize()\n");
return -1;
}
/* ----------------
* Examine the result and put the OID's into the struct
* ----------------
*/
for (n = 0; n < PQntuples(res); n++)
{
fname = PQgetvalue(res, n, 0);
foid = (Oid) atoi(PQgetvalue(res, n, 1));
if (!strcmp(fname, "lo_open"))
lobjfuncs->fn_lo_open = foid;
else if (!strcmp(fname, "lo_close"))
lobjfuncs->fn_lo_close = foid;
else if (!strcmp(fname, "lo_creat"))
lobjfuncs->fn_lo_creat = foid;
else if (!strcmp(fname, "lo_unlink"))
lobjfuncs->fn_lo_unlink = foid;
else if (!strcmp(fname, "lo_lseek"))
lobjfuncs->fn_lo_lseek = foid;
else if (!strcmp(fname, "lo_tell"))
lobjfuncs->fn_lo_tell = foid;
else if (!strcmp(fname, "loread"))
lobjfuncs->fn_lo_read = foid;
else if (!strcmp(fname, "lowrite"))
lobjfuncs->fn_lo_write = foid;
}
PQclear(res);
/* ----------------
* Finally check that we really got all large object
* interface functions.
* ----------------
*/
if (lobjfuncs->fn_lo_open == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function lo_open\n");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_close == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function lo_close\n");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_creat == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function lo_creat\n");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_unlink == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function lo_unlink\n");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_lseek == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function lo_lseek\n");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_tell == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function lo_tell\n");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_read == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function loread\n");
free(lobjfuncs);
return -1;
}
if (lobjfuncs->fn_lo_write == 0)
{
strcpy(conn->errorMessage,
"ERROR: Cannot determine OID for function lowrite\n");
free(lobjfuncs);
return -1;
}
/* ----------------
* Put the structure into the connection control
* ----------------
*/
conn->lobjfuncs = lobjfuncs;
return 0;
}
post2.log (text/plain, 3.3K) download
FindExec: found "/usr/local/postgres/bin/postgres" using argv[0]
binding ShmemCreate(key=52e2c1, size=1063936)
/usr/local/postgres/bin/postmaster: ServerLoop: handling reading 4
/usr/local/postgres/bin/postmaster: ServerLoop: handling reading 4
/usr/local/postgres/bin/postmaster: ServerLoop: handling writing 4
/usr/local/postgres/bin/postmaster: BackendStartup: environ dump:
-----------------------------------------
REMOTEHOST=baikonur.home.hottis.intern
HZ=100
HOSTNAME=talshiar
PS1=\h:\w\$
USER=postgres
MACHTYPE=i486-pc-linux-gnu
MAIL=/var/spool/mail/wn
DISPLAY=imhotep:0.0
LOGNAME=postgres
SHLVL=3
HUSHLOGIN=FALSE
SHELL=/bin/bash
PGLIB=/usr/local/postgres/lib
HOSTTYPE=i486
OSTYPE=linux-gnu
TERM=xterm
HOME=/var/local/postgres/home
PGDATA=/var/local/postgres/data
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/local/postgres/bin:/usr/local/samba/bin
_=/usr/local/postgres/bin/postmaster
POSTPORT=5432
POSTID=2147483647
PG_USER=wn
IPC_KEY=5432000
-----------------------------------------
/usr/local/postgres/bin/postmaster: BackendStartup: pid 13256 user wn db adrema socket 4
/usr/local/postgres/bin/postmaster child[13256]: starting with (/usr/local/postgres/bin/postgres -d3 -v131072 -p adrema )
FindExec: found "/usr/local/postgres/bin/postgres" using argv[0]
debug info:
User = wn
RemoteHost = localhost
RemotePort = 0
DatabaseName = adrema
Verbose = 3
Noversion = f
timings = f
dates = Normal
bufsize = 64
sortmem = 512
query echo = f
InitPostgres
StartTransactionCommand
NOTICE: WN: xact: StartTransaction
query: select proname, oid from pg_proc where proname = 'lo_open' or proname = 'lo_close' or proname = 'lo_creat' or proname = 'lo_unlink' or proname = 'lo_lseek' or proname = 'lo_tell' or proname = 'loread' or proname = 'lowrite'
ProcessQuery
CommitTransactionCommand
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 1
StartTransactionCommand
NOTICE: WN: xact: StartTransaction
CommitTransactionCommand
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 1
StartTransactionCommand
NOTICE: WN: xact: StartTransaction
NOTICE: LOopen(19425,131072)
NOTICE: WN: be-fsstubs: lo_open: lobjDesc: 821fcd0
NOTICE: WN: be-fsstubs: newLOfd: MAX_LOBJ_FDS: 256, lobjCookie: 821fcd0
NOTICE: WN: be-fsstubs: newLOfd: i: 0, cookies: 0
NOTICE: WN: be-fsstubs: newLOfd: cookie found!
NOTICE: WN: be-fsstubs: newLOfd: cookies[0]=821fcd0
NOTICE: WN: be-fsstubs: lo_open: fd: 0
NOTICE: WN: be-fsstubs: lo_open: return: 0
CommitTransactionCommand
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 1
NOTICE: WN: be-fsstubs: cookies[0]=NULL!
StartTransactionCommand
NOTICE: WN: xact: StartTransaction
NOTICE: WN: be-fsstubs: lo_lseek, fd: 0
NOTICE: WN: be-fsstubs: lo_lseek: cookies[0]=0
ERROR: lo_lseek: invalid large obj descriptor (0)
AbortCurrentTransaction
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 0
proc_exit(0) [#0]
shmem_exit(0) [#0]
NOTICE: WN: xact: StartTransaction
pq_flush: send() failed: Bad file descriptor
NOTICE: WN: be-fsstubs: lo_commit, isCommit: 1
pq_flush: send() failed: Bad file descriptor
exit(0)
/usr/local/postgres/bin/postmaster: reaping dead processes...
/usr/local/postgres/bin/postmaster: CleanupProc: pid 13256 exited with status 0
pmdie 15
proc_exit(0) [#0]
shmem_exit(0) [#0]
exit(0)
* problems with large-objects
@ 1999-10-02 07:48 Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:06 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 13+ messages in thread
From: Wolfgang Hottgenroth @ 1999-10-02 07:48 UTC (permalink / raw)
To: pgsql-interfaces@hub.org
Hi,
did anyone of you ever saw some strange problems with large objects and
the postgresql 6.51?
Sometimes ago I want to switch from 6.4.2 to 6.5.1 and my tools, using
the large objects through the perl-library refused to work. lo_import
didn't work anymore.
I traced it down through the libpq, where I found that a test with
lo_lseek in the lo_open-function (file: fe-lobj.c) fails. Going deeper
into the source and drawing a trace of debug-output across some files I
switched tracing from the libpq to the backend.
Here I found in be-fsstubs.c, that after the function lo_open sets a
cookie using the function newLOfd, just this cookie will be cleared by
lo_commit, which is called by CommitTransaction (in xact.c). The handle,
handed out to the application by the library-function lo_open is now not
longer valid and the next access through this handle fails.
Did I understood something wrong in the source or is there a bug?
I attach the test-application (in Perl, named db_test), the debug-output
of the application (and the outputs in the libpq) (db_test2.log), the
debug-output of postmaster
(post2.log) and the files I patched with my debug-output (you will find
them easily because in every line I changed there is the string 'WN').
May be anyone of you, especially from the developing-guys, find time to
have a look into the attachments?
Thanky you,
Wolfgang
Attachments:
postgres_err.tar.gz (application/x-gzip, 23.0K) download
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
@ 1999-10-02 15:06 ` Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 15:32 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1 sibling, 1 reply; 13+ messages in thread
From: Thomas Lockhart @ 1999-10-02 15:06 UTC (permalink / raw)
To: woho@ieee.org; +Cc: pgsql-interfaces@hub.org
> did anyone of you ever saw some strange problems with large objects and
> the postgresql 6.51?
I don't work with Large Objects, but I am pretty sure that the
behavior was changed/fixed for v6.5 to *require* all large object
manipulation to happen within a transaction. This was always a
requirement, but Postgres did not enforce it and had some potentially
bad behavior lurking as a result.
- Thomas
--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:06 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
@ 1999-10-02 15:32 ` Wolfgang Hottgenroth <wn@kja-essen.de>
0 siblings, 0 replies; 13+ messages in thread
From: Wolfgang Hottgenroth @ 1999-10-02 15:32 UTC (permalink / raw)
To: Thomas Lockhart <lockhart@alumni.caltech.edu>; +Cc: woho@ieee.org; pgsql-interfaces@hub.org
Hi Thomas, hi all,
Thomas Lockhart wrote:
>
> > did anyone of you ever saw some strange problems with large objects and
> > the postgresql 6.51?
>
> I don't work with Large Objects, but I am pretty sure that the
> behavior was changed/fixed for v6.5 to *require* all large object
> manipulation to happen within a transaction. This was always a
> requirement, but Postgres did not enforce it and had some potentially
> bad behavior lurking as a result.
Have many thanks for that fast answer. It is new to me, that
large-object-operations requires transactions. Did I read the manual to
fast or isn't it written down there?
I will try it.
Wolfgang
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
@ 1999-10-02 15:33 ` Tom Lane <tgl@sss.pgh.pa.us>
1999-10-02 15:36 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-03 04:14 ` Re: [INTERFACES] problems with large-objects Douglas Thomson <dougt@mugc.cc.monash.edu.au>
1 sibling, 2 replies; 13+ messages in thread
From: Tom Lane @ 1999-10-02 15:33 UTC (permalink / raw)
To: woho@ieee.org; +Cc: pgsql-interfaces@hub.org
Wolfgang Hottgenroth <wn@kja-essen.de> writes:
> Sometimes ago I want to switch from 6.4.2 to 6.5.1 and my tools, using
> the large objects through the perl-library refused to work. lo_import
> didn't work anymore.
6.5.* only allows large objects to be kept open within a single
transaction; you have to wrap BEGIN...END around anything that involves
lo_open ... lo_read/write ... lo_close. In particular, since lo_import
and lo_export are just such sequences, they need to be called inside
a transaction.
This restriction has been documented as required all along, but older
Postgres releases didn't enforce it (they'd just fail once in a while
if you didn't do it :-().
It'd be nice if lo_import could start/finish a transaction for itself,
but I don't see any good way to do that without breaking things for
the case that you want to do several lo_imports in one transaction.
regards, tom lane
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
@ 1999-10-02 15:36 ` Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 16:08 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1 sibling, 1 reply; 13+ messages in thread
From: Wolfgang Hottgenroth @ 1999-10-02 15:36 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: woho@ieee.org; pgsql-interfaces@hub.org
Hi Tom,
thank you for that fast answer. So I failed when reading the manual,
because I never realized, that using transactions are a requirement when
working with LO's.
Wolfgang
Tom Lane wrote:
>
> Wolfgang Hottgenroth <wn@kja-essen.de> writes:
> > Sometimes ago I want to switch from 6.4.2 to 6.5.1 and my tools, using
> > the large objects through the perl-library refused to work. lo_import
> > didn't work anymore.
>
> 6.5.* only allows large objects to be kept open within a single
> transaction; you have to wrap BEGIN...END around anything that involves
> lo_open ... lo_read/write ... lo_close. In particular, since lo_import
> and lo_export are just such sequences, they need to be called inside
> a transaction.
>
> This restriction has been documented as required all along, but older
> Postgres releases didn't enforce it (they'd just fail once in a while
> if you didn't do it :-().
>
> It'd be nice if lo_import could start/finish a transaction for itself,
> but I don't see any good way to do that without breaking things for
> the case that you want to do several lo_imports in one transaction.
>
> regards, tom lane
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
1999-10-02 15:36 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
@ 1999-10-02 16:08 ` Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 16:13 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
0 siblings, 1 reply; 13+ messages in thread
From: Thomas Lockhart @ 1999-10-02 16:08 UTC (permalink / raw)
To: woho@ieee.org; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-interfaces@hub.org
> So I failed when reading the manual,
> because I never realized, that using transactions are a requirement when
> working with LO's.
> > This restriction has been documented as required all along,
Well, just because we think we've documented it doesn't mean that it
is obvious. Now that you know, do the docs you were reading make this
clear and you just missed it, or should it be mentioned in more places
than it is currently? Let us know if you have suggestions for
improvements...
- Thomas
--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
1999-10-02 15:36 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 16:08 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
@ 1999-10-02 16:13 ` Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 17:11 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
0 siblings, 1 reply; 13+ messages in thread
From: Wolfgang Hottgenroth @ 1999-10-02 16:13 UTC (permalink / raw)
To: Thomas Lockhart <lockhart@alumni.caltech.edu>; +Cc: woho@ieee.org, Tom Lane <tgl@sss.pgh.pa.us>; pgsql-interfaces@hub.org
Hi again,
Thomas Lockhart wrote:
>
> > So I failed when reading the manual,
> > because I never realized, that using transactions are a requirement when
> > working with LO's.
> > > This restriction has been documented as required all along,
>
> Well, just because we think we've documented it doesn't mean that it
> is obvious. Now that you know, do the docs you were reading make this
> clear and you just missed it, or should it be mentioned in more places
> than it is currently? Let us know if you have suggestions for
> improvements...
Of course I take an immediate look into the docs, and I found it: In the
sample program of the Large-Object-chapter (Chapter 15). Is this the
only place where it is documented? If this is the only place, then I
think, it should be mentioned in more places. Or can you tell me, where
I can read more about it?
Wolfgang
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
1999-10-02 15:36 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 16:08 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 16:13 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
@ 1999-10-02 17:11 ` Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-04 11:44 ` Re: [INTERFACES] problems with large-objects Peter Mount <peter@retep.org.uk>
0 siblings, 1 reply; 13+ messages in thread
From: Thomas Lockhart @ 1999-10-02 17:11 UTC (permalink / raw)
To: woho@ieee.org; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-interfaces@hub.org
> Of course I take an immediate look into the docs, and I found it: In the
> sample program of the Large-Object-chapter (Chapter 15). Is this the
> only place where it is documented? If this is the only place, then I
> think, it should be mentioned in more places. Or can you tell me, where
> I can read more about it?
I don't know. I just added a note to the top of the "Interfaces"
portion of that chapter which mentions this explicitly. Maybe that is
enough...
- Thomas
--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
1999-10-02 15:36 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 16:08 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 16:13 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 17:11 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
@ 1999-10-04 11:44 ` Peter Mount <peter@retep.org.uk>
1999-10-04 14:33 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
0 siblings, 1 reply; 13+ messages in thread
From: Peter Mount @ 1999-10-04 11:44 UTC (permalink / raw)
To: Thomas Lockhart <lockhart@alumni.caltech.edu>; +Cc: woho@ieee.org, Tom Lane <tgl@sss.pgh.pa.us>; pgsql-interfaces@hub.org
On Sat, 2 Oct 1999, Thomas Lockhart wrote:
> > Of course I take an immediate look into the docs, and I found it: In the
> > sample program of the Large-Object-chapter (Chapter 15). Is this the
> > only place where it is documented? If this is the only place, then I
> > think, it should be mentioned in more places. Or can you tell me, where
> > I can read more about it?
>
> I don't know. I just added a note to the top of the "Interfaces"
> portion of that chapter which mentions this explicitly. Maybe that is
> enough...
We need to add a reference to this in the JDBC docs, as I'm answering
quite a few "bug" reports where LargeObjects are not working.
For jdbc, you need to use the setAutoCommit() method to false.
ie:
Connection mycon;
...
mycon.setAutoCommit(false);
... now use Large Objects
I'm thinking of adding a more descriptive error message when LO's fail.
Peter
--
Peter T Mount peter@retep.org.uk
Main Homepage: http://www.retep.org.uk
PostgreSQL JDBC Faq: http://www.retep.org.uk/postgres
Java PDF Generator: http://www.retep.org.uk/pdf
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
1999-10-02 15:36 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 16:08 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 16:13 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 17:11 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-04 11:44 ` Re: [INTERFACES] problems with large-objects Peter Mount <peter@retep.org.uk>
@ 1999-10-04 14:33 ` Thomas Lockhart <lockhart@alumni.caltech.edu>
0 siblings, 0 replies; 13+ messages in thread
From: Thomas Lockhart @ 1999-10-04 14:33 UTC (permalink / raw)
To: Peter Mount <peter@retep.org.uk>; +Cc: woho@ieee.org, Tom Lane <tgl@sss.pgh.pa.us>; pgsql-interfaces@hub.org
> We need to add a reference to this in the JDBC docs, as I'm answering
> quite a few "bug" reports where LargeObjects are not working.
I'll add this in. I'm making slight changes to the markup, but am
reminded that *most* of the jdbc doc has not been marked up at all
yet. I wasn't sure how to handle the ascii class diagrams, and how
stable the doc is. If we go through the trouble of marking it up, can
you then make changes to it? Or did the text come from somewhere else,
where you maintain the original docos?
- Thomas
--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [INTERFACES] problems with large-objects
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
@ 1999-10-03 04:14 ` Douglas Thomson <dougt@mugc.cc.monash.edu.au>
1 sibling, 0 replies; 13+ messages in thread
From: Douglas Thomson @ 1999-10-03 04:14 UTC (permalink / raw)
To: tgl@sss.pgh.pa.us; +Cc: woho@ieee.org; pgsql-interfaces@hub.org
Tom Lane <tgl@sss.pgh.pa.us> writes:
> 6.5.* only allows large objects to be kept open within a single
> transaction; you have to wrap BEGIN...END around anything that involves
> lo_open ... lo_read/write ... lo_close. In particular, since lo_import
> and lo_export are just such sequences, they need to be called inside
> a transaction.
Note that the Perl interface to large objects is broken in both 6.4.2
and 6.5.1, in that a BLOB containing a NUL character will be truncated
at the first NUL when read back using the Perl lo_read method. I have
reported this problem both to this list and via various other channels
over the past few months, but I just checked the CVS repository and
there is still no fix there.
For what it is worth, in case anyone reading this wants to either fix
their own copies or better yet fix the main source repository, the
necessary patch is simply:
======cut=here========================================================
inline patch 1: perl5.fixed/Pg.xs Sun Oct 3 14:00:35 1999 (+3/-3 lines) download inline-1.patch
--- perl5/Pg.xs Sat Aug 14 12:46:35 1999
+++ perl5.fixed/Pg.xs Sun Oct 3 14:00:35 1999
@@ -1,6 +1,6 @@
/*-------------------------------------------------------
*
- * $Id: Pg.xs,v 1.12 1999/02/19 23:27:17 tgl Exp $
+ * $Id: Pg.xs,v 1.12 1999/02/19 23:27:17 tgl Exp $ (NUL bug patched)
*
* Copyright (c) 1997, 1998 Edmund Mergl
*
@@ -643,7 +643,7 @@
}
OUTPUT:
RETVAL
- buf
+ buf sv_setpvn((SV*)ST(2), buf, RETVAL);
int
lo_write(conn, fd, buf, len)
@@ -1029,7 +1029,7 @@
}
OUTPUT:
RETVAL
- buf
+ buf sv_setpvn((SV*)ST(2), buf, RETVAL);
int
======cut=here========================================================
* Problems with large objects
@ 1999-10-07 10:51 LAN ZHANG <zhang-lan@usa.net>
0 siblings, 0 replies; 13+ messages in thread
From: LAN ZHANG @ 1999-10-07 10:51 UTC (permalink / raw)
To: pgsql-interfaces
Hi,
I am using the new jdbc6.5-1.2 to store the large objects.
The code is the following:
LargeObjectManager lom =
((postgresql.Connection)con).getLargeObjectAPI();
System.out.println("creating blob to store image...");
oid1 = lom.create();
LargeObject blob = lom.open(oid1);
blob.write(buf,0,buf.length);
blob.close();
But when it execute the oid1=lom.create(), it gave me this error message:
FastPath protocol error: Z
The old jdbc6.5-1.2 works fine for this program.
-- Lan
____________________________________________________________________
Get free email and a permanent address at http://www.netaddress.com/?N=1
^ permalink raw reply [nested|flat] 13+ messages in thread
end of thread, other threads:[~1999-10-07 10:51 UTC | newest]
Thread overview: 13+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
1999-08-27 17:21 problems with large objects Wolfgang Hottgenroth <woho@ieee.org>
1999-10-02 07:48 problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:06 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 15:32 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 15:33 ` Re: [INTERFACES] problems with large-objects Tom Lane <tgl@sss.pgh.pa.us>
1999-10-02 15:36 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 16:08 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-02 16:13 ` Re: [INTERFACES] problems with large-objects Wolfgang Hottgenroth <wn@kja-essen.de>
1999-10-02 17:11 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-04 11:44 ` Re: [INTERFACES] problems with large-objects Peter Mount <peter@retep.org.uk>
1999-10-04 14:33 ` Re: [INTERFACES] problems with large-objects Thomas Lockhart <lockhart@alumni.caltech.edu>
1999-10-03 04:14 ` Re: [INTERFACES] problems with large-objects Douglas Thomson <dougt@mugc.cc.monash.edu.au>
1999-10-07 10:51 Problems with large objects LAN ZHANG <zhang-lan@usa.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox