From: Tom Lane <tgl@sss.pgh.pa.us>
To: Ross J. Reedstrom <reedstrm@rice.edu>
Cc: pgsql-performance@postgresql.org
Subject: Re: TCP network cost
Date: Mon, 23 Feb 2009 16:17:00 -0500
Message-ID: <27042.1235423820@sss.pgh.pa.us> (raw)
In-Reply-To: <20090223194212.GA14802@cooker>
References: <20090217070403.GB2486@cooker>
<1ca1c1410902171013q478f2620n13fcd68cb8b3d911@mail.gmail.com>
<20090217070403.GB2486@cooker>
<3FAF940B-9B0B-4DBA-8728-6096022129DF@infogears.com>
<alpine.DEB.1.10.0902170032440.25606@asgard.lang.hm>
<20090217070403.GB2486@cooker>
<3FAF940B-9B0B-4DBA-8728-6096022129DF@infogears.com>
<20090217200416.GA23707@cooker>
<87hc2r272g.fsf@oxford.xeocode.com>
<20090223194212.GA14802@cooker>
"Ross J. Reedstrom" <reedstrm@rice.edu> writes:
> Summary: C client and large-object API python both send bits in
> reasonable time, but I suspect there's still room for improvement in
> libpq over TCP: I'm suspicious of the 6x difference. Detailed analysis
> will probably find it's all down to memory allocation and extra copying
> of bits around (client side)
I wonder if the backend isn't contributing to the problem too. It chops
its sends up into 8K units, which doesn't seem to create huge overhead
in my environment but maybe it does in yours. It'd be interesting to see
what results you get from the attached quick-and-dirty patch (against
HEAD, but it should apply back to at least 8.1).
regards, tom lane
inline patch 1: src/backend/libpq/pqcomm.c (+28/-9 lines) download inline-1.patchIndex: src/backend/libpq/pqcomm.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/libpq/pqcomm.c,v
retrieving revision 1.199
diff -c -r1.199 pqcomm.c
*** src/backend/libpq/pqcomm.c 1 Jan 2009 17:23:42 -0000 1.199
--- src/backend/libpq/pqcomm.c 23 Feb 2009 21:09:45 -0000
***************
*** 124,129 ****
--- 124,130 ----
static void pq_close(int code, Datum arg);
static int internal_putbytes(const char *s, size_t len);
static int internal_flush(void);
+ static int internal_send(const char *bufptr, size_t len);
#ifdef HAVE_UNIX_SOCKETS
static int Lock_AF_UNIX(unsigned short portNumber, char *unixSocketName);
***************
*** 1041,1046 ****
--- 1042,1056 ----
if (PqSendPointer >= PQ_BUFFER_SIZE)
if (internal_flush())
return EOF;
+ + /*+ * If buffer is empty and we'd fill it, just push the data immediately+ * rather than copying it into PqSendBuffer.+ */+ if (PqSendPointer == 0 && len >= PQ_BUFFER_SIZE)+ return internal_send(s, len);+ + /* Else put (some of) the data into the buffer */
amount = PQ_BUFFER_SIZE - PqSendPointer;
if (amount > len)
amount = len;
***************
*** 1075,1090 ****
static int
internal_flush(void)
{
static int last_reported_send_errno = 0;
! char *bufptr = PqSendBuffer;
! char *bufend = PqSendBuffer + PqSendPointer;
while (bufptr < bufend)
{
int r;
! r = secure_write(MyProcPort, bufptr, bufend - bufptr);
if (r <= 0)
{
--- 1085,1115 ----
static int
internal_flush(void)
{
+ int r;+ + r = internal_send(PqSendBuffer, PqSendPointer);+ + /*+ * On error, we drop the buffered data anyway so that processing can+ * continue, even though we'll probably quit soon.+ */+ PqSendPointer = 0;+ + return r;+ }+ + static int+ internal_send(const char *bufptr, size_t len)+ {
static int last_reported_send_errno = 0;
! const char *bufend = bufptr + len;
while (bufptr < bufend)
{
int r;
! r = secure_write(MyProcPort, (void *) bufptr, bufend - bufptr);
if (r <= 0)
{
***************
*** 1108,1118 ****
errmsg("could not send data to client: %m")));
}
- /*- * We drop the buffered data anyway so that processing can- * continue, even though we'll probably quit soon.- */- PqSendPointer = 0;
return EOF;
}
--- 1133,1138 ----
***************
*** 1120,1126 ****
bufptr += r;
}
- PqSendPointer = 0;
return 0;
}
--- 1140,1145 ----
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: tgl@sss.pgh.pa.us, reedstrm@rice.edu
Subject: Re: TCP network cost
In-Reply-To: <27042.1235423820@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox