Received: from localhost (unknown [200.46.204.183]) by mail.postgresql.org (Postfix) with ESMTP id 658BE633072 for ; Tue, 17 Feb 2009 14:13:52 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 46919-07 for ; Tue, 17 Feb 2009 14:13:46 -0400 (AST) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from yx-out-2324.google.com (yx-out-2324.google.com [74.125.44.30]) by mail.postgresql.org (Postfix) with ESMTP id A0780633168 for ; Tue, 17 Feb 2009 14:13:42 -0400 (AST) Received: by yx-out-2324.google.com with SMTP id 3so1141869yxj.73 for ; Tue, 17 Feb 2009 10:13:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=d74a7PTNML4yR4XqhrDr9/sXttRPXEWOi/PVtUymSg8=; b=vK5hKyPE+JJKLD1yCAHOnkY+hArgeB1oOmJ9/DRhhUXpKPMw+60xGlNzEBE+W/kzN4 ubavcdpJd/JegZmp8qrVGfSa2QyUAbRGI13Yn1n2tbj0R1NVPAXmiUcwmC50yMaHAgId pwHzVANKsFwgLcnIU5VwEPkypA8/FfX4PJYjA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=FJDeS/bIThJaK3gXfWqLdaEy8xTs2M7ma05KOmfO8arJABQCNlnHEbrkR5IOLayaPU hlVYdWNyw5+FGbC/OSL170QLe+GGq/YMsW6dpy7yQan/mkNlOPlbzcgX1fdzoXXdDlWy U10MFyLkJvOg3lZo/1Rg9dY+0mtkqKSFQaNYE= MIME-Version: 1.0 Received: by 10.142.113.17 with SMTP id l17mr61591wfc.293.1234894420851; Tue, 17 Feb 2009 10:13:40 -0800 (PST) In-Reply-To: <20090217070403.GB2486@cooker> References: <20090217070403.GB2486@cooker> Date: Tue, 17 Feb 2009 10:13:40 -0800 Message-ID: <1ca1c1410902171013q478f2620n13fcd68cb8b3d911@mail.gmail.com> Subject: Re: TCP network cost From: Aaron Turner To: pgsql-performance Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests=none X-Spam-Level: X-Archive-Number: 200902/197 X-Sequence-Number: 32778 On Mon, Feb 16, 2009 at 11:04 PM, Ross J. Reedstrom wrote: > Recently I've been working on improving the performance of a system that > delivers files stored in postgresql as bytea data. I was surprised at > just how much a penalty I find moving from a domain socket connection to > a TCP connection, even localhost. For one particular 40MB file (nothing > outragous) I see ~ 2.5 sec. to download w/ the domain socket, but ~ 45 sec > for a TCP connection (either localhost, name of localhost, or from > another machine 5 hops away (on campus - gigabit LAN) Similar numbers > for 8.2.3 or 8.3.6 (on Linux/Debian etch + backports) > > So, why the 20 fold penalty for using TCP? Any clues on how to trace > what's up in the network IO stack? TCP has additional overhead as well as going through the IP stack which for non-tuned Linux kernels is pretty limiting. long story short, there are things in /proc you can use to increase buffers and window sizes which will help with large TCP streams (like a 40MB file for example). There's a lot of documentation on the net for how to tune the Linux IP stack so I won't repeat it here. Now, having your DB box 5 hops away is going to add a lot of latency and any packet loss is going to kill TCP throughput- especially if you increase window sizes. I'd recommend something like "mtr" to map the network traffic (make sure you run it both ways in case you have an asymmetric routing situation) for a long period of time to look for hiccups. -- Aaron Turner http://synfin.net/ http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety. -- Benjamin Franklin