X-Original-To: pgsql-general@postgresql.org Received: from spampd.localdomain (postgresql.org [64.49.215.8]) by postgresql.org (Postfix) with ESMTP id 2E86C475F0D for ; Sat, 10 May 2003 13:08:35 -0400 (EDT) Received: from flake.decibel.org (flake.decibel.org [66.143.173.58]) by postgresql.org (Postfix) with SMTP id 4C200475CE5 for ; Sat, 10 May 2003 13:08:34 -0400 (EDT) Received: (qmail 19146 invoked by uid 1001); 10 May 2003 17:08:33 -0000 Date: Sat, 10 May 2003 12:08:33 -0500 From: "Jim C. Nasby" To: Adam Siegel Cc: pgsql-hackers@postgresql.org, pgsql-performance@postgresql.org, pgsql-general@postgresql.org Subject: Re: realtime data inserts Message-ID: <20030510120833.K66185@flake.decibel.org> Reply-To: jim@nasby.net References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from adam@sycamorehq.com on Sat, May 10, 2003 at 11:25:16AM -0400 X-Operating-System: FreeBSD 4.8-RELEASE i386 X-Distributed: Join the Effort! http://www.distributed.net X-Spam-Status: No, hits=-39.4 required=5.0 tests=BAYES_01,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,REPLY_WITH_QUOTES,SIGNATURE_SHORT_SPARSE, USER_AGENT_MUTT version=2.50 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.50 (1.173-2003-02-20-exp) X-Archive-Number: 200305/324 X-Sequence-Number: 41979 Are you binding your insert? IE: prepare statement INSERT INTO blah VALUES (?, ?, ?); execute statement (a, b, c) Instead of just "INSERT INTO blah VALUES(a, b, c)" On Sat, May 10, 2003 at 11:25:16AM -0400, Adam Siegel wrote: > I have realtime data flowing at a rate of 500, 512 byte packets per second. > I want to log the info in a database table with two other columns, one for a > timestamp and one for a name of the packet. The max rate I can achieve is > 350 inserts per second on a sun blade 2000. The inserts are grouped in a > transaction and I commit every 1200 records. I am storing the binary data > in a bytea. I am using the libpq conversion function. Not sure if that is > slowing me down. But I think it is the insert not the conversion. > > Any thoughts on how to achive this goal? -- Jim C. Nasby (aka Decibel!) jim@nasby.net Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?"