DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com;
	h=Received:User-Agent:Date:Subject:From:To:CC:Message-ID:In-Reply-To:Mime-version:Content-type:Content-transfer-encoding;
	b=1OToURhp9PxWy6HIdRVIavt+C59i+wcyRtNCFkXeA3GBrqajs+NcNQkPc5WoHrEYBhMYsOvQ+df5uJRtiAbGp/R3r85PBggLlTJ1J3pLMDCRLWcEPrfrqp7R8SoAFansDwd1yH7JGGY7BCyuIzYqXC91s00EtkHmK0VsKSr13r4=
	;
User-Agent: Microsoft-Entourage/11.1.0.040913
Date: Fri, 19 Aug 2005 19:02:35 -0400
Subject: Re: sustained update load of 1-2k/sec 
From: Mark Cotner <mcotner@yahoo.com>
To: PFC <lists@boutiquenumerique.com>, Tom Lane <tgl@sss.pgh.pa.us>,
	Bob Ippolito <bob@redivi.com>
Cc: <pgsql-performance@postgresql.org>
Message-ID: <BF2BDD4B.EA1D%mcotner@yahoo.com>
In-Reply-To: <op.svrot4mxth1vuj@localhost>
Mime-version: 1.0
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit

:)  Most of the ppl on this list are systems programmers, however I am not.
The tool of choice for this app is Ruby and the libraries don't support
async SNMP at the moment.

I've done a good deal of async snmp and the libraries that actually pull it
off generally aren't that good(Net-SNMP and Perl's Net::SNMP).  Granted, UDP
is connectionless to an extent, but you still have to send the PDU, and bind
to the return socket and wait.  If you batch the outgoing PDUs then you can
get away with sending them out synchronously and listening on the returning
socket synchronously, but it would require that your libraries support this.
I understand the concepts well enough, maybe I'll put together a patch.  It
would be much lower overhead than managing all those threads.  Looks like
it's gonna be a fun weekend.

Thanks again for all the great feedback.

'njoy,
Mark


On 8/19/05 2:11 PM, "PFC" <lists@boutiquenumerique.com> wrote:

> 
>> While I agree that hundreds of threads seems like overkill, I think the
>> above advice might be going too far in the other direction.  The problem
>> with single-threaded operation is that any delay affects the whole
>> system --- eg, if you're blocked waiting for disk I/O, the CPU doesn't
> 
> You use UDP which is a connectionless protocol... then why use threads ?
> 
> I'd advise this :
> 
> Use asynchronous network code (one thread) to do your network stuff. This
> will lower the CPU used by this code immensely.
> Every minute, dump a file contianing everything to insert into the table.
> Use another thread to COPY it into the DB, in a temporary table if you
> wish, and then INSERT INTO ... SELECT.
> This should be well adapted to your requirements.
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster