From: Mark Cotner <mcotner@yahoo.com>
To: PFC <lists@boutiquenumerique.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
To: Bob Ippolito <bob@redivi.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: sustained update load of 1-2k/sec
Date: Fri, 19 Aug 2005 19:02:35 -0400
Message-ID: <BF2BDD4B.EA1D%mcotner@yahoo.com> (raw)
In-Reply-To: <op.svrot4mxth1vuj@localhost>
:) Most of the ppl on this list are systems programmers, however I am not.
The tool of choice for this app is Ruby and the libraries don't support
async SNMP at the moment.
I've done a good deal of async snmp and the libraries that actually pull it
off generally aren't that good(Net-SNMP and Perl's Net::SNMP). Granted, UDP
is connectionless to an extent, but you still have to send the PDU, and bind
to the return socket and wait. If you batch the outgoing PDUs then you can
get away with sending them out synchronously and listening on the returning
socket synchronously, but it would require that your libraries support this.
I understand the concepts well enough, maybe I'll put together a patch. It
would be much lower overhead than managing all those threads. Looks like
it's gonna be a fun weekend.
Thanks again for all the great feedback.
'njoy,
Mark
On 8/19/05 2:11 PM, "PFC" <lists@boutiquenumerique.com> wrote:
>
>> While I agree that hundreds of threads seems like overkill, I think the
>> above advice might be going too far in the other direction. The problem
>> with single-threaded operation is that any delay affects the whole
>> system --- eg, if you're blocked waiting for disk I/O, the CPU doesn't
>
> You use UDP which is a connectionless protocol... then why use threads ?
>
> I'd advise this :
>
> Use asynchronous network code (one thread) to do your network stuff. This
> will lower the CPU used by this code immensely.
> Every minute, dump a file contianing everything to insert into the table.
> Use another thread to COPY it into the DB, in a temporary table if you
> wish, and then INSERT INTO ... SELECT.
> This should be well adapted to your requirements.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mcotner@yahoo.com, lists@boutiquenumerique.com, tgl@sss.pgh.pa.us, bob@redivi.com
Subject: Re: sustained update load of 1-2k/sec
In-Reply-To: <BF2BDD4B.EA1D%mcotner@yahoo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox