public inbox for [email protected]  

help / color / mirror / Atom feed

From: Marc G. Fournier <[email protected]>
To: Josh Berkus <[email protected]>
Cc: John Hansen <[email protected]>
Cc: [email protected]
Cc: Jim C. Nasby <[email protected]>
Subject: Re: Infrastructure monitoring
Date: Fri, 13 Jan 2006 22:16:59 -0400 (AST)
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>

On Fri, 13 Jan 2006, Josh Berkus wrote:

> Jim,
>
>> Search has been down for at least 2 days now, and this certainly isn't
>> the first time it's happened. There's also been cases of archives
>> getting stuck, and probably other outages besides those that went on
>> until someone email'd about it.
>>
>> Would it be difficult to setup something to monitor these various
>> services? I know there's at least one OSS tool to do it, though I have
>> no idea how hard it would be to tie that into the current
>> infrastructure.
>
> We have an open offer of Hyperic licenses, and they support FreeBSD now.

Not to discount the offer ... but, what exactly would that provide us?  We 
already monitor the *servers*, its what is inside of the servers that 
needs better monitoring ... knowing nothing about Hyperic, does that 
provide something for that?

In the case of the archives, for instance, the problem was a perl process 
that for some unknown reason got stuck randomly ... removed that in favor 
of an awk script, and it hasn't done it since ... i also redirected cron's 
email to [email protected], so that any errors show up in my mailbox 
instead of roots, so I get an hourly reminder that things are running well 
...

In the case of search ... John would be better at answering that, but when 
he and I talked this past week, he mentioned that he was moving it all 
over to two new servers, which I changed the DNS for on Wednesday ...

As I've said above ... physical servers are being monitored, so if anyone 
has some ideas on how we can improve "content monitoring", for lack of a 
better word, I know I'm all ears ...

Again, if Hyperic can offer something for this, let me know ...

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: [email protected]           Yahoo!: yscrappy              ICQ: 7615664

view thread (33+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Infrastructure monitoring
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox