pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Mark Harrison <mh@pixar.com>
To: pgsql-admin@postgresql.org
Subject: pg-watchdog
Date: Thu, 05 Feb 2004 13:53:02 -0800
Message-ID: <4022BB3E.4040802@pixar.com> (raw)

Here is the final version of my script that will watch for disconnected
client machines and kill their associated server side processes.

If I'm doing anything really stupid I would appreciate hearing about it.
If it's useful to anyone, feel free to pass it along.

Thanks,
Mark


#!/bin/sh
# the next line restarts using wish \
exec tclsh8.3 "$0" "$@"

# pg-watchdog -- make sure postgres clients are reachable.
#
# This program is a watchdog for postgresql.  It monitors all
# postgresql client processes and kills them if the host upon
# which they are running becomes inaccessible over the network.
#
# This is necessary because if a machine goes down "hard" or
# if a network cable becomes disconnected, it can be several
# hours before postgres will be informed of this (for details,
# see chapter 23 of W. Richard Steven's TCP/IP Illustrated,
# volume 1).  If a postgres client on that machine has a
# transaction lock, all other clients will be blocked waiting
# for that lock to be released.
#
# This program uses the Scotty package available at:
#     http://wwwhome.cs.utwente.nl/~schoenw/scotty
# If you do not wish to install Scotty you can change
# the icmp command to instead use whatever flavor of
# ping is available on your system.
#
# Please send feedback, improvements, etc, to
# Mark Harrison, mh@pixar.com

package require Tnm

proc doit {} {
     after 60000 doit
     set pids("") ""
     unset pids("")
     set lines [exec ps auxww | awk {$11=="postgres:" {print $2 "/" $14}}]
     foreach s [split $lines \n] {
         set x [split $s /]
         set pid [lindex $x 0]
         set ip  [lindex $x 1]
         if {[regexp {[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+} $ip]} {
             lappend pids($ip) $pid
         }
     }
     if {[array size pids] > 0} {
         set times [icmp -timeout 20 echo [lsort [array names pids]]]
         foreach pair $times {
             set ip [lindex $pair 0]
             set time  [lindex $pair 1]
             if {$time == -1} {
                 foreach pid $pids($ip) {
                     puts "killing $pid from $ip"
                     exec kill $pid
                 }
             }
         }
     }
}

after 0 doit
vwait forever


-- 
Mark Harrison
Pixar Animation Studios




Message-ID: <4022BB3E.4040802@pixar.com>
Permalink:  ../4022BB3E.4040802@pixar.com/
Also on:    postgresql.org/message-id/4022BB3E.4040802@pixar.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: mh@pixar.com
  Subject: Re: pg-watchdog
  In-Reply-To: <4022BB3E.4040802@pixar.com>
  References: <4022BB3E.4040802@pixar.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox