Message-ID: <491F1861.4060403@reedyriver.com>
Date: Sat, 15 Nov 2008 13:43:45 -0500
From: "H. Hall" <hhall1001@reedyriver.com>
User-Agent: Thunderbird 2.0.0.17 (Windows/20080914)
MIME-Version: 1.0
To: Mischa Sandberg <mischa_sandberg@telus.net>,
 pgsql-admin@postgresql.org
Subject: Re: Reliably determining whether the server came up
References: <1226527535.491b532f4c45e@legacywebmail.telus.net>
In-Reply-To: <1226527535.491b532f4c45e@legacywebmail.telus.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Mischa Sandberg wrote:
> I've been trying to work out a reliable script to determine, 
> after pg_ctl start, that the server is done attempting
> to come up, and that it has either succeeded OR FAILED.
> This is for several hundred unattended appliance-type servers,
> currently on PG 8.0 but soon to be on 8.3
>   

Why don't you try to create a connection to a db on each server?

HH
> Haven't found anything in the archives.
> I want to determine success/failure without time-outs, since:
> the db is restarted every time a server gets an upgrade,
> and it can get several upgrades in a batch, and the cpu/disk load 
> during an upgrade is highly variable; a restart with no
> recovery may still require as much as a minute to get to 'ready'.
>
> We also need to restart the server several hundred times
> in our in-house system tests.
>
> So pg_ctl -w start is not an option, even if the timeout were
> configurable to under a minute.
>
> The best I have that doesn't involve modifying pg_ctl is:
>
> # Hand-compute $NEXT_LOG from postgresql.conf
> # parameters (log_directory) and (log_filename).
> # Replace %S format with a '??' wildcard (yech).
>
> $ TEMP_LOG=/tmp/pg.$PGPORT.log
> $ touch $NEXT_LOG >$TEMP_LOG
> $ FROM=`awk 'END {print NR+1}' $NEXT_LOG`
> $ pg_ctl start -s -l $TEMPLOG
> $ while tail +$FROM $NEXT_LOG | ! egrep -hw
> 'FATAL|PANIC|DETAIL|ready|shutting|^postmaster cannot' $TEMP_LOG -; do
> sleep 1; done
>
> The nasty cases are when the server fails (exits)
> without being able to create its std log file (e.g. 
> error in postgresql.conf). 
>
> So I'm down to patching start_postmaster in pg_ctl.c 
> to use popen("... & echo $!") instead of system("... &"),
> then make test_postmaster_connection do a kill(0,pid)
> if PQsetdbLogin fails.
>
> Any suggestions appreciated.
>