X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id C1133EAED89 for ; Fri, 22 Oct 2004 18:31:20 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 97130-03 for ; Fri, 22 Oct 2004 17:31:17 +0000 (GMT) Received: from cmailm2.svr.pol.co.uk (cmailm2.svr.pol.co.uk [195.92.193.210]) by svr1.postgresql.org (Postfix) with ESMTP id 276CAEADBBD for ; Fri, 22 Oct 2004 18:31:07 +0100 (BST) Received: from modem-558.snake.dialup.pol.co.uk ([62.137.114.46] helo=[192.168.0.102]) by cmailm2.svr.pol.co.uk with esmtp (Exim 4.41) id 1CL3G3-0001Tx-K5; Fri, 22 Oct 2004 18:31:20 +0100 Subject: Re: replication using WAL archives From: Simon Riggs To: Gaetano Mendola Cc: Robert Treat , pgsql-admin@postgresql.org, iain@mst.co.jp In-Reply-To: <4179390B.10700@bigfoot.com> References: <002801c4b739$68450870$7201a8c0@mst1x5r347kymb> <1098384082.15573.14.camel@camel> <41782EEF.5040708@bigfoot.com> <011a01c4b818$b7370a20$06e887d9@Nightingale> <4179390B.10700@bigfoot.com> Content-Type: text/plain Organization: 2nd Quadrant Message-Id: <1098466150.20926.13.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Fri, 22 Oct 2004 18:29:10 +0100 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200410/310 X-Sequence-Number: 15303 On Fri, 2004-10-22 at 17:44, Gaetano Mendola wrote: > | Gaetano - skim-reading your script, how do you handle the situation when a > | new xlog file has been written within 10 seconds? That way the current file > | number will have jumped by 2, so when your script looks for the "Last wal" > | using head -1 it will find the N+2 and the intermediate file will never be > | copied. Looks like a problem to me... > > > Yes, the only window failure I seen ( but I don't know if it's possible ) > > Master: > ~ log N created > log N filled > archive log N > log N+1 created > log N+1 filled > ~ log N+2 created > ~ <---- the master die here before to archive the log N+1 > ~ archive log N+1 > > > in this case as you underline tha last log archived is the N and the N+2 > partial wal is added to archived wal collection and in the recovery fase > the recovery stop after processing the log N. > > Is it possible that the postmaster create the N+2 file without finish to archive > the N+1 ? ( I suspect yes :-( ) > > The only cure I see here is to look for not archived WAL ( if possible ). > Hmm...well you aren't looking for archived wal, you're just looking at wal...which is a different thing... Situation I thought I saw was: - copy away current partial filled xlog N - xlog N fills, N+1 starts - xlog N+1 fills, N+2 starts - copy away current partial filled xlog: N+2 (+10 secs later) i.e. if time to fill xlog (is ever) < time to copy away current xlog, then you miss one. So problem: you can miss one and never know you've missed one until the recovery can't find it, which it never returns from...so it just hangs. [Just so we're all clear: we're talking about Gaetano's script, not the PostgreSQL archver. The postgresql archiver doesn't do it that way, so it never misses one.] -- Best Regards, Simon Riggs