Received: from localhost (unknown [200.46.204.183]) by postgresql.org (Postfix) with ESMTP id A369965010B for ; Thu, 30 Oct 2008 13:59:02 -0300 (ADT) Received: from postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 56636-02 for ; Thu, 30 Oct 2008 13:58:59 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.170]) by postgresql.org (Postfix) with ESMTP id AD10E64FDF3 for ; Thu, 30 Oct 2008 13:58:58 -0300 (ADT) Received: by ug-out-1314.google.com with SMTP id k40so1025755ugc.7 for ; Thu, 30 Oct 2008 09:58:56 -0700 (PDT) Received: by 10.66.233.14 with SMTP id f14mr1893583ugh.12.1225385935429; Thu, 30 Oct 2008 09:58:55 -0700 (PDT) Received: by 10.66.240.6 with HTTP; Thu, 30 Oct 2008 09:58:55 -0700 (PDT) Message-ID: <9c64d57f0810300958l7f3465e8nb4724ad291fa4d47@mail.gmail.com> Date: Thu, 30 Oct 2008 17:58:55 +0100 From: "Ori Garin" To: pgsql-admin@postgresql.org Subject: Fwd: Problems waking up from a warm standby In-Reply-To: <9c64d57f0810290938m246e6862of8658bbd3e543309@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5845_19493506.1225385935421" References: <9c64d57f0810290938m246e6862of8658bbd3e543309@mail.gmail.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.001 tagged_above=0 required=5 tests=HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 200810/185 X-Sequence-Number: 30641 ------=_Part_5845_19493506.1225385935421 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi. I have a problem with a standby server running on Windows 2003 R2, Enterprise x64 edition. I use Postgres 8.3 (installed to C:\Program Files (x86)\ ) Everything was working fine (base backup, archiving, recovery), until I wanted to test failover. To do that I create a trigger file, the recovery command returns a nonzero code, and I get something like: 2008-10-29 16:39:23 CET LOG: restored log file "0000000100000011000000CF" from archive 2008-10-29 16:42:25 CET LOG: restored log file "0000000100000011000000D0" from archive 2008-10-29 16:45:56 CET LOG: restored log file "0000000100000011000000D1" from archive 2008-10-29 16:49:02 CET LOG: restored log file "0000000100000011000000D2" from archive 2008-10-29 16:58:45 CET LOG: could not open file "pg_xlog/0000000100000011000000D3" (log file 17, segment 211): No such file or directory Now postgres seems to be stuck, or dead. Using Process Explorer I see that one of the postgres.exe processes is running drwtsn32.exe (Dr Watson Postmortem debugger) indefinitely. If I kill drwtsn32, postgres dies too: 2008-10-29 16:58:45 CET LOG: could not open file "pg_xlog/0000000100000011000000D3" (log file 17, segment 211): No such file or directory 2008-10-29 17:20:27 CET LOG: startup process (PID 2952) was terminated by exception 0xC000000D 2008-10-29 17:20:27 CET HINT: See C include file "ntstatus.h" for a description of the hexadecimal value. 2008-10-29 17:20:27 CET LOG: aborting startup due to startup process failure At first I was thinking that the message "could not open file..." is the problem, cause I figured postgres must think that the recovery command succeeded, otherwise it wouldn't try to use the new WAL file, but I'm not sure anymore if that's true. As the recovery command, I've used my own perl script (compiled), switched to pg_standby, and then reverted back to the script. Nothing works, even killing the restore_command process causes the same problem. I just noticed an Application Error in the Event Log: Event Type: Error Event Source: Application Error Event Category: (100) Event ID: 1000 Date: 29-10-2008 Time: 16:58:47 User: N/A Computer: S1217 Description: Faulting application postgres.exe, version 8.3.1.876, faulting module msvcr80.dll, version 8.0.50727.762, fault address 0x0001e879. For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp. Data: 0000: 41 70 70 6c 69 63 61 74 Applicat 0008: 69 6f 6e 20 46 61 69 6c ion Fail 0010: 75 72 65 20 20 70 6f 73 ure pos 0018: 74 67 72 65 73 2e 65 78 tgres.ex 0020: 65 20 38 2e 33 2e 31 2e e 8.3.1. 0028: 38 37 36 20 69 6e 20 6d 876 in m 0030: 73 76 63 72 38 30 2e 64 svcr80.d 0038: 6c 6c 20 38 2e 30 2e 35 ll 8.0.5 0040: 30 37 32 37 2e 37 36 32 0727.762 0048: 20 61 74 20 6f 66 66 73 at offs 0050: 65 74 20 30 30 30 31 65 et 0001e 0058: 38 37 39 879 Does anyone has any ideas about this?? Thanks in advance! Ori ------=_Part_5845_19493506.1225385935421 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
Hi.

I have a problem with a standby server running on Windows 2003 R2, Enterprise x64 edition. I use Postgres 8.3 (installed to C:\Program Files (x86)\ )
Everything was working fine (base backup, archiving, recovery), until I wanted to test failover.

To do that I create a trigger file, the recovery command returns a nonzero code, and I get something like:

2008-10-29 16:39:23 CET LOG:  restored log file "0000000100000011000000CF" from archive
2008-10-29 16:42:25 CET LOG:  restored log file "0000000100000011000000D0" from archive
2008-10-29 16:45:56 CET LOG:  restored log file "0000000100000011000000D1" from archive
2008-10-29 16:49:02 CET LOG:  restored log file "0000000100000011000000D2" from archive
2008-10-29 16:58:45 CET LOG:  could not open file "pg_xlog/0000000100000011000000D3" (log file 17, segment 211): No such file or directory

Now postgres seems to be stuck, or dead. Using Process Explorer I see that one of the postgres.exe processes is running drwtsn32.exe (Dr Watson Postmortem debugger) indefinitely. If I kill drwtsn32, postgres dies too:

2008-10-29 16:58:45 CET LOG:  could not open file "pg_xlog/0000000100000011000000D3" (log file 17, segment 211): No such file or directory
2008-10-29 17:20:27 CET LOG:  startup process (PID 2952) was terminated by exception 0xC000000D
2008-10-29 17:20:27 CET HINT:  See C include file "ntstatus.h" for a description of the hexadecimal value.
2008-10-29 17:20:27 CET LOG:  aborting startup due to startup process failure

At first I was thinking that the message "could not open file..." is the problem, cause I figured postgres must think that the recovery command succeeded, otherwise it wouldn't try to use the new WAL file, but I'm not sure anymore if that's true.
As the recovery command, I've used my own perl script (compiled), switched to pg_standby, and then reverted back to the script. Nothing works, even killing the restore_command process causes the same problem.

I just noticed an Application Error in the Event Log:

Event Type:    Error
Event Source:    Application Error
Event Category:    (100)
Event ID:    1000
Date:        29-10-2008
Time:        16:58:47
User:        N/A
Computer:    S1217
Description:
Faulting application postgres.exe, version 8.3.1.876, faulting module msvcr80.dll, version 8.0.50727.762, fault address 0x0001e879.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74   Applicat
0008: 69 6f 6e 20 46 61 69 6c   ion Fail
0010: 75 72 65 20 20 70 6f 73   ure  pos
0018: 74 67 72 65 73 2e 65 78   tgres.ex
0020: 65 20 38 2e 33 2e 31 2e   e 8.3.1.
0028: 38 37 36 20 69 6e 20 6d   876 in m
0030: 73 76 63 72 38 30 2e 64   svcr80.d
0038: 6c 6c 20 38 2e 30 2e 35   ll 8.0.5
0040: 30 37 32 37 2e 37 36 32   0727.762
0048: 20 61 74 20 6f 66 66 73    at offs
0050: 65 74 20 30 30 30 31 65   et 0001e
0058: 38 37 39                  879    


Does anyone has any ideas about this??
Thanks in advance!

Ori

------=_Part_5845_19493506.1225385935421--