Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b8IUT-00060h-Gd for pgsql-admin@arkaria.postgresql.org; Thu, 02 Jun 2016 02:36:09 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1b8IUS-0000xw-KC for pgsql-admin@arkaria.postgresql.org; Thu, 02 Jun 2016 02:36:08 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1b8IUR-0000xd-Oy for pgsql-admin@postgresql.org; Thu, 02 Jun 2016 02:36:08 +0000 Received: from mail-pa0-x230.google.com ([2607:f8b0:400e:c03::230]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1b8IUK-0002bp-4Z for pgsql-admin@postgresql.org; Thu, 02 Jun 2016 02:36:06 +0000 Received: by mail-pa0-x230.google.com with SMTP id bz2so1740324pad.1 for ; Wed, 01 Jun 2016 19:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=iZqY61juSqjtVCDCn6V4oNU/knK219TY+/tIhVihTY8=; b=J7FuW0jrB5D5/uR8DP6lohqq8UtWeFKGzarGBZ3A6jRTYOcFCDZK4bSYKGLaKBN2Hj 9I/Avz7LP1oY9gu1MAG4Xja1gg7TPUh4OAVX7SJP8UjHNZfnW5VUC8EFe54eyGhFgcGL NeZJCuFEMVm5jBN+Z0KoUha9T9IfYRhklXdJw2hbLlSRcVSqnsf+vQmqOmfVJgLRFGt9 2ahBpgnswK+zH8ist3wyLh8vKKc2tLtTbr2ggauL3KfcRZ4g+SCPMjqwRHqcvjhg72X6 6fctSVOzUhtvwebM8/M7taWRkH/0BbObJWvE1fUBXtvA4tuijGcWZedYjXVNC+R9rmtc xhbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=iZqY61juSqjtVCDCn6V4oNU/knK219TY+/tIhVihTY8=; b=XKuKYipGVSjmjNdk/N0Ip+bdRioRq3jleusheqm5I/C+j6cpwd5amZCVMmFtqeFI1l /OGApEyDT1IIj9sCN/L33V3MB7dekgFV8LBMlexj322EU8rVvjsHQrGAhRKC48FP067H wMWW01/sKA75mOdTeeg5KVrtHjMV6lJj4g5oxVz1QVkgFEJPzUX8C0Kq2MdJDCALJgVW l/csnvDS73RGRG1hES/9pUGVUnqpNWHNDIEp9wXH1oaH5/R+igaaN7RCoWtOil0hczaR iVg/pnXCjpQttI8DXn5lNRzkx1d/yX/jCwqD1+3a5a9M3XvdnOE0l0gYOsKhe+iNwSpv dgfQ== X-Gm-Message-State: ALyK8tJ5aKwwOwi6cT942n/QoqmEYh8kAQRZRNV22oCl/UqYzXCkxcvkjxiWygqxlTpg5/lGs6slSLjzcnoBCnFefq8zan/qPmwtjEpGa/OfT9DopbQ5FGmqE6AgNnCFX053c7qHCeSRln+CD+cXEVc05mPpxgd5VWfQcH0QO66/3DV1k8FCSIw/FNcpEw== X-Received: by 10.66.43.232 with SMTP id z8mr2136048pal.17.1464834958612; Wed, 01 Jun 2016 19:35:58 -0700 (PDT) Received: from [192.168.1.2] (p1013045-ipngn12301marunouchi.tokyo.ocn.ne.jp. [153.201.44.45]) by smtp.gmail.com with ESMTPSA id z26sm22568192pff.0.2016.06.01.19.35.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Jun 2016 19:35:58 -0700 (PDT) Subject: Re: WAL segment NOT FOUND - Postgres 9.2 To: Patrick Baker , pgsql-admin@postgresql.org References: From: Ian Barwick Message-ID: <574F9B8B.9030000@2ndquadrant.com> Date: Thu, 2 Jun 2016 11:35:55 +0900 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-admin Precedence: bulk Sender: pgsql-admin-owner@postgresql.org On 06/02/2016 10:49 AM, Patrick Baker wrote: > Hi guys, > > > I got one master and two slaves. > > - slave01 = replication from master - working fine > - slave02 = server is configured to delay for 345600 seconds (4 days). > > The issue is on the slave02 server > > *Recovery.conf:* > > restore_command = 'exec nice -n 19 ionice -c 2 -n 7 ../../bin/segment.sh > "../wal_archive/%f" "%p" 345600' > > At about 10 days ago, I had an issue with my MASTER, and the server had to > be restarted. > > *PROBLEM:* > > The replication on the slave02 server is not working. When looking the > logs, I see: > > WAL segment `../wal_archive/00000002000011E800000012` not found > > I tried to copy manually that file, from my backups, then I get another > error: > > Archived WAL segment `../wal_archive/00000002000011E800000012` is > newer than the configured delay (345600 seconds) This message is presumably generated by your "segment.sh" script. I'd guess the file you copied is ending up with a very recent timestamp which is causing it to be rejected by the script. > So it seems the database hasn't been replicated since MAY 17: > > *on slave02 server:* > > ls -la /var/lib/pgsql/9.2/wal_archive: > > postgres postgres 16777216 May 17 18:16 00000002000011E80000000F > postgres postgres 16777216 May 17 18:18 00000002000011E800000010 > postgres postgres 16777216 May 17 18:18 00000002000011E800000011 Guessing again - you want to be shipping the logs to the standby, but for some reason they're not arriving in that directory. Is the archive_command on the master working correctly? Are there any errors in the log on the master? Regards Ian Barwick -- Ian Barwick http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin