Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1X0Cr5-0002JL-GF for pgsql-admin@arkaria.postgresql.org; Thu, 26 Jun 2014 16:48:59 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1X0Cr4-00007G-SY for pgsql-admin@arkaria.postgresql.org; Thu, 26 Jun 2014 16:48:58 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) (Exim 4.80) (envelope-from ) id 1X0Cr4-000079-4v for pgsql-admin@postgresql.org; Thu, 26 Jun 2014 16:48:58 +0000 Received: from mail-yk0-x230.google.com ([2607:f8b0:4002:c07::230]) by magus.postgresql.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.80) (envelope-from ) id 1X0Cr1-0005i3-2Q for pgsql-admin@postgresql.org; Thu, 26 Jun 2014 16:48:57 +0000 Received: by mail-yk0-f176.google.com with SMTP id 131so2142911ykp.21 for ; Thu, 26 Jun 2014 09:48:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=FtYWVbVULEyprzDgLm5e2un/Yt14urnoXKl2lex3ug0=; b=GHEU1p69Bok/hx0m/ROwd4tlBV8a1h/7KSli6EkcsW3AdmgV8mP4pZDB6GBfDrg9Op xKRmY8UZ4/ybykXzGGz5R0rZrhi8JDZCPasvate/Pjp8ks57tmS6rJcT3g5QCKQImNgG +FHBi7l6saCBVV7LwCN8rrX4zYPjiaLtQHzSSVnTuouwzadzPHcToEcvt4Qku74qO2M9 5ioT9lu4uNWQD/SzZqhk726ulDyBxOxng7cXgIwzM13lHFUDBNnrMhlKHeyGCoupGQMf EViDLCyEZ1piP3Jrm6eFg7OXyOiFidt3PvaYr2JN6YNzy+s5H7v5+J6GCdjlPd4KuJZz zlfA== X-Received: by 10.236.108.176 with SMTP id q36mr23109315yhg.87.1403801331302; Thu, 26 Jun 2014 09:48:51 -0700 (PDT) Received: from [192.168.254.4] (h77.89.102.166.static.ip.windstream.net. [166.102.89.77]) by mx.google.com with ESMTPSA id o42sm10697729yhe.51.2014.06.26.09.48.50 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 26 Jun 2014 09:48:50 -0700 (PDT) Message-ID: <53AC4EEC.2000803@gmail.com> Date: Thu, 26 Jun 2014 12:48:44 -0400 From: John Scalia User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "pgsql-admin@postgresql.org" Subject: Re: Wal archive way behind in streaming replication References: <53A9B567.5070301@gmail.com> <53A9C139.2040704@gmail.com> <3709F6AF-032E-4095-B0CB-DC1CE36F6B4B@breakthroughfuel.com> In-Reply-To: <3709F6AF-032E-4095-B0CB-DC1CE36F6B4B@breakthroughfuel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Pg-Spam-Score: -2.0 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-admin Precedence: bulk Sender: pgsql-admin-owner@postgresql.org OK, starting over... I killed off both standby servers and I rebuilt my databases off of the primary server (10.10.1.128) using: pg_basebackup -h 10.10.1.128 -p 5432 -D /var/lib/pgsql/9.3/data -X s -c fast It took about 40 minutes to complete for each standby. Immediately after it completed, I copied my previously saved recovery.conf file into each's data directory and started the standby servers. I did not do anything to the files in the directory specified by the restore_command. Now, I would expect the primary to continue writing WAL segments into the directory while the pg_basebackup is running, but it apparently did not. Checking the primary's pg_xlog directory showed that it was now writing *DF, and each standby had only received up to *D5. That means each standby's WALs are about 1 hour behind. I would expect each to be at *DE as that should be the last completely written WAL file, and I would guess that the primary is still writing *DF. The archive process on the primary is now writing WAL segments to the directory in archive_command, as I just saw *D6 pop into it, but it's never going to catch up without help as it's only copying 1 file every 10 minutes as a new one gets generated. Am I missing a flag for pg_basebackup? Or should I just accept that the standby's could never catch up with WAL segments and just turn archiving off? -- Jay -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin