Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1YTR6d-0007SV-TC for pgsql-general@arkaria.postgresql.org; Thu, 05 Mar 2015 08:26:08 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1YTR6c-0000by-OG for pgsql-general@arkaria.postgresql.org; Thu, 05 Mar 2015 08:26:06 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) (Exim 4.80) (envelope-from ) id 1YTR6b-0000bm-LE for pgsql-general@postgresql.org; Thu, 05 Mar 2015 08:26:05 +0000 Received: from mail-bn1on0678.outbound.protection.outlook.com ([2a01:111:f400:fc10::678] helo=na01-bn1-obe.outbound.protection.outlook.com) by makus.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1YTR6X-0007Hx-Qb for pgsql-general@postgresql.org; Thu, 05 Mar 2015 08:26:03 +0000 Received: from decina.attlocal.net (2602:306:ce8b:3460:75e0:5448:6acd:4893) by BY1PR11MB0344.namprd11.prod.outlook.com (25.160.206.16) with Microsoft SMTP Server (TLS) id 15.1.99.14; Thu, 5 Mar 2015 08:25:52 +0000 Message-ID: <54F81306.9060404@BlueTreble.com> Date: Thu, 5 Mar 2015 02:25:42 -0600 From: Jim Nasby User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Sergey Shchukin , Radovan Jablonovsky , Subject: Re: Re: [pgadmin-support] Issue with a hanging apply process on the replica db after vacuum works on primary References: <54EC7FAE.8050003@gmail.com> <54EEBC5B.5000402@gmail.com> <54F03052.6050900@BlueTreble.com> <54F050D2.7030406@gmail.com> In-Reply-To: <54F050D2.7030406@gmail.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [2602:306:ce8b:3460:75e0:5448:6acd:4893] X-ClientProxiedBy: DM2PR07CA0037.namprd07.prod.outlook.com (10.141.52.165) To BY1PR11MB0344.namprd11.prod.outlook.com (25.160.206.16) Authentication-Results: postgresql.org; dkim=none (message not signed) header.d=none; X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR11MB0344; X-Microsoft-Antispam-PRVS: X-Forefront-Antispam-Report: BMV:0; SFV:NSPM; SFS:(10009020)(6009001)(479174004)(51704005)(377454003)(24454002)(50986999)(40100003)(107886001)(86362001)(77156002)(46102003)(87976001)(76176999)(65816999)(33656002)(87266999)(54356999)(83506001)(19580395003)(80316001)(50466002)(23676002)(59896002)(2950100001)(92566002)(53416004)(65956001)(62966003)(42186005)(47776003)(65806001)(15975445007)(36756003)(3826002)(85282002)(18886065003); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR11MB0344; H:decina.attlocal.net; FPR:; SPF:None; MLV:sfv; LANG:en; X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(5005006)(5001007); SRVR:BY1PR11MB0344; BCL:0; PCL:0; RULEID:; SRVR:BY1PR11MB0344; X-Forefront-PRVS: 05066DEDBB X-OriginatorOrg: bluetreble.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Mar 2015 08:25:52.6904 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR11MB0344 X-Pg-Spam-Score: -1.9 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-general Precedence: bulk Sender: pgsql-general-owner@postgresql.org On 2/27/15 5:11 AM, Sergey Shchukin wrote: > > show max_standby_streaming_delay; > max_standby_streaming_delay > ----------------------------- > 30s We both need to be more clear about which server we're talking about (master or replica). What are max_standby_streaming_delay and max_standby_archive_delay set to *on the replica*? My hope is that one or both of those is set to somewhere around 8 minutes on the replica. That would explain everything. If that's not the case then I suspect what's happening is there's something running on the replica that isn't checking for interrupts frequently enough. That would also explain it. When replication hangs, is the replication process using a lot of CPU? Or is it just sitting there? What's the process status for the replay process show? Can you get a trace of the replay process on the replica when this is happening to see where it's spending all it's time? How are you generating these log lines? Tue Feb 24 15:05:07 MSK 2015 Stream: MASTER-masterdb:79607161592048 SLAVE:79607161550576 Replay:79607160986064 :: REPLAY 592 KBytes (00:00:00.398376 seconds) Do you see the confl_* fields in pg_stat_database_conflicts on the *replica* increasing? -- Jim Nasby, Data Architect, Blue Treble Consulting Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general