Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cA03a-0000ks-Ll for pgsql-admin@arkaria.postgresql.org; Thu, 24 Nov 2016 19:51:42 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1cA03Y-00018o-9U for pgsql-admin@arkaria.postgresql.org; Thu, 24 Nov 2016 19:51:40 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1cA03X-00018h-QE for pgsql-admin@postgresql.org; Thu, 24 Nov 2016 19:51:39 +0000 Received: from smtprelay0065.b.hostedemail.com ([64.98.42.65] helo=smtprelay.b.hostedemail.com) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1cA03T-00079e-MU for pgsql-admin@postgresql.org; Thu, 24 Nov 2016 19:51:38 +0000 Received: from filter.hostedemail.com (10.5.19.248.rfc1918.com [10.5.19.248]) by smtprelay06.b.hostedemail.com (Postfix) with ESMTP id BBC788D247; Thu, 24 Nov 2016 19:51:33 +0000 (UTC) X-Session-Marker: 616C76686572726540616C76682E6E6F2D69702E6F7267 X-Spam-Summary: 50,0,0,,d41d8cd98f00b204,alvherre@alvh.no-ip.org,:::,RULES_HIT:41:355:379:599:967:973:988:989:1028:1260:1277:1311:1312:1313:1314:1345:1359:1437:1515:1516:1518:1519:1534:1540:1593:1594:1595:1596:1711:1730:1747:1777:1792:2393:2525:2553:2561:2564:2682:2685:2693:2828:2859:2895:2909:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4362:4416:4605:5007:6119:6261:7903:9025:9121:10004:10400:10848:11232:11658:11914:12043:12438:12555:13069:13071:13161:13229:13311:13357:13439:13845:13894:13895:14180:14181:14721:14764:21060:21063:21080:21088:21365:21433:30016:30041:30054:30090,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: kite00_d9073cd5cb40 X-Filterd-Recvd-Size: 1834 Received: from alvin.alvh.no-ip.org (unknown [186.9.131.3]) (Authenticated sender: alvherre@alvh.no-ip.org) by omf11.b.hostedemail.com (Postfix) with ESMTPA; Thu, 24 Nov 2016 19:51:33 +0000 (UTC) Received: by alvin.alvh.no-ip.org (Postfix, from userid 1000) id EBAA91112; Thu, 24 Nov 2016 16:51:29 -0300 (CLST) Date: Thu, 24 Nov 2016 16:51:29 -0300 From: Alvaro Herrera To: Phil Frost Cc: pgsql-admin@postgresql.org Subject: Re: Replica lag, high read IO, vacuum index scanning bug? Message-ID: <20161124195129.uuunjnwb4qz4k4bc@alvherre.pgsql> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3fe69397-9feb-707a-032f-968a8d48ad0f@postmates.com> User-Agent: NeoMutt/20161104 (1.7.1) X-Pg-Spam-Score: -1.9 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-admin Precedence: bulk Sender: pgsql-admin-owner@postgresql.org Phil Frost wrote: > We are seeing an issue on RDS 9.4.5 where replica lag gets really high. > This lag seems to be correlated to a saturation of disk read throughput, > and vacuums of one of our larger and more frequently updated tables. > > I've heard that there's a bug where the replica will scan indexes > unnecessarily when doing the WAL replay when the vacuums happen. Can > anyone confirm this and maybe point me towards the commit where it was > fixed? Has there been a backport to the 9.4 series? Hi Phil, The 9.4 backport of the patch in question is this commit by yours truly: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=30e3cb307d494990cb38cebff6bfc8b4ce27d7ab This will appear in the next minor release of the 9.4 series (as well as 9.5), due out sometime during February 2017. Thanks, -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin