Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h1CbH-0006Rp-Qx for pgsql-hackers@arkaria.postgresql.org; Tue, 05 Mar 2019 16:07:28 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1h1CbG-0005LB-9q for pgsql-hackers@arkaria.postgresql.org; Tue, 05 Mar 2019 16:07:26 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h1CbF-0005L0-U3 for pgsql-hackers@lists.postgresql.org; Tue, 05 Mar 2019 16:07:26 +0000 Received: from smtp-fw-9101.amazon.com ([207.171.184.25]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h1CbD-00074p-4e for pgsql-hackers@postgresql.org; Tue, 05 Mar 2019 16:07:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1551802043; x=1583338043; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=i0bBWsYR9tStWtQuxrbrWI4BdNn0jPED1CROd+DB1uE=; b=wDeZML6f9wvv/QdvJx2O+CFXa0O4Aaz78a8VHEC2q0/fgE+cX+JBLpP/ k1Dz2LLP+fZyTUwah421+0qpU6sY76wYtGUf6yaGc4GzO2CuQDobgLXJf DExJi8pBV8H4kbJ/vXC4vmkTcmCdOMtvvi8oB4XI51Ai0ErJwid+bXrWx U=; X-IronPort-AV: E=Sophos;i="5.58,444,1544486400"; d="scan'208";a="791551641" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-2c-2225282c.us-west-2.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 05 Mar 2019 16:07:19 +0000 Received: from EX13MTAUWC001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2c-2225282c.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id x25G7Ijd043861 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 5 Mar 2019 16:07:19 GMT Received: from EX13D05UWC002.ant.amazon.com (10.43.162.92) by EX13MTAUWC001.ant.amazon.com (10.43.162.135) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 5 Mar 2019 16:07:17 +0000 Received: from f01898859afd.ant.amazon.com (10.43.161.23) by EX13D05UWC002.ant.amazon.com (10.43.162.92) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 5 Mar 2019 16:07:17 +0000 Date: Tue, 5 Mar 2019 08:07:15 -0800 From: Shawn Debnath To: Thomas Munro CC: Andres Freund , Dmitry Dolgov <9erthalion6@gmail.com>, Robert Haas , PostgreSQL-development Subject: Re: Refactoring the checkpointer's fsync request queue Message-ID: <20190305160715.GA26468@f01898859afd.ant.amazon.com> References: <20190222224853.ahvm3bf52zqr7k7h@alap3.anarazel.de> <20190222230916.5mfsqn3oesd2nxiz@alap3.anarazel.de> <20190222234549.GA7397@f01898859afd.ant.amazon.com> <20190222235742.4oed46wocpjfpoa5@alap3.anarazel.de> <20190227212653.GA7171@f01898859afd.ant.amazon.com> <2a1667c64cc0485bb781704876ac5a0a@EX13D05UWC002.ant.amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.3 (2019-02-01) X-Originating-IP: [10.43.161.23] X-ClientProxiedBy: EX13D08UWB004.ant.amazon.com (10.43.161.232) To EX13D05UWC002.ant.amazon.com (10.43.162.92) Precedence: Bulk List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Tue, Mar 05, 2019 at 10:45:37PM +1300, Thomas Munro wrote: > On Tue, Mar 5, 2019 at 2:25 PM Shawn Debnath wrote: > > [v11 patch] > > Thanks. Hmm, something is wrong here because make check is > dramatically slower -- for example the "insert" test runs in ~8-13 > seconds instead of the usual ~0.2 seconds according to Travis, > AppVeyor and my local FreeBSD system (note that fsync is disabled so > it's not that -- it must be bogus queue-related CPU?) Confirmed. Patch shows 8900 ms vs 192 ms on master for the insert test. Interesting! It's reproducible so should be able to figure out what's going on. The only thing we do in ForwardSyncRequest() is split up the 8 bits into 2x4 bits and copy the FileTagData structure to the checkpointer queue. Will report back what I find. -- Shawn Debnath Amazon Web Services (AWS)