Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lN5Uy-0007jA-G1 for pgsql-hackers@arkaria.postgresql.org; Fri, 19 Mar 2021 03:08:28 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lN5Uw-0000Xj-Uw for pgsql-hackers@arkaria.postgresql.org; Fri, 19 Mar 2021 03:08:26 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lN5Uw-0000VQ-O6 for pgsql-hackers@lists.postgresql.org; Fri, 19 Mar 2021 03:08:26 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lN5Uu-0005pq-7f for pgsql-hackers@postgresql.org; Fri, 19 Mar 2021 03:08:25 +0000 Received: from hnk.local (p2350207-ipbf2126funabasi.chiba.ocn.ne.jp [114.167.184.207]) by oss.nttdata.com (Postfix) with ESMTPSA id A1586627A3; Fri, 19 Mar 2021 12:08:21 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.102.3 at oss.nttdata.com Subject: Re: fdatasync performance problem with large number of DB files From: Fujii Masao To: Thomas Munro Cc: Bruce Momjian , Paul Guo , Tom Lane , Michael Brown , pgsql-hackers References: <823DAB20-5A5E-4F7B-BF8C-D61FA574DEEA@vmware.com> <76de0e61-a553-6003-aeec-cb35ada791cf@oss.nttdata.com> <52453bfc-40fb-7771-05ee-cbd04b6b9b6e@oss.nttdata.com> <4CB921E5-772B-42F9-B14D-D310E709E8F2@vmware.com> <20210318135411.GB8529@momjian.us> <20210318140352.GC8529@momjian.us> <3d1086bd-0881-e0a3-9f4f-bda608f07fdb@oss.nttdata.com> Message-ID: Date: Fri, 19 Mar 2021 12:08:19 +0900 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 2021/03/19 11:22, Fujii Masao wrote: > > > On 2021/03/19 10:37, Thomas Munro wrote: >> On Fri, Mar 19, 2021 at 2:16 PM Thomas Munro wrote: >>> PS: For illustration/discussion, I've also attached a "none" patch.  I >>> also couldn't resist rebasing my "wal" mode patch, which I plan to >>> propose for PG15 because there is not enough time left for this >>> release. >> >> Erm... I attached the wrong version by mistake.  Here's a better one. 0002 patch looks good to me. Thanks! I have minor comments. - * Issue fsync recursively on PGDATA and all its contents, or issue syncfs for - * all potential filesystem, depending on recovery_init_sync_method setting. + * Issue fsync recursively on PGDATA and all its contents, issue syncfs for + * all potential filesystem, or do nothing, depending on + * recovery_init_sync_method setting. The comment in SyncDataDirectory() should be updated so that it mentions "none" method, as the above? + This is only safe if all buffered data is known to have been flushed + to disk already, for example by a tool such as + pg_basebackup. It is not a good idea to Isn't it better to add something like "without --no-sync" to "pg_basebackup" part? Which would prevent users from misunderstanding that pg_basebackup always ensures that whatever options are specified. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION