Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rp9lK-005Ln8-51 for pgsql-hackers@arkaria.postgresql.org; Tue, 26 Mar 2024 16:34:58 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rp9lJ-000meO-8j for pgsql-hackers@arkaria.postgresql.org; Tue, 26 Mar 2024 16:34:57 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rp9lI-000meF-NX for pgsql-hackers@lists.postgresql.org; Tue, 26 Mar 2024 16:34:56 +0000 Received: from mail-io1-xd34.google.com ([2607:f8b0:4864:20::d34]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rp9lE-006YjP-9a; Tue, 26 Mar 2024 16:34:55 +0000 Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-7cc0e0dba0fso233059139f.3; Tue, 26 Mar 2024 09:34:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711470892; x=1712075692; darn=postgresql.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=fK/wdU9ACNc092zRpChPO7g44eYaKfG30NQf1Dm0AlE=; b=JqkeN4u0kOIttiKT4+oF+sv+16vKO8/1kCSRBhHg7pie/JA+rBQ5Mo9Ucmsy/9r62x sWr2hXx7Y/bO3Fv0Zijtbz4t7LffFfkTJNJiEnIgyy+kR2lmO11RGs6x42m7U7zpH+Vc icFvYqgprRK+pqMh8DwB+ZKPIP9jY48acB35tFz8KQk9b3uISWf3iRDXvM67o7j6wT68 UXk9sWkOyxr00yRScXaMW0Km3Hgb60MVZryk3pU+WbvakJDrMGG2noQKbGGNf91Ai/NH /foxArsVxpXpulgLJGSPHDx0a/gU30S+JJU2NoIkJdFav4vw+dwLCcFg+8iCC+qUfb14 qOSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711470892; x=1712075692; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=fK/wdU9ACNc092zRpChPO7g44eYaKfG30NQf1Dm0AlE=; b=CX2CTu73gm5dWE4cor+8PA4kVhXJqfeKmS/m21YZs/cfmSJVs/f0pSp7OEzwHcF4ji yaDj9CqAyG1ws/ekmPXyZhD+3d7qMu8v6BJqis2KIE8raVwKOnqEpB5gCyEwCeZlvAlB 0e0m+45LrhSzxCAAoJnK1pF4Qc6GRYCbAHqSIG6308mA+p3zCqkpcC6nCsc90OZkXJI2 ja7PWim0eztFk+CK2fqxmzCTTjYKG2/YhZKM54s00ZBtjGtrNd6C91PZG5Wm6kgDU0dg fGHHbOznxY3h5fLRiaz/bH531kaaFnvNR/ENAkEy6t4Y2y93bDR6zYPaPnhVwjum7SVE 14ew== X-Forwarded-Encrypted: i=1; AJvYcCV67vNqjIlWZNPFdlNeXgqh6dkgJWSGQ+eeAllRabGei+JjxkpeQGkVJ0wqBLfvh+kcfmhvR/EMRjCS5IfJmU5HcSFQf9rBC4XYoqp5MTpYKFoYdck6zmgKnmY7ebCrwZJ5+oc= X-Gm-Message-State: AOJu0YxBIk9X2Jvg2Qqs4JymMpt9228c7bb/k5Dd/kKMoL/zwAt3strG kq+jJqERsYKWb6Yiy1C9Uzh7cJmrIp5ak3rGvDQBF3krDoqra8BV X-Google-Smtp-Source: AGHT+IGZ5BaZcCT+Esr2UbHlTP6QfvznkLaF74o2V5zC2evmg27wpunCMzDv5FzGvmH8y+vQzWH8QA== X-Received: by 2002:a6b:6d0d:0:b0:7cc:5d3:3c9 with SMTP id a13-20020a6b6d0d000000b007cc05d303c9mr11983330iod.8.1711470892595; Tue, 26 Mar 2024 09:34:52 -0700 (PDT) Received: from nathanxps13 (162-195-168-172.lightspeed.stlsmo.sbcglobal.net. [162.195.168.172]) by smtp.gmail.com with ESMTPSA id ei12-20020a05663829ac00b0047d6aa1ee2fsm623045jab.13.2024.03.26.09.34.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 09:34:51 -0700 (PDT) Date: Tue, 26 Mar 2024 11:34:49 -0500 From: Nathan Bossart To: Peter Eisentraut Cc: Robert Haas , Nathan Bossart , "pgsql-hackers@postgresql.org" Subject: Re: pgsql: Allow using syncfs() in frontend utilities. Message-ID: <20240326163449.GA3299108@nathanxps13> References: <20240326151131.GA3182007@nathanxps13> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="SUOF0GtieIMvvwua" Content-Disposition: inline In-Reply-To: <20240326151131.GA3182007@nathanxps13> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --SUOF0GtieIMvvwua Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Mar 26, 2024 at 10:11:31AM -0500, Nathan Bossart wrote: > On Tue, Mar 26, 2024 at 11:18:57AM +0100, Peter Eisentraut wrote: >> On 22.03.24 17:52, Robert Haas wrote: >>> I'd like to complain about this commit's addition of a new appendix. >> >> I already complained about that at >> and some follow-up was announced but didn't happen. It was on my list to >> look into cleaning up during beta. > > Sorry about this, I lost track of it. Here's a first attempt at a patch based on Robert's suggestion from upthread. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com --SUOF0GtieIMvvwua Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0001-Adjust-documentation-for-syncfs.patch" From c51703257821079016456b152e4a9f41169dad96 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 26 Mar 2024 11:26:56 -0500 Subject: [PATCH v1 1/1] Adjust documentation for syncfs(). Commit 8c16ad3b43 created a new appendix for syncfs(), which is excessive for the small amount of content it contains. This commit moves the description of the caveats to be aware of when using syncfs() back to the documentation for recovery_init_sync_method. The documentation for the other utilities with syncfs() support now directs readers to recovery_init_sync_method for information about these caveats. Reported-by: Peter Eisentraut Suggested-by: Robert Haas Discussion: https://postgr.es/m/42804669-7063-1320-ed37-3226d5f1067d%40eisentraut.org Discussion: https://postgr.es/m/CA%2BTgmobUiqKr%2BZMCLc5Qap-sXBnjfGUU%2BZBmzYEjUuWyjsGr1g%40mail.gmail.com --- doc/src/sgml/config.sgml | 12 ++++++--- doc/src/sgml/filelist.sgml | 1 - doc/src/sgml/postgres.sgml | 1 - doc/src/sgml/ref/initdb.sgml | 4 +-- doc/src/sgml/ref/pg_basebackup.sgml | 4 +-- doc/src/sgml/ref/pg_checksums.sgml | 4 +-- doc/src/sgml/ref/pg_combinebackup.sgml | 4 +-- doc/src/sgml/ref/pg_dump.sgml | 5 ++-- doc/src/sgml/ref/pg_rewind.sgml | 4 +-- doc/src/sgml/ref/pgupgrade.sgml | 4 +-- doc/src/sgml/syncfs.sgml | 36 -------------------------- 11 files changed, 24 insertions(+), 55 deletions(-) delete mode 100644 doc/src/sgml/syncfs.sgml diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 65a6e6c408..5468637e2e 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -10870,9 +10870,15 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir' On Linux, syncfs may be used instead, to ask the operating system to synchronize the file systems that contain the data directory, the WAL files and each tablespace (but not any other - file systems that may be reachable through symbolic links). See - for more information about using - syncfs(). + file systems that may be reachable through symbolic links). This may + be a lot faster than the fsync setting, because it + doesn't need to open each file one by one. On the other hand, it may + be slower if a file system is shared by other applications that + modify a lot of files, since those files will also be written to disk. + Furthermore, on versions of Linux before 5.8, I/O errors encountered + while writing data to disk may not be reported to + PostgreSQL, and relevant error messages may + appear only in kernel logs. This parameter can only be set in the diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index e0dca81cb2..c92a16c7ac 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -182,7 +182,6 @@ - diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml index 2c107199d3..381af69be2 100644 --- a/doc/src/sgml/postgres.sgml +++ b/doc/src/sgml/postgres.sgml @@ -289,7 +289,6 @@ break is not needed in a wider output rendering. &acronyms; &glossary; &color; - &syncfs; &obsolete; diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml index 377c3cb20a..dc9011b40e 100644 --- a/doc/src/sgml/ref/initdb.sgml +++ b/doc/src/sgml/ref/initdb.sgml @@ -394,8 +394,8 @@ PostgreSQL documentation On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file systems that contain the data directory, the WAL files, and each tablespace. See - for more information about using - syncfs(). + for information about + the caveats to be aware of when using syncfs. This option has no effect when is used. diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml index 208530f393..82d0c8e008 100644 --- a/doc/src/sgml/ref/pg_basebackup.sgml +++ b/doc/src/sgml/ref/pg_basebackup.sgml @@ -642,8 +642,8 @@ PostgreSQL documentation backup directory. When the plain format is used, pg_basebackup will also synchronize the file systems that contain the WAL files and each tablespace. See - for more information about using - syncfs(). + for information about + the caveats to be aware of when using syncfs. This option has no effect when is used. diff --git a/doc/src/sgml/ref/pg_checksums.sgml b/doc/src/sgml/ref/pg_checksums.sgml index db5b29505c..95043aa329 100644 --- a/doc/src/sgml/ref/pg_checksums.sgml +++ b/doc/src/sgml/ref/pg_checksums.sgml @@ -152,8 +152,8 @@ PostgreSQL documentation On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file systems that contain the data directory, the WAL files, and each tablespace. See - for more information about using - syncfs(). + for information about + the caveats to be aware of when using syncfs. This option has no effect when is used. diff --git a/doc/src/sgml/ref/pg_combinebackup.sgml b/doc/src/sgml/ref/pg_combinebackup.sgml index 8a0a600c2b..6f90dba281 100644 --- a/doc/src/sgml/ref/pg_combinebackup.sgml +++ b/doc/src/sgml/ref/pg_combinebackup.sgml @@ -176,8 +176,8 @@ PostgreSQL documentation backup directory. When the plain format is used, pg_combinebackup will also synchronize the file systems that contain the WAL files and each tablespace. See - for more information about using - syncfs(). + for information about + the caveats to be aware of when using syncfs. This option has no effect when is used. diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml index 8edf03a03d..b99793e414 100644 --- a/doc/src/sgml/ref/pg_dump.sgml +++ b/doc/src/sgml/ref/pg_dump.sgml @@ -1319,8 +1319,9 @@ PostgreSQL documentation On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file system that contains the - archive directory. See for more information - about using syncfs(). + archive directory. See + for information about the caveats to be aware of when using + syncfs. This option has no effect when is used or diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml index 841bd49fc7..dc039d8756 100644 --- a/doc/src/sgml/ref/pg_rewind.sgml +++ b/doc/src/sgml/ref/pg_rewind.sgml @@ -297,8 +297,8 @@ PostgreSQL documentation On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file systems that contain the data directory, the WAL files, and each tablespace. See - for more information about using - syncfs(). + for information about + the caveats to be aware of when using syncfs. This option has no effect when is used. diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml index 8de52bf752..80a50377b1 100644 --- a/doc/src/sgml/ref/pgupgrade.sgml +++ b/doc/src/sgml/ref/pgupgrade.sgml @@ -204,8 +204,8 @@ PostgreSQL documentation On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file systems that contain the upgraded cluster's data directory, its WAL files, and each tablespace. - See for more information about using - syncfs(). + See for information + about the caveats to be aware of when using syncfs. This option has no effect when is used. diff --git a/doc/src/sgml/syncfs.sgml b/doc/src/sgml/syncfs.sgml deleted file mode 100644 index 00457d2457..0000000000 --- a/doc/src/sgml/syncfs.sgml +++ /dev/null @@ -1,36 +0,0 @@ - - - - <function>syncfs()</function> Caveats - - - syncfs - - - - On Linux syncfs() may be specified for some - configuration parameters (e.g., - ), server applications (e.g., - pg_upgrade), and client applications (e.g., - pg_basebackup) that involve synchronizing many - files to disk. syncfs() is advantageous in many cases, - but there are some trade-offs to keep in mind. - - - - Since syncfs() instructs the operating system to - synchronize a whole file system, it typically requires many fewer system - calls than using fsync() to synchronize each file one by - one. Therefore, using syncfs() may be a lot faster than - using fsync(). However, it may be slower if a file - system is shared by other applications that modify a lot of files, since - those files will also be written to disk. - - - - Furthermore, on versions of Linux before 5.8, I/O errors encountered while - writing data to disk may not be reported to the calling program, and relevant - error messages may appear only in kernel logs. - - - -- 2.25.1 --SUOF0GtieIMvvwua--