From: Michael Paquier <michael@paquier.xyz>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Replace off_t by pgoff_t in I/O routines
Date: Thu, 13 Nov 2025 03:43:12 +0000
Message-ID: <E1vJOEq-006LN7-1B@gemulon.postgresql.org> (raw)
Replace off_t by pgoff_t in I/O routines
PostgreSQL's Windows port has never been able to handle files larger
than 2GB due to the use of off_t for file offsets, only 32-bit on
Windows. This causes signed integer overflow at exactly 2^31 bytes when
trying to handle files larger than 2GB, for the routines touched by this
commit.
Note that large files are forbidden by ./configure (3c6248a828af) and
meson (recent change, see 79cd66f28c65). This restriction also exists
in v16 and older versions for the now-dead MSVC scripts.
The code base already defines pgoff_t as __int64 (64-bit) on Windows for
this purpose, and some function declarations in headers use it, but many
internals still rely on off_t. This commit switches more routines to
use pgoff_t, offering more portability, for areas mainly related to file
extensions and storage.
These are not critical for WAL segments yet, which have currently a
maximum size allowed of 1GB (well, this opens the door at allowing a
larger size for them). This matters more for segment files if we want
to lift the large file restriction in ./configure and meson in the
future, which would make sense to remove once/if all traces of off_t are
gone from the tree. This can additionally matter for out-of-core code
that may want files larger than 2GB in places where off_t is four bytes
in size.
Note that off_t is still used in other parts of the tree like
buffile.c, WAL sender/receiver, base backup, pg_combinebackup, etc.
These other code paths can be addressed separately, and their update
will be required if we want to remove the large file restriction in the
future. This commit is a good first cut in itself towards more
portability, hopefully.
On Unix-like systems, pgoff_t is defined as off_t, so this change only
affects Windows behavior.
Author: Bryan Green <dbryan.green@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/0f238ff4-c442-42f5-adb8-01b762c94ca1@gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/84fb27511dbeaa0fb0d48249083d71f6cbbd4b98
Modified Files
--------------
src/backend/access/transam/xlogreader.c | 2 +-
src/backend/access/transam/xlogrecovery.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/backend/storage/file/fd.c | 36 +++++++++++-----------
src/backend/storage/smgr/md.c | 50 +++++++++++++++----------------
src/common/file_utils.c | 4 +--
src/include/common/file_utils.h | 4 +--
src/include/port/pg_iovec.h | 4 +--
src/include/port/win32_port.h | 4 +--
src/include/storage/fd.h | 26 ++++++++--------
src/port/win32pread.c | 6 ++--
src/port/win32pwrite.c | 6 ++--
src/tools/pgindent/typedefs.list | 1 +
13 files changed, 76 insertions(+), 71 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: michael@paquier.xyz, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Replace off_t by pgoff_t in I/O routines
In-Reply-To: <E1vJOEq-006LN7-1B@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox