pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Jianghua Yang <yjhjstz@gmail.com>
To: pgsql-hackers@lists.postgresql.org
Subject: [PATCH] Fix fd leak in pg_dump compression backends when dup()+fdopen() fails
Date: Thu, 19 Mar 2026 09:21:45 -0700
Message-ID: <62bbe34d-2315-4b42-b768-56d901aa83e1@gmail.com> (raw)

Hi,

   I found a file descriptor leak in the pg_dump compression backends
   (gzip, lz4, zstd, and the uncompressed "none" backend), introduced
   in commit e9960732a9 ("Introduce a generic pg_dump compression API",
   PostgreSQL 16).

   == The Bug ==

   All four compression open functions use this pattern when an existing
   file descriptor is passed in:

       if (fd >= 0)
           fp = fdopen(dup(fd), mode);   /* or gzdopen() */

       if (fp == NULL)
           return false;                 /* dup'd fd is leaked here */

   The problem is that dup(fd) and fdopen()/gzdopen() are two separate
   steps, and their failure modes must be handled independently:

   1. If dup() fails (returns -1), no fd is created -- this case was
      not checked at all in the original code.

   2. If dup() succeeds but fdopen()/gzdopen() fails (e.g., due to a
      failed malloc(3) for the FILE structure), POSIX explicitly states:

        "If the fdopen() function fails, the file descriptor is
         not closed."
        -- POSIX.1-2017, fdopen() specification

      The duplicated fd therefore remains open with no owner, leaking
      until the process exits.

   == When Can dup() Fail? ==

   The most realistic trigger for dup() returning EMFILE is parallel
   pg_dump (pg_dump -j N) against a large database.  Each worker opens
   multiple file descriptors for tables, indexes, TOC files, and
   compression streams simultaneously.  On systems with a low per-process
   fd limit (e.g., ulimit -n 1024), or when dumping a schema with a very
   large number of objects, the process fd count can approach the limit.
   At that point, dup() fails with EMFILE and the subsequent NULL-check
   on fp returns false without any cleanup.

   While fdopen() failure (EMFILE/OOM) is less common, it is equally
   incorrect to ignore -- and it is precisely the case that POSIX calls
   out as the caller's responsibility to close.

   == The Fix ==

   Save the result of dup() in a local variable.  Check it immediately.
   If fdopen()/gzdopen() subsequently fails, explicitly close the
   duplicated fd before returning false.

       if (fd >= 0)
       {
           int dup_fd = dup(fd);

           if (dup_fd < 0)
               return false;
           fp = fdopen(dup_fd, mode);
           if (fp == NULL)
           {
               close(dup_fd);   /* POSIX: fdopen does not close on 
failure */
               return false;
           }
       }
       else
       {
           fp = fopen(path, mode);
           if (fp == NULL)
               return false;
       }

   The zstd fix additionally ensures that the previously allocated
   zstdcs structure is freed on all new failure paths.

   == Affected Versions ==

   PostgreSQL 16 and 17, and current master.
   The compress_* files were introduced in commit e9960732a9 (Feb 2023).

   == Patch ==

   Patch attached.  It applies cleanly to current master (as of
   commit 18bcdb75d15).

   Regards,
   Jianghua Yang

Attachments:

0001-Fix-file-descriptor-leak-when-dup-fdopen-gzdopen-fai.patch (application/octet-stream, 4.2K) download From 59a3c0425960483f73b6c352268949a0c44b677a Mon Sep 17 00:00:00 2001 From: Jianghua Yang <yjhjstz@gmail.com> Date: Thu, 19 Mar 2026 09:06:31 -0700 Subject: [PATCH] Fix file descriptor leak when dup()+fdopen()/gzdopen() fails in pg_dump compression backends When fd >= 0, the compression open functions called fdopen(dup(fd), mode) or gzdopen(dup(fd), mode) directly. If dup() succeeded but fdopen()/gzdopen() failed, the duplicated file descriptor was leaked because POSIX specifies that fdopen() does not close the fd on failure. Fix by saving the dup() result in a local variable and explicitly calling close() on it if the subsequent wrapping call fails. Affected backends: compress_gzip.c, compress_lz4.c, compress_none.c, compress_zstd.c. The zstd fix also ensures pg_free(zstdcs) is called on all failure paths after the allocation. --- src/bin/pg_dump/compress_gzip.c | 20 ++++++++++++++++---- src/bin/pg_dump/compress_lz4.c | 27 ++++++++++++++++++++++----- src/bin/pg_dump/compress_none.c | 20 ++++++++++++++++---- src/bin/pg_dump/compress_zstd.c | 28 ++++++++++++++++++++++------ 4 files changed, 76 insertions(+), 19 deletions(-) diff --git a/src/bin/pg_dump/compress_gzip.c b/src/bin/pg_dump/compress_gzip.c index c9ce8a53aaa..60c553ba25a 100644 --- a/src/bin/pg_dump/compress_gzip.c +++ b/src/bin/pg_dump/compress_gzip.c @@ -386,12 +386,24 @@ Gzip_open(const char *path, int fd, const char *mode, CompressFileHandle *CFH) strcpy(mode_compression, mode); if (fd >= 0) - gzfp = gzdopen(dup(fd), mode_compression); + { + int dup_fd = dup(fd); + + if (dup_fd < 0) + return false; + gzfp = gzdopen(dup_fd, mode_compression); + if (gzfp == NULL) + { + close(dup_fd); + return false; + } + } else + { gzfp = gzopen(path, mode_compression); - - if (gzfp == NULL) - return false; + if (gzfp == NULL) + return false; + } CFH->private_data = gzfp; diff --git a/src/bin/pg_dump/compress_lz4.c b/src/bin/pg_dump/compress_lz4.c index 0aa519fbb67..0a7872116e7 100644 --- a/src/bin/pg_dump/compress_lz4.c +++ b/src/bin/pg_dump/compress_lz4.c @@ -715,13 +715,30 @@ LZ4Stream_open(const char *path, int fd, const char *mode, LZ4State *state = (LZ4State *) CFH->private_data; if (fd >= 0) - state->fp = fdopen(dup(fd), mode); + { + int dup_fd = dup(fd); + + if (dup_fd < 0) + { + state->errcode = errno; + return false; + } + state->fp = fdopen(dup_fd, mode); + if (state->fp == NULL) + { + state->errcode = errno; + close(dup_fd); + return false; + } + } else - state->fp = fopen(path, mode); - if (state->fp == NULL) { - state->errcode = errno; - return false; + state->fp = fopen(path, mode); + if (state->fp == NULL) + { + state->errcode = errno; + return false; + } } return true; diff --git a/src/bin/pg_dump/compress_none.c b/src/bin/pg_dump/compress_none.c index d862d8ca6e9..743e2ce94b5 100644 --- a/src/bin/pg_dump/compress_none.c +++ b/src/bin/pg_dump/compress_none.c @@ -231,12 +231,24 @@ open_none(const char *path, int fd, const char *mode, CompressFileHandle *CFH) Assert(CFH->private_data == NULL); if (fd >= 0) - CFH->private_data = fdopen(dup(fd), mode); + { + int dup_fd = dup(fd); + + if (dup_fd < 0) + return false; + CFH->private_data = fdopen(dup_fd, mode); + if (CFH->private_data == NULL) + { + close(dup_fd); + return false; + } + } else + { CFH->private_data = fopen(path, mode); - - if (CFH->private_data == NULL) - return false; + if (CFH->private_data == NULL) + return false; + } return true; } diff --git a/src/bin/pg_dump/compress_zstd.c b/src/bin/pg_dump/compress_zstd.c index cf2db2649ac..68f1d815917 100644 --- a/src/bin/pg_dump/compress_zstd.c +++ b/src/bin/pg_dump/compress_zstd.c @@ -523,14 +523,30 @@ Zstd_open(const char *path, int fd, const char *mode, } if (fd >= 0) - fp = fdopen(dup(fd), mode); - else - fp = fopen(path, mode); + { + int dup_fd = dup(fd); - if (fp == NULL) + if (dup_fd < 0) + { + pg_free(zstdcs); + return false; + } + fp = fdopen(dup_fd, mode); + if (fp == NULL) + { + close(dup_fd); + pg_free(zstdcs); + return false; + } + } + else { - pg_free(zstdcs); - return false; + fp = fopen(path, mode); + if (fp == NULL) + { + pg_free(zstdcs); + return false; + } } zstdcs->fp = fp; -- 2.50.1 (Apple Git-155)

view thread (6+ messages)  latest in thread

Message-ID: <62bbe34d-2315-4b42-b768-56d901aa83e1@gmail.com>
Permalink:  ../62bbe34d-2315-4b42-b768-56d901aa83e1@gmail.com/
Also on:    postgresql.org/message-id/62bbe34d-2315-4b42-b768-56d901aa83e1@gmail.com

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: yjhjstz@gmail.com
  Subject: Re: [PATCH] Fix fd leak in pg_dump compression backends when dup()+fdopen() fails
  In-Reply-To: <62bbe34d-2315-4b42-b768-56d901aa83e1@gmail.com>
  References: <62bbe34d-2315-4b42-b768-56d901aa83e1@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox