MIME-Version: 1.0
From: Jianghua Yang <yjhjstz@gmail.com>
Date: Fri, 20 Mar 2026 10:02:44 -0700
Message-ID: 
 <CAAZLFmQNJ0QNArpWEOZXwv=vbumcWKEHz-b1me5gBqRqG67EwQ@mail.gmail.com>
Subject: basebackup: add missing deflateEnd() in gzip compression sink
To: pgsql-hackers@lists.postgresql.org
Content-Type: multipart/mixed; boundary="000000000000954c52064d77a91e"
Archived-At: 
 <https://www.postgresql.org/message-id/CAAZLFmQNJ0QNArpWEOZXwv%3DvbumcWKEHz-b1me5gBqRqG67EwQ%40mail.gmail.com>
Precedence: bulk

--000000000000954c52064d77a91e
Content-Type: multipart/alternative; boundary="000000000000954c51064d77a91c"

--000000000000954c51064d77a91c
Content-Type: text/plain; charset="UTF-8"

Hi hackers,

  While reviewing the backup compression backends, I noticed that the gzip
  sink (basebackup_gzip.c) never calls deflateEnd(), unlike the lz4 and
  zstd sinks which properly free their compression contexts.

  Specifically:

  - lz4 calls LZ4F_freeCompressionContext() in both end_archive and a
    dedicated bbsink_lz4_cleanup() callback.
  - zstd calls ZSTD_freeCCtx() in both end_archive and a dedicated
    bbsink_zstd_cleanup() callback.
  - gzip calls deflateInit2() in begin_archive but never calls
    deflateEnd(). Its cleanup callback is bbsink_forward_cleanup,
    which just forwards to the next sink without releasing the zlib
    context.

  This means each archive (one per tablespace) leaks ~256KB of zlib
  internal state until the backup's memory context is destroyed. On the
  ERROR path, the zlib context is not released at all since there is no
  gzip-specific cleanup callback.

  The attached patch adds deflateEnd() at the end of
  bbsink_gzip_end_archive() for the normal path, and introduces a new
  bbsink_gzip_cleanup() for the error path. The new cleanup callback
  follows the exact same call chain as bbsink_lz4_cleanup and
  bbsink_zstd_cleanup:

    PG_FINALLY  (basebackup.c:1063)
      -> bbsink_cleanup(sink)  (basebackup.c:1065)
        -> sink->bbs_ops->cleanup(sink)  (basebackup_sink.h:269)
          -> bbsink_gzip_cleanup()  -- now calls deflateEnd()

  Previously the gzip slot in this chain pointed to
  bbsink_forward_cleanup, which just forwarded to the next sink
  without doing any gzip-specific resource release.

  Since z_stream is an embedded struct (not a pointer like LZ4F_cctx or
  ZSTD_CCtx), a bool flag "zstream_initialized" is used to track whether
  deflateEnd() needs to be called.

  Tested with pg_basebackup using gzip compression at levels 1, 5
  (default), and 9, including server-side compression. All produced
  valid .tar.gz files that pass gzip -t integrity checks and restore
  correctly.

  Regards,
  Jianghua Yang

--000000000000954c51064d77a91c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi hackers,<br><br>=C2=A0 While reviewing the backup compr=
ession backends, I noticed that the gzip<br>=C2=A0 sink (basebackup_gzip.c)=
 never calls deflateEnd(), unlike the lz4 and<br>=C2=A0 zstd sinks which pr=
operly free their compression contexts.<br><br>=C2=A0 Specifically:<br><br>=
=C2=A0 - lz4 calls LZ4F_freeCompressionContext() in both end_archive and a<=
br>=C2=A0 =C2=A0 dedicated bbsink_lz4_cleanup() callback.<br>=C2=A0 - zstd =
calls ZSTD_freeCCtx() in both end_archive and a dedicated<br>=C2=A0 =C2=A0 =
bbsink_zstd_cleanup() callback.<br>=C2=A0 - gzip calls deflateInit2() in be=
gin_archive but never calls<br>=C2=A0 =C2=A0 deflateEnd(). Its cleanup call=
back is bbsink_forward_cleanup,<br>=C2=A0 =C2=A0 which just forwards to the=
 next sink without releasing the zlib<br>=C2=A0 =C2=A0 context.<br><br>=C2=
=A0 This means each archive (one per tablespace) leaks ~256KB of zlib<br>=
=C2=A0 internal state until the backup&#39;s memory context is destroyed. O=
n the<br>=C2=A0 ERROR path, the zlib context is not released at all since t=
here is no<br>=C2=A0 gzip-specific cleanup callback.<br><br>=C2=A0 The atta=
ched patch adds deflateEnd() at the end of<br>=C2=A0 bbsink_gzip_end_archiv=
e() for the normal path, and introduces a new<br>=C2=A0 bbsink_gzip_cleanup=
() for the error path. The new cleanup callback<br>=C2=A0 follows the exact=
 same call chain as bbsink_lz4_cleanup and<br>=C2=A0 bbsink_zstd_cleanup:<b=
r><br>=C2=A0 =C2=A0 PG_FINALLY =C2=A0(basebackup.c:1063)<br>=C2=A0 =C2=A0 =
=C2=A0 -&gt; bbsink_cleanup(sink) =C2=A0(basebackup.c:1065)<br>=C2=A0 =C2=
=A0 =C2=A0 =C2=A0 -&gt; sink-&gt;bbs_ops-&gt;cleanup(sink) =C2=A0(basebacku=
p_sink.h:269)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 -&gt; bbsink_gzip_clean=
up() =C2=A0-- now calls deflateEnd()<br><br>=C2=A0 Previously the gzip slot=
 in this chain pointed to<br>=C2=A0 bbsink_forward_cleanup, which just forw=
arded to the next sink<br>=C2=A0 without doing any gzip-specific resource r=
elease.<br><br>=C2=A0 Since z_stream is an embedded struct (not a pointer l=
ike LZ4F_cctx or<br>=C2=A0 ZSTD_CCtx), a bool flag &quot;zstream_initialize=
d&quot; is used to track whether<br>=C2=A0 deflateEnd() needs to be called.=
<br><br>=C2=A0 Tested with pg_basebackup using gzip compression at levels 1=
, 5<br>=C2=A0 (default), and 9, including server-side compression. All prod=
uced<br>=C2=A0 valid .tar.gz files that pass gzip -t integrity checks and r=
estore<br>=C2=A0 correctly.<br><br>=C2=A0 Regards,<br>=C2=A0 Jianghua Yang<=
/div>

--000000000000954c51064d77a91c--

--000000000000954c52064d77a91e
Content-Type: application/octet-stream;
	name="0001-basebackup-add-missing-deflateEnd-calls-in-gzip-comp.patch"
Content-Disposition: attachment;
	filename="0001-basebackup-add-missing-deflateEnd-calls-in-gzip-comp.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mmz5cp1w0>
X-Attachment-Id: f_mmz5cp1w0

RnJvbSA1ZDFlZGVjZWFmZDQyMjA3MmMxNzAyNmRmZDFmODYyYTE5ZmViNzkyIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBKaWFuZ2h1YSBZYW5nIDx5amhqc3R6QGdtYWlsLmNvbT4KRGF0
ZTogRnJpLCAyMCBNYXIgMjAyNiAwOTo1NzoxNyAtMDcwMApTdWJqZWN0OiBbUEFUQ0hdIGJhc2Vi
YWNrdXA6IGFkZCBtaXNzaW5nIGRlZmxhdGVFbmQoKSBjYWxscyBpbiBnemlwCiBjb21wcmVzc2lv
biBzaW5rCgpUaGUgZ3ppcCBiYXNlYmFja3VwIHNpbmsgY2FsbGVkIGRlZmxhdGVJbml0MigpIGlu
IGJlZ2luX2FyY2hpdmUgYnV0Cm5ldmVyIGNhbGxlZCBkZWZsYXRlRW5kKCksIGxlYWtpbmcgemxp
YidzIGludGVybmFsIGNvbXByZXNzaW9uIHN0YXRlCih+MjU2S0IgcGVyIGFyY2hpdmUpIHVudGls
IHRoZSBtZW1vcnkgY29udGV4dCB3YXMgZGVzdHJveWVkLiBJdCBhbHNvCmxhY2tlZCBhIGRlZGlj
YXRlZCBjbGVhbnVwIGNhbGxiYWNrLCBzbyBvbiBFUlJPUiB0aGUgemxpYiBjb250ZXh0CndhcyBu
b3QgcmVsZWFzZWQgYXQgYWxsLgoKQWRkIGRlZmxhdGVFbmQoKSBhdCB0aGUgZW5kIG9mIGJic2lu
a19nemlwX2VuZF9hcmNoaXZlKCkgZm9yIHRoZQpub3JtYWwgcGF0aCwgYW5kIGFkZCBhIG5ldyBi
YnNpbmtfZ3ppcF9jbGVhbnVwKCkgZnVuY3Rpb24gZm9yIHRoZQplcnJvciBwYXRoLiAgVGhpcyBi
cmluZ3MgdGhlIGd6aXAgc2luayBpbiBsaW5lIHdpdGggdGhlIGx6NCBhbmQgenN0ZApzaW5rcywg
d2hpY2ggYWxyZWFkeSBwcm9wZXJseSBmcmVlIHRoZWlyIGNvbXByZXNzaW9uIGNvbnRleHRzIGlu
IGJvdGgKZW5kX2FyY2hpdmUgYW5kIGNsZWFudXAgY2FsbGJhY2tzLgoKQXV0aG9yOiBKaWFuZ2h1
YSBZYW5nIDxqaWFuZ2h1YS55amhAYWxpYmFiYS1pbmMuY29tPgotLS0KIHNyYy9iYWNrZW5kL2Jh
Y2t1cC9iYXNlYmFja3VwX2d6aXAuYyB8IDI3ICsrKysrKysrKysrKysrKysrKysrKysrKysrLQog
MSBmaWxlIGNoYW5nZWQsIDI2IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKCmRpZmYgLS1n
aXQgYS9zcmMvYmFja2VuZC9iYWNrdXAvYmFzZWJhY2t1cF9nemlwLmMgYi9zcmMvYmFja2VuZC9i
YWNrdXAvYmFzZWJhY2t1cF9nemlwLmMKaW5kZXggMWJhMjUwMTVhYjcuLmMyNjRhODMzNDI5IDEw
MDY0NAotLS0gYS9zcmMvYmFja2VuZC9iYWNrdXAvYmFzZWJhY2t1cF9nemlwLmMKKysrIGIvc3Jj
L2JhY2tlbmQvYmFja3VwL2Jhc2ViYWNrdXBfZ3ppcC5jCkBAIC0zMiw2ICszMiw5IEBAIHR5cGVk
ZWYgc3RydWN0IGJic2lua19nemlwCiAKIAkvKiBOdW1iZXIgb2YgYnl0ZXMgc3RhZ2VkIGluIG91
dHB1dCBidWZmZXIuICovCiAJc2l6ZV90CQlieXRlc193cml0dGVuOworCisJLyogSGFzIHRoZSB6
c3RyZWFtIGJlZW4gaW5pdGlhbGl6ZWQgd2l0aCBkZWZsYXRlSW5pdDI/ICovCisJYm9vbAkJenN0
cmVhbV9pbml0aWFsaXplZDsKIH0gYmJzaW5rX2d6aXA7CiAKIHN0YXRpYyB2b2lkIGJic2lua19n
emlwX2JlZ2luX2JhY2t1cChiYnNpbmsgKnNpbmspOwpAQCAtMzksNiArNDIsNyBAQCBzdGF0aWMg
dm9pZCBiYnNpbmtfZ3ppcF9iZWdpbl9hcmNoaXZlKGJic2luayAqc2luaywgY29uc3QgY2hhciAq
YXJjaGl2ZV9uYW1lKTsKIHN0YXRpYyB2b2lkIGJic2lua19nemlwX2FyY2hpdmVfY29udGVudHMo
YmJzaW5rICpzaW5rLCBzaXplX3QgbGVuKTsKIHN0YXRpYyB2b2lkIGJic2lua19nemlwX21hbmlm
ZXN0X2NvbnRlbnRzKGJic2luayAqc2luaywgc2l6ZV90IGxlbik7CiBzdGF0aWMgdm9pZCBiYnNp
bmtfZ3ppcF9lbmRfYXJjaGl2ZShiYnNpbmsgKnNpbmspOworc3RhdGljIHZvaWQgYmJzaW5rX2d6
aXBfY2xlYW51cChiYnNpbmsgKnNpbmspOwogc3RhdGljIHZvaWQgKmd6aXBfcGFsbG9jKHZvaWQg
Km9wYXF1ZSwgdW5zaWduZWQgaXRlbXMsIHVuc2lnbmVkIHNpemUpOwogc3RhdGljIHZvaWQgZ3pp
cF9wZnJlZSh2b2lkICpvcGFxdWUsIHZvaWQgKmFkZHJlc3MpOwogCkBAIC01MSw3ICs1NSw3IEBA
IHN0YXRpYyBjb25zdCBiYnNpbmtfb3BzIGJic2lua19nemlwX29wcyA9IHsKIAkubWFuaWZlc3Rf
Y29udGVudHMgPSBiYnNpbmtfZ3ppcF9tYW5pZmVzdF9jb250ZW50cywKIAkuZW5kX21hbmlmZXN0
ID0gYmJzaW5rX2ZvcndhcmRfZW5kX21hbmlmZXN0LAogCS5lbmRfYmFja3VwID0gYmJzaW5rX2Zv
cndhcmRfZW5kX2JhY2t1cCwKLQkuY2xlYW51cCA9IGJic2lua19mb3J3YXJkX2NsZWFudXAKKwku
Y2xlYW51cCA9IGJic2lua19nemlwX2NsZWFudXAKIH07CiAjZW5kaWYKIApAQCAtMTQxLDYgKzE0
NSw3IEBAIGJic2lua19nemlwX2JlZ2luX2FyY2hpdmUoYmJzaW5rICpzaW5rLCBjb25zdCBjaGFy
ICphcmNoaXZlX25hbWUpCiAJCWVyZXBvcnQoRVJST1IsCiAJCQkJZXJyY29kZShFUlJDT0RFX0lO
VEVSTkFMX0VSUk9SKSwKIAkJCQllcnJtc2coImNvdWxkIG5vdCBpbml0aWFsaXplIGNvbXByZXNz
aW9uIGxpYnJhcnkiKSk7CisJbXlzaW5rLT56c3RyZWFtX2luaXRpYWxpemVkID0gdHJ1ZTsKIAog
CS8qCiAJICogQWRkICIuZ3oiIHRvIHRoZSBhcmNoaXZlIG5hbWUuIE5vdGUgdGhhdCB0aGUgcGdf
YmFzZWJhY2t1cCAteiBwcm9kdWNlcwpAQCAtMjY2LDYgKzI3MSwxMCBAQCBiYnNpbmtfZ3ppcF9l
bmRfYXJjaGl2ZShiYnNpbmsgKnNpbmspCiAJCW15c2luay0+Ynl0ZXNfd3JpdHRlbiA9IDA7CiAJ
fQogCisJLyogUmVsZWFzZSB0aGUgY29tcHJlc3Npb24gcmVzb3VyY2VzLiAqLworCWRlZmxhdGVF
bmQoenMpOworCW15c2luay0+enN0cmVhbV9pbml0aWFsaXplZCA9IGZhbHNlOworCiAJLyogTXVz
dCBhbHNvIHBhc3Mgb24gdGhlIGluZm9ybWF0aW9uIHRoYXQgdGhpcyBhcmNoaXZlIGhhcyBlbmRl
ZC4gKi8KIAliYnNpbmtfZm9yd2FyZF9lbmRfYXJjaGl2ZShzaW5rKTsKIH0KQEAgLTMwMSw0ICsz
MTAsMjAgQEAgZ3ppcF9wZnJlZSh2b2lkICpvcGFxdWUsIHZvaWQgKmFkZHJlc3MpCiAJcGZyZWUo
YWRkcmVzcyk7CiB9CiAKKy8qCisgKiBJbiBjYXNlIHRoZSBiYWNrdXAgZmFpbHMsIG1ha2Ugc3Vy
ZSB3ZSBmcmVlIHRoZSBjb21wcmVzc2lvbiBjb250ZXh0IGJ5CisgKiBjYWxsaW5nIGRlZmxhdGVF
bmQoKSBpZiBuZWVkZWQgdG8gYXZvaWQgYSByZXNvdXJjZSBsZWFrLgorICovCitzdGF0aWMgdm9p
ZAorYmJzaW5rX2d6aXBfY2xlYW51cChiYnNpbmsgKnNpbmspCit7CisJYmJzaW5rX2d6aXAgKm15
c2luayA9IChiYnNpbmtfZ3ppcCAqKSBzaW5rOworCisJaWYgKG15c2luay0+enN0cmVhbV9pbml0
aWFsaXplZCkKKwl7CisJCWRlZmxhdGVFbmQoJm15c2luay0+enN0cmVhbSk7CisJCW15c2luay0+
enN0cmVhbV9pbml0aWFsaXplZCA9IGZhbHNlOworCX0KK30KKwogI2VuZGlmCi0tIAoyLjUwLjEg
KEFwcGxlIEdpdC0xNTUpCgo=
--000000000000954c52064d77a91e--