Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1viRjv-006U7v-2V for pgsql-hackers@arkaria.postgresql.org; Wed, 21 Jan 2026 06:30:52 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1viRju-005B8e-2w for pgsql-hackers@arkaria.postgresql.org; Wed, 21 Jan 2026 06:30:51 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1viRju-005B8P-1Z for pgsql-hackers@lists.postgresql.org; Wed, 21 Jan 2026 06:30:50 +0000 Received: from mail-dy1-x1343.google.com ([2607:f8b0:4864:20::1343]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1viRjr-001Xu2-2D for pgsql-hackers@postgresql.org; Wed, 21 Jan 2026 06:30:49 +0000 Received: by mail-dy1-x1343.google.com with SMTP id 5a478bee46e88-2b714f30461so193327eec.0 for ; Tue, 20 Jan 2026 22:30:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1768977047; cv=none; d=google.com; s=arc-20240605; b=K88AOqnAnR8t3l19NKGkPT3X0h6xzeiIu3PGDlO1C83o0WDyBHlZ3iQRFbtKa10UCU k9j4rb5C1hqUUB97UvQMtKSVEWJsjpy7y4a1rL83qdO1J6rF1c9THMtyFpS38StgWvSf qhFya7lsXi1qkBMPjt1j122oHbOmLEc+g0HdiAVNAsIkbTxYd9lfl4/gvOv51nKicDa7 1eyPdCuyvEVaECzG2ayj6EmKKgoy45WAeOfO7hICdUQ0yn4P+7wkmKfTt5Slg/KYQuNj /HKO5xcIWPQ8Oh6HAI+QMaTQ9upzOFMdIN4ytBQe02cQivo/572oZrgJBOVYydLoPN2H Ztcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=9CrqpnpgdtJ5OjA23480fqq0fzD0uFWif1d4+yuA+c8=; fh=G/7x+qdFIj3vjCl9da8xrevBF6Z9d3eOi9BwY4QuWH8=; b=A0ZoAb+gtmw07w8RWolXXOshkRwIaY4OiO3W6bC6HmVDhXCeBndZTo0MUrm7WP5pF8 9w0qLQUgUPAB3IVzpfHD60E5H31XL41A0VDsU5iVH1rrTy98VRyQCMdFbsP/Z6gwosT3 PF9iRPmGoXrl6GKTcoGmVb21J+joW5NAr+pnK8jOY397vgsquSGE0pE05aiJFkS9/wJ1 fjyRCdn1sli9YQUDl0b0KzwyUC5g/A8puLUCObk2/xFYeg895uZhjxKZ3z2S1uBF+Cbn faFaptJ4WgkDbG8NxSPFCIQiqEi9PJWjJYBnQRPNPklTO1rCY4B1E5YKptayQOotVFIg nzpQ==; darn=postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768977047; x=1769581847; darn=postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=9CrqpnpgdtJ5OjA23480fqq0fzD0uFWif1d4+yuA+c8=; b=lmSnkuUAs52HcYT+wQDEWhFD4Z+8z4fixFG7LOjo4dKg+5OONr/2MRgs2Jwducqpab 3r2p5u9szwr8SSfR64+SbKQCXbk0sLW/GmPUpKTFT8YAcc53OhL1Q21aBtJFujTWHFSb mQoL6UYKP0vwfJkmZxnc1XnVa2InGVMvWfbTHMUreVu1bgLZSuA/R9TdKcEHbXyakpNz +ObjhPjq+QQnBdLi0aLhmrv0jjkqccMBHmykiEP7QBn6/3g7uJVXEactYVT4rAg5fxu3 ir5BfL/J6ezIHrGxDCakQKpiw4gUa5nAPjif3X0Kz+pf9FX5O3v2DWSR91KAXjaD8rEN bBLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768977047; x=1769581847; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=9CrqpnpgdtJ5OjA23480fqq0fzD0uFWif1d4+yuA+c8=; b=dVu+W8HyG6akxrhD67KUuN98Ptxm6/xYixbL5AvkEQUzW7eXCinIWgquywlk1YYLc9 ilbZNpMdd/dYYeY8wMpmKRd9hrAH4s04PlEIeVUIB7BQQCXN4iuftq9rDYOVP9b6vZhm 0o5tH/EiJC+lbD2dUi5ZsBS0VASwM8tzGZMYcGsKZlp2zJKojc8T9ideJoId63ldyJnl MsIje3/lQCEilxF41H+06DgDqPMKC5WLjuF4syG3rFdxV6y5vx3aokRHXXoAElFRpdwP uSgk4JWY6r9Oe3TJ7mSQb3KRg9UYqmvGKNp3C4UC9JjfErYvd5FTqEDkDKaDDzeoOKiC cPfQ== X-Gm-Message-State: AOJu0YzjTKN2aDEP/ItllAs3BF6wd88oThOe1VgbPCe/YcUgdJAXi496 TY1fut4tQcGPs0QSj/OhdcW4VsQlLvUhg0jkYWpmeHMIJFW+P1Q7Y+K4hx6qyn8noAcSLE2TtCQ MvrxozX4LnkMce032SpDFROJDDpecI3cp+uqj X-Gm-Gg: AZuq6aKbePxo7qsQWzBG5VvitPw5Uj5OmHV9tRL9NYEF7fhUdi6Ek0botUZ10tBJcdV yP3PLPSydhF7iQiwuTH9dZ1q/9M9+44QEpkTaJwvIUUbBHiuCWSC3qgeWM8pGDLkqjqka8cyZVT UZz/FBXc3N2N7I2wDIvnVPdz+yGBX60cNS8d6hoOr9tWVMDY/U9HiA/6hZt3TJBGlxoUtjIeJyM XUs0tdBMuZhlQ8nfNfJcplkiZZkAvqEc7Afu0N8y20rXYcA2MeDYCZLG8g2q11vUVjYvvEZuMbI 4Pqirg== X-Received: by 2002:a05:7301:100d:b0:2b7:140c:c700 with SMTP id 5a478bee46e88-2b7140cc765mr664839eec.20.1768977047302; Tue, 20 Jan 2026 22:30:47 -0800 (PST) MIME-Version: 1.0 References: <7bcc0420-d457-4af5-a459-a4e5d929a665@vondra.me> <7gbzkoxrpqvj2sgxof4pqirz7pf6wb5vizak3mxrz2zpvrf3pa@lvnwdsy3em56> In-Reply-To: From: lakshmi Date: Wed, 21 Jan 2026 12:02:58 +0530 X-Gm-Features: AZwV_QgEPJ4AIKO_MylHnj8HFoXEW45yen0LLIg-rj3GXfG9kRy9Y-hjIvYMpYg Message-ID: Subject: Re: Proposal: Adding compression of temporary files To: pgsql-hackers Cc: Filip Janus , zsolt.parragi@percona.com Content-Type: multipart/mixed; boundary="00000000000085a7b90648e01013" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000085a7b90648e01013 Content-Type: multipart/alternative; boundary="00000000000085a7b60648e01011" --00000000000085a7b60648e01011 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable HI all, While testing the temp file compression patch,noticed that the new temp_file_compression GUC isn't documented yet.I put together a small docs patch to add a short description and clarify that the effect of compression depends on the workload(for example ,hash join spills may not show visible size reduction due to fixed_size chunks). patch is attached.Happy to adjust the wording if needed. thanks, lakshmi On Tue, Jan 20, 2026 at 4:21=E2=80=AFPM lakshmi wr= ote: > Hi Filip, > > I tested both patches on current master using git am -3 .They apply > cleanly,build fine,and the temp_file _compression GUC works as expected. > Query results are unchanged. > > For hash join spill test,temp files were created as expected,but the > logged size were same for no,lz4,and pglz,which seems consistent with > fixed-size fileset chunking.It might be helpful to briefly note this in t= he > documentation to avoid confusion. > > Thanks for working on this . > best regards, > lakshmi > > On Tue, Jan 20, 2026 at 4:10=E2=80=AFAM Zsolt Parragi > wrote: > >> Hello! >> >> I tried to review the code. It compiled, the test suite passed. >> >> I noticed two typos: >> >> buffile.c:77 - "Disaled" >> buffile.c:133 - "mathods" >> >> And a few other small findings: >> >> buffile.h:35 and buffile.c:63 - same constants defined first as an >> Enum and then as #defines - code builds properly without the defines. >> >> buffile.c:121 - compress_tempfile is defined, set to false at :167, >> but never used otherwise >> >> guc_tables.c:470 - the comment says that pglz isn't supported yet, but >> we have a value for it, and I see support for it in the code >> >> buffile.c:659: (and at other places) if USE_LZ4 is undefined, the >> codepath doesn't do anything. I think these ifdefs should follow how >> other compression code works, such as wal compression where there's an >> #else path with elog(ERROR, ...) >> Similarly, maybe there should be an explicit TEMP_NONE_COMPRESSION >> branch that does nothing, and the default branch should be an error? >> >> buffile.c:265: If seek isn't supported/limited, shouldn't there be at >> least an assertion about it in BufFileSeek? And tell isn't mentioned, >> but it seems to me that tell also doesn't work properly. >> > --00000000000085a7b60648e01011 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
HI all,
While testing the temp file compression patch,n= oticed that the new temp_file_compression GUC isn't documented yet.I pu= t together a small docs patch to add a short description and clarify that t= he effect=C2=A0of compression depends on the workload(for example ,hash joi= n spills may not show visible size reduction due to fixed_size chunks).
=
patch is attached.Happy to adjust the wording if needed.
thanks,
= lakshmi

On Tue, Jan 20, 2026 at 4:21=E2=80=AFPM laksh= mi <lakshmigcdac@gmail.com= > wrote:
Hi Filip,

I tested both patches on current master using= git am -3 .They apply cleanly,build fine,and the temp_file _compression GU= C works as expected.
Query results are unchanged.

For hash join s= pill test,temp files were created as expected,but the logged size were same= for no,lz4,and pglz,which seems consistent with fixed-size fileset chunkin= g.It might be helpful to briefly note this in the documentation to avoid co= nfusion.

Thanks for working on this .
best regards,
lakshmi
On = Tue, Jan 20, 2026 at 4:10=E2=80=AFAM Zsolt Parragi <zsolt.parragi@percona.com>= ; wrote:
Hello!<= br>
I tried to review the code. It compiled, the test suite passed.

I noticed two typos:

buffile.c:77 - "Disaled"
buffile.c:133 - "mathods"

And a few other small findings:

buffile.h:35 and buffile.c:63 - same constants defined first as an
Enum and then as #defines - code builds properly without the defines.

buffile.c:121 - compress_tempfile is defined, set to false at :167,
but never used otherwise

guc_tables.c:470 - the comment says that pglz isn't supported yet, but<= br> we have a value for it, and I see support for it in the code

buffile.c:659: (and at other places) if USE_LZ4 is undefined, the
codepath doesn't do anything. I think these ifdefs should follow how other compression code works, such as wal compression where there's an<= br> #else path with elog(ERROR, ...)
Similarly, maybe there should be an explicit TEMP_NONE_COMPRESSION
branch that does nothing, and the default branch should be an error?

buffile.c:265: If seek isn't supported/limited, shouldn't there be = at
least an assertion about it in BufFileSeek? And tell isn't mentioned, but it seems to me that tell also doesn't work properly.
--00000000000085a7b60648e01011-- --00000000000085a7b90648e01013 Content-Type: text/x-patch; charset="US-ASCII"; name="doc-temp-file-compression-doc.patch" Content-Disposition: attachment; filename="doc-temp-file-compression-doc.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_mknnc46m0 ZGlmZiAtLWdpdCBhL2RvYy9zcmMvc2dtbC9jb25maWcuc2dtbCBiL2RvYy9zcmMvc2dtbC9jb25m aWcuc2dtbAppbmRleCAwZmFkMzRkYTZlYi4uNTdhOGFmMmEyZmMgMTAwNjQ0Ci0tLSBhL2RvYy9z cmMvc2dtbC9jb25maWcuc2dtbAorKysgYi9kb2Mvc3JjL3NnbWwvY29uZmlnLnNnbWwKQEAgLTE5 NTUsNiArMTk1NSwyNiBAQCBpbmNsdWRlX2RpciAnY29uZi5kJwogICAgICAgPC9saXN0aXRlbT4K ICAgICAgPC92YXJsaXN0ZW50cnk+CiAKKyAgICA8dmFybGlzdGVudHJ5IGlkPSJndWMtdGVtcC1m aWxlLWNvbXByZXNzaW9uIj4KKyAgICAgPHRlcm0+PHZhcm5hbWU+dGVtcF9maWxlX2NvbXByZXNz aW9uPC92YXJuYW1lPiAoPHR5cGU+ZW51bTwvdHlwZT4pPC90ZXJtPgorICAgICA8bGlzdGl0ZW0+ CisgICAgIDxwYXJhPgorICAgICAgRW5hYmxlcyB0cmFuc3BhcmVudCBjb21wcmVzc2lvbiBvZiB0 ZW1wb3JhcnkgZmlsZXMgdXNlZCBieSBxdWVyeSBleGVjdXRpb24uCisgICAgICBTdXBwb3J0ZWQg dmFsdWVzIGFyZSA8bGl0ZXJhbD5ubzwvbGl0ZXJhbD4sIDxsaXRlcmFsPmx6NDwvbGl0ZXJhbD4s IGFuZAorICAgICA8bGl0ZXJhbD5wZ2x6PC9saXRlcmFsPi4KKyAgICAgPC9wYXJhPgorCisgICAg IDxwYXJhPgorICAgICBUaGUgZWZmZWN0aXZlbmVzcyBvZiB0ZW1wb3JhcnkgZmlsZSBjb21wcmVz c2lvbiBkZXBlbmRzIG9uIHRoZSB3b3JrbG9hZC4KKyAgICAgRm9yIGV4YW1wbGUsIHRlbXBvcmFy eSBmaWxlcyBjcmVhdGVkIGJ5IGhhc2ggam9pbiBzcGlsbHMgdXNlIGZpeGVkLXNpemUKKyAgICAg Y2h1bmtzLCBzbyBvbi1kaXNrIGZpbGUgc2l6ZXMgbWF5IG5vdCB2aXNpYmx5IHNocmluayBldmVu IHdoZW4gY29tcHJlc3Npb24KKyAgICAgaXMgZW5hYmxlZC4gU3RhdGlzdGljcyBzdWNoIGFzIDxs aXRlcmFsPnRlbXBfYnl0ZXM8L2xpdGVyYWw+IHJlcG9ydCBsb2dpY2FsCisgICAgIGJ5dGVzIHdy aXR0ZW4gYmVmb3JlIGNvbXByZXNzaW9uLgorICAgICA8L3BhcmE+CisgICAgIDwvbGlzdGl0ZW0+ CisgICAgIDwvdmFybGlzdGVudHJ5PgorCisKICAgICAgPHZhcmxpc3RlbnRyeSBpZD0iZ3VjLWhh c2gtbWVtLW11bHRpcGxpZXIiIHhyZWZsYWJlbD0iaGFzaF9tZW1fbXVsdGlwbGllciI+CiAgICAg ICA8dGVybT48dmFybmFtZT5oYXNoX21lbV9tdWx0aXBsaWVyPC92YXJuYW1lPiAoPHR5cGU+Zmxv YXRpbmcgcG9pbnQ8L3R5cGU+KQogICAgICAgPGluZGV4dGVybT4K --00000000000085a7b90648e01013--