Date: Wed, 21 Jun 2023 10:04:28 -0400
From: Sean O'Grady <sean@thingee.com>
To: Scott Ribe <scott_ribe@elevated-dev.com>
Cc: "pgsql-admin@lists.postgresql.org" <pgsql-admin@lists.postgresql.org>,
 "Wetmore, Matthew (CTR)" <matthew.wetmore@express-scripts.com>
Message-ID: <1c92a69086aa7046c2141a3849b1eff92eb68a37@hey.com>
In-Reply-To: <CDC433EB-E5B0-46CC-AD36-4957419B517C@elevated-dev.com>
Subject: Re: Question about wal_compression and what to expect
Content-Type: multipart/alternative;
 boundary="--==_mimepart_6493036ce824d_1551d108111b";
 charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?R24zaFpSRXZiaGpCSDdQNlVZQ3pOSzZ1aXZua2llZFRqdnlRdEJ5ZG1nczF2?=
 =?utf-8?B?elF3VTFFeFUxNFlsZlJQQW9oTjUvcGV4ZHBhamhDRzZDNzQ5TURocVlXMEN6?=
 =?utf-8?B?N2FuNGFRa2pBR2FONmtwZTNCSkxBd0V0bHBmQ1kyNjBCMlRLRDlxMlFoWHdX?=
 =?utf-8?B?alR5QkJLM1AvMXJBVzlhbmVMelJVaEJvQUplSlBqT0ZrUUdaa1N2WENCMDN3?=
 =?utf-8?B?NXl4WHlyWXdVMkk4TEp5enZoVHJmaCs2RDN5K25YYjh2Nnc5TFdEV1hoWDRT?=
 =?utf-8?B?S2xDSkF1dzRIT1FwaHB2Y0V4UDJiM2dkTXBmQkJjTU5YSVl1Rkk5aWJaV3dP?=
 =?utf-8?B?Mi9LUGYydGpuRytqUUwwbzNqMG9YWVA5NTJYMDhCaWtjaG13WUJBNUp4Wjk2?=
 =?utf-8?B?dUNqUUY0T01sQ2ZsVHFoR3JIN0VMejAzTEN5a2lMdGs5UFZaWDVBU2hldUJs?=
 =?utf-8?B?TnRnRDR3ZUV4SDZ5UDJ2SlFyNThleERWTkZwd1NjNis0SGsvS01HK04rdWNR?=
 =?utf-8?B?YnBubXhQTytPeUFMMHljeW43QjUxck13QU41VXNSNlVNamNqN2dGQXBhbllL?=
 =?utf-8?B?aXVOWlNqTjcrS3lvN3NsY1dNemM0RkVoUFlKV2ZzazNLL3psMTU3ZG5kTUtn?=
 =?utf-8?B?eC9CS2JXZDVMakJjUjlxLzBjZ2ZkeUQ5d29MODFZVWFRaW1pdzIzRnZxamlU?=
 =?utf-8?B?N0pjaGd5M2lRVTdYZUFxWXVEcmlFeis0bWs3eVhJODNVenR5QXJzY3FWZ2Ry?=
 =?utf-8?B?U0NkZ3BYZHRTTytVZUpMWVRLM2hYR0trR0M4cy8xQlBrWEZZbCtCYjJ5RWk0?=
 =?utf-8?B?R3BlSlgvMWRhK2QwSnR0enByVy93b2xiOE1Sb1dubHRGVXEvSUMwLzZmTEJ3?=
 =?utf-8?B?bm53dDd6azBpRmxDZ3B6eXNxVkNjYlhDWHhTWlorYTZvQ25tclZiQ2lWb0Mx?=
 =?utf-8?B?a1BzVk1HS2JacVJBekoyOTdSMVY0S3QxekFJOVZhUWFmWnYrenFBcXZBRWdX?=
 =?utf-8?B?ODdvZ21kWjNNYjZaaFJYcFJ0c3A4T291NmZrL2RXc0c0Z2JuWTZ1M2toNHh5?=
 =?utf-8?B?QnFkc0NWYU9YS3ZoWmFwbmg1c3dneTM4cmJDVWxSbFNzTmlrZnNUcFE0cmpz?=
 =?utf-8?B?ampJVkhCM2d0NFcyaWRoWWcrR1ZleTRCVm42WStBLzZES0VnQU1VbzBxbndt?=
 =?utf-8?B?QTFvNjZ2eWttNUxNbk5oUGdVdDdDdTBTQzdHYld4Wll3Z0ZCbHN5MHVkQ00z?=
 =?utf-8?B?QnZRL2tENm1Dc2FmZmJFK0tHODVJVDBGUko0VlFMVnQzT04vbWc5MC81THBZ?=
 =?utf-8?B?YkZITmhXMktWWUdmYzI2cUlBZVA2TEJ4eXlNbEJ5S2NDNGxJemN6MFI2NW9H?=
 =?utf-8?B?bGQ5V0d0TW1JVi9xcmJ0UW1ZcFo1WHBKWm1mZ0o3eHdsVFpVdDk3UTZ0TEgx?=
 =?utf-8?B?Q1dyN2lYTWtUVUxNZnpWOVFHbFFORGJRYWZGYmdCQnppQUJZTFhKVHFTQXZw?=
 =?utf-8?B?em5UMUR0cmFrUGkwODM3N0dnRzFMVVZpRTZydmpDSmw3VE1kSWo4Sis3SjRD?=
 =?utf-8?B?YVpsS0IyVTNVZnBnMVF5MUNtblBwVlA4RGw2Ly9SV05qY2hvMjJMQlEwNXRp?=
 =?utf-8?B?bVYxOHBLYXZRZ2dST0hURXNWcGhoLzUyL1I3UEVTcTJ3UTZVeEJFM3BDS2R5?=
 =?utf-8?B?K0E2dnkzZ1A0ZVl6R0tzNXUzOVhtdzhlNnl0SUhhZFNvdlFYUStGSUkzTXI2?=
 =?utf-8?B?dzUrREFoejlReGlydldUYSsxVW9uN0tNTXRsZ0t4RTE0Uzl6WEdVNzRGUm1D?=
 =?utf-8?B?N1Z6b21TeDlSYVhHUmc4VmRyVW1ZRkJSZk9tcDJIdFpPbmIxZkVzcmlkQ0Fu?=
 =?utf-8?B?YXlxK2NoR0ZoNzhPblN4cVhpeWtBQnV5Z3N4dHVwVjJoUFUvYW85NWtqNHBS?=
 =?utf-8?B?Sk5mWUp4NzNOTmhkT3B3MGplU3M3S1NRZWk4RmxsWjZScXVsUUJDVThXOVFM?=
 =?utf-8?B?amhnUzB4Uy9ZTEt0Sm0zMHpSZEs3TE9BZzVUZ2c5SWljeS9WcEMzU2pya3pn?=
 =?utf-8?B?NE54emlwZWRwV1BtZG1NRktaU2txRHF0ZWhMMDk1U3RVT1pKaXltczRGeFZs?=
 =?utf-8?Q?UNR2bJod3htmojyVs8PihieyL?=
X-OriginatorOrg: thingee.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 
 cee21e79-019d-45f4-e9d7-08db72606ea5
X-MS-Exchange-CrossTenant-AuthSource: MN2PR19MB2782.namprd19.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2023 14:04:30.2954
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: afe33c34-7512-429f-b00e-f8690f958841
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 
 NfVVPDTuomejgJaZ75CMcakXV/k1rN2lsaEJ4A5WqdagdJ7kbhf2K3sjlEIt6Dt+s4j1+PRUOvlyjNeNHu0/Lw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR19MB3950
List-Id: <pgsql-admin.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-admin@lists.postgresql.org>
List-Owner: <mailto:pgsql-admin-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-admin>
Archived-At: 
 <https://www.postgresql.org/message-id/1c92a69086aa7046c2141a3849b1eff92eb68a37%40hey.com>
Precedence: bulk

----==_mimepart_6493036ce824d_1551d108111b
Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: quoted-printable

With archive_timeout set I think WAL files will often (at least in my
case) be created before they are filled, and in those cases there should
be a larger benefit, correct?

Thanks


On June 21, 2023, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
> > On Jun 21, 2023, at 7:51 AM, Sean O'Grady <sean@thingee.com> wrote:
> >=C2=A0
> > Yes, my question was essentially do I need to separately compress
> WAL files for storage/archiving even with wal_compression set. So, I
> think you've answered my question.
>
> If WAL files use per-page compression internally, you won't get much
> benefit from compressing them. If they use lz4 internally, then sure,
> bzip'ing them will get a little benefit, but nowhere near the normal
> amount.

----==_mimepart_6493036ce824d_1551d108111b
Content-Type: text/html;
 charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8">
   =20
    <meta name=3D"viewport" id=3D"viewport" content=3D"width=3Ddevice-width=
,minimum-scale=3D1.0,maximum-scale=3D10.0,initial-scale=3D1.0">

    <style>
      html{-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}h1{font-=
size:1.3em;line-height:1.2;margin:0}ul,ol{margin:0;padding:0}ul li,ol li,li=
 li{margin:0 0 0 36px}[dir=3Drtl] li{margin:0 18px 0 0}blockquote{border-co=
lor:#dfdee1;border-style:solid;border-width:0 0 0 1px;margin:0;padding:0 0 =
0 1em}[dir=3Drtl] blockquote,blockquote[dir=3Drtl]{border-width:0 1px 0 0;p=
adding:0 1em 0 0}pre{font-family:"SFMono-Regular",Consolas,"Liberation Mono=
",Menlo,Courier,monospace;font-size:.9em;margin:0;padding:1rem;background-c=
olor:#f6f5f3;white-space:pre-wrap;word-wrap:break-word;overflow:visible}.me=
ssage-content{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Robot=
o,Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI=
 Symbol";line-height:1.4}.attachment{display:inline-block;margin:0;padding:=
0}.attachment__caption{padding:0;text-align:center}.attachment__caption a[h=
ref]{text-decoration:none;color:#333}.attachment--preview{width:100%;text-a=
lign:center;margin:.625em 0}.attachment--preview img{border:1px solid #dfde=
e1;vertical-align:middle;width:auto;max-width:100%;max-height:640px}.attach=
ment--preview .attachment__caption{color:#716d7b;font-size:.85em;margin-top=
:.625em}.attachment--file{color:#282138;line-height:1;margin:0 2px 2px 0;pa=
dding:.4em 1em;border:1px solid #dfdee1;border-radius:5px}.permalink{color:=
inherit}.txt--xx-small{font-size:14px}.flush{margin:0;padding:0}.push--bott=
om{margin-bottom:8px}.border--top{border-top:1px solid #ece9e6}.btn{padding=
:.2em .4em;font-weight:500;text-decoration:none;border-radius:3rem;white-sp=
ace:nowrap;background:#5522fa;border-color:#5522fa;color:#fff}.btn--email{d=
isplay:inline-block;text-align:center;font-weight:500;font-size:1em;text-de=
coration:none;border-radius:2em;white-space:nowrap;background:#5522fa;borde=
r-color:#5522fa;color:#fff;border-top:.3em solid #5522fa;border-left:1em so=
lid #5522fa;border-bottom:.3em solid #5522fa;border-right:1em solid #5522fa=
}.shaded{padding:1em;border-radius:4px;background-color:#f6f5f3;border:1px =
solid #dfdee1}

    </style>
  </head>

  <body>
    <div class=3D"message-content">
      <div class=3D"trix-content">
  <div>With archive_timeout set I think WAL files will often (at least in m=
y case) be created before they are filled, and in those cases there should =
be a larger benefit, correct?<br><br>Thanks<br><br><br></div><div>On June 2=
1, 2023, Scott Ribe &lt;scott_ribe@elevated-dev.com&gt; wrote:</div><blockq=
uote>&gt; On Jun 21, 2023, at 7:51 AM, Sean O'Grady &lt;sean@thingee.com&gt=
; wrote:<br>&gt;&nbsp;<br>&gt; Yes, my question was essentially do I need t=
o separately compress WAL files for storage/archiving even with wal_compres=
sion set. So, I think you've answered my question.<br><br>If WAL files use =
per-page compression internally, you won't get much benefit from compressin=
g them. If they use lz4 internally, then sure, bzip'ing them will get a lit=
tle benefit, but nowhere near the normal amount.</blockquote>
</div>


    </div>
  </body>
</html>

----==_mimepart_6493036ce824d_1551d108111b--