Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qByF4-0005nq-2Y for pgsql-admin@arkaria.postgresql.org; Wed, 21 Jun 2023 13:51:26 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1qByF2-0007LW-Qm for pgsql-admin@arkaria.postgresql.org; Wed, 21 Jun 2023 13:51:24 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qByF2-0007Kl-3E for pgsql-admin@lists.postgresql.org; Wed, 21 Jun 2023 13:51:24 +0000 Received: from mail-co1nam11on2047.outbound.protection.outlook.com ([40.107.220.47] helo=NAM11-CO1-obe.outbound.protection.outlook.com) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qByEy-003RIB-UM for pgsql-admin@lists.postgresql.org; Wed, 21 Jun 2023 13:51:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lD4yIGjsfov8rS+eYGexEOr/aFf76E5IqZPgJfhKZbBcHBti+EAg+01/3+JCEmQ1XFSD56lPUBzB74s+R7AXSeNNOi+GAdL4YZGbUpOHxqMwDnnPHpeh+Kh6YfJ/D5Y93EuRzZ4kkB1cYEUYSoF56j1VEpwWlkNoEvQTHNnNi3ymmF2RVH/KB9NGNJXC6nM5Pngp5n9qiA8VDeeTtRbWqcrc9S2j+Y/5rGLAVlIBVtk5XU5D4iOddpDZza3w1TLVKJa2wWTSWBf08ftgiJKGAvQR8g900KxtYDuf3eGjg7dhVTBHXW7Oql4YKXMQstZZb8etkH13ZlxCqBeQKMRuCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3JV0SaD3X9uAbwzpNMzyQy/tlUAdmgT2Y3U1aLtSSpM=; b=cGwdSkyurpGxS5l83ERJSPoeZIO59YhjLoclJyoRX+px0ILJqbm4lgluCz1U+1JiU9H7PB8IgVgbhKgIkvl1NCI7BN1q1kgtKg59eZbqVmyxBDH+cl4+OApJe3TJ94MrpeQhHqRD43/osOnlIDqKf6hnej3L5+R6VUHT02WGsMi6v5hdXrYD4VvLy3bVhYg+Di41I9GCztq3FZJWDt56Br//GY/ujV9h7p05andAeVfjBPhNYR89rjnLUWUzEGkQxU+vdfYxzuB0pEnGLnPIgn6IfWuZJ/6hi09AD4xapLBIQ3PeG9K0DCD07i1jgrUWv1ex/osNB149wVT6FhbzMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=thingee.com; dmarc=pass action=none header.from=thingee.com; dkim=pass header.d=thingee.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=thingee.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3JV0SaD3X9uAbwzpNMzyQy/tlUAdmgT2Y3U1aLtSSpM=; b=kDKD6HhPHFPgbtMJolhcoPdj1sVRnrfWVt7yh1J9dkgIP8LqsWSGrepYjNVhPBn8mvQNMgU9V++P1Pgtu3c8hZWBCXFGDU+dRYAlGlQyVngVGfs2B4/DEfMqhRlmdo+C6IXm0Yjc+oT72GQLC7VsHS/S3XWXVffWBaHgboKfzwA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=thingee.com; Received: from MN2PR19MB2782.namprd19.prod.outlook.com (2603:10b6:208:fa::31) by MW6PR19MB8038.namprd19.prod.outlook.com (2603:10b6:303:23f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.21; Wed, 21 Jun 2023 13:51:17 +0000 Received: from MN2PR19MB2782.namprd19.prod.outlook.com ([fe80::678b:a7ad:523c:b4cf]) by MN2PR19MB2782.namprd19.prod.outlook.com ([fe80::678b:a7ad:523c:b4cf%4]) with mapi id 15.20.6521.020; Wed, 21 Jun 2023 13:51:17 +0000 Date: Wed, 21 Jun 2023 09:51:15 -0400 From: Sean O'Grady To: "pgsql-admin@lists.postgresql.org" , "Wetmore, Matthew (CTR)" Message-ID: In-Reply-To: Subject: Re: Question about wal_compression and what to expect Content-Type: multipart/alternative; boundary="--==_mimepart_64930053ecc48_381d1053722"; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: BL1PR13CA0368.namprd13.prod.outlook.com (2603:10b6:208:2c0::13) To MN2PR19MB2782.namprd19.prod.outlook.com (2603:10b6:208:fa::31) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR19MB2782:EE_|MW6PR19MB8038:EE_ X-MS-Office365-Filtering-Correlation-Id: b6ad2f10-7438-43a0-fec1-08db725e95da X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HOpWH3crgSWYcm5QNVYPfsifE7fWKrSrCCFA2+qJZbI/S9jlTA9wAdqIWQz+qNMcxENjROjb0c4Uf0Hp5nziFgobi1fZyyWuvz0sL1VAroG8yEBaSQz4baO8yEVPdDaXCVnJVDqrtrplElbwtqaaA/jbykwpBHkV54IGeRmTKFy/OJynLsDIepcEsfqemUOr0Oom147bPX+IvA8zZMbWPggx/sABVBsCpRqYV+quMsIsekPIJW6dJG0iGBy2Mub1YHCNMiufz2PfrSt+y6P7RNadtJKLYP9FT/GWlqzE3GpQU2M7DwagvH76/ViRD/6pSqOSTwLEajezxyJvrH2VQHmWZJ0t+uqYbKsggEDiuJPUY4RV2/Tw9c9NorrTH1ww8pEX2QHuXhZOd8aoz5GEnUa95d+NRShRq1wylC2uorWdrrrHD1BXil9weicsUxbIudiBL+DaWqhp2DF9SuPe53rAQb1Yqx02OqxlITc63KGX09X4OOE55Vhkvu54FcgzlaaSaus1BecSUpdzcABpSCjAKrrO/cREHmjA6A+Z1wzA9l4Wc2Kcok0fCfW9FH1xgweeif1/M8MoG5Zj4Gu6tci49f2FpuIbMggoAs13Nh8tP1hZP96hOYn9S4ZL7nLyRZt0gTYLor7Ab7+OXz0tTA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR19MB2782.namprd19.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(396003)(39830400003)(136003)(376002)(366004)(346002)(451199021)(38350700002)(478600001)(8936002)(8676002)(38100700002)(110136005)(6486002)(33964004)(52116002)(36756003)(66476007)(86362001)(2906002)(83380400001)(66946007)(186003)(66556008)(26005)(6512007)(9686003)(53546011)(6506007)(316002)(41300700001)(5660300002)(43580500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TzFMMHNRUHFLQ2k2TCtHRDg3K0oxb1RuSWpXMVpROTVuWnBrc3F1N1pxSlhS?= =?utf-8?B?QUlHRFRvdEhla3ErQnUzdkk0OTBUTXdVOWJWRlA2R3JjQ0VINnp3QTBkTVZ6?= =?utf-8?B?V2drTlhVcExSWDBMU1N0R1dISzVTWHNBb3BZdy9ON2t2dmF0RUowcnA5cGFQ?= =?utf-8?B?R1I4Ynl1K1N2am56VUNrQkxSTGNpczdLb0FTR0tYaGpkczk1dlNKQkI3SENE?= =?utf-8?B?NkJJQjZNMldMSGRZeDRZSjBRTTIrTnNkcDhrMDcwQjI0NjlTUU9wMUQ3ZDAx?= =?utf-8?B?Y0xySGhML3o2VG8zVWRVdm5hSGgxUUNoTGJ5TWo1dXRqUlJTZTNIT3RMbDRy?= =?utf-8?B?K25QNFNLQ1RrcmdrZkVFcmZtL2pFaVFWM2I3anRpZjRSaUxHY1FiWU45Z1JV?= =?utf-8?B?d0xqMThmYjRadFdwZkdTSHdoNEtTNis1eGlUQUN5YWg5RWtraStUODlvREJ3?= =?utf-8?B?cnpZSklxQlExNERGK2hGSHNDNUp4bkdJT2Q0Z0FXWlRuWE55di8rdHRKTWNG?= =?utf-8?B?d3REb0hJTnJpcnFRcWJKWFkwVERiV01QaW9zMXVBTVRtTVBmbjA4LzFsN2c4?= =?utf-8?B?cTRyVXFtMzlZYXFlWWNzZW5sZkdITWwwT0NnOUV3WlFaSjVpa2xtMlZJTGRw?= =?utf-8?B?YjByWXJwTDhMR3VJRjBOekRUSG84YzJvbjZTdUdTUzZpK05kVjZscTVUdHBD?= =?utf-8?B?L3NtSDJ3UmhsNmF4RmZTMEhVTno3cHE0d2IxOE5IT3ZWVWhhMTFVVExZQStC?= =?utf-8?B?aUVleHFCOGdDV1VKUlBhUEhzeFl0SExKVU8wc293WGRQang4eVhTcDNIK0N5?= =?utf-8?B?RThTZGRma0JJUDN6YXc1U1ltY2IrNSs4dFAyRGRyZklkRDl3WTBwYVFhbnFT?= =?utf-8?B?Q3NLMUpkSWpDQlZESTZUck4rQ1RGYStGS003TnhOLys3eHZEM0o5ZTJZSXZI?= =?utf-8?B?VmYrTmRGNXI4eFM1M292OGZ3dUNTQ2hzekdTV1AvVUxvdVlnUFBaa1NINGpB?= =?utf-8?B?c3FrOTNBVysvVmN4eVBoOWorajE0Q1JlUitER2dqbGxZR0YyRlpzZjBuTm56?= =?utf-8?B?TVFuZU9sdFJqc0pEZDJMblpzZldscGhhamwzazdnbFlQRiswK3RsME9kTVAr?= =?utf-8?B?QklYSmVnRkdDSmd3UnZLcERYVUthdGRmRzV5ZW01cGVvUTlKNG84QmZRWEhP?= =?utf-8?B?dDlZMDRUTlZUOG03ZTRlcTVUY216Q3NRVHBFZ0Y5TmZHckJ4akd3K0g3T0pm?= =?utf-8?B?Tk9LSHAxUWlBVnBQbU03bjVKOU1aQUlxM0F2QWlBcFFEK09pcTA3YTZUT0g5?= =?utf-8?B?eWdmMk9ac0VaU2EzQ2UwWmFTS2huMW1oWGNxSHYyRzV3ZnBRYUIzRG16NXpL?= =?utf-8?B?TFY5bFR3QWFUcFhIeVNEYUpaWkVYb2gwcHdlMWozMnRSNmtoRjl1T1RrcEh3?= =?utf-8?B?eVVTbk1DRm9BcEoza3hiUjdwMUt1SXFwTmJ5a1JQUXRZZk03NDVMRFBrQlUv?= =?utf-8?B?aWpyRkdkT0RtSzZoMkd6Q0RPQ3N2aVhhUU9QYytMdmVRaklxU1NUQUFCSSsz?= =?utf-8?B?MEZwWGtvNDVSTS9yUG5rOWlwVWkrTzlOaXppeTI2am5LeGRHUXdsWXQ3ZkE2?= =?utf-8?B?eThUV2xpWW9DcklUeW1WYkwzc0JUeG15Uk9iRmE4OTg4UlBOWGh1L3o2RTN1?= =?utf-8?B?ZGt4NitWQy9MYlRDWXJjMk8wNFZxWnhHb3ArMjVLR2VIa0VoYU01czFJV1RE?= =?utf-8?B?UWd0VUdCdGZCRENMTFVHTmZUVWRjc2trajZuK2V3WEM3R251R1NDMVNYVW9x?= =?utf-8?B?QTNSOTA1V3NkeFhQbHlHajVsSEp6ZEdKSndCVUxUbjk4dXBYR0w0ZWxWcjhI?= =?utf-8?B?dHFHRkxNa0JCSnQ3YUNDNGNhQ1hXdkVGSjU2TUtKMSs5QlRRekdyN2dXUzYw?= =?utf-8?B?cndRRCtkTndkVzFjRzlkUTJHVHRBeG9sdFdSVmlyVWNva2tZcklobzFOUTh5?= =?utf-8?B?bVhOaDZsSlNQWURIeWxwS0NPSDRNRXdScFMxOWlaTnlEU3ZwMXQyTmJHbVFk?= =?utf-8?B?dnBYNUVHOFZJRk5xbElsdjgyMVpreE12RFcrUWk2aHlHSnNHaUFFZ2RxZGoy?= =?utf-8?Q?EJmmZmP+bHQTH4rOLgTqEmMVl?= X-OriginatorOrg: thingee.com X-MS-Exchange-CrossTenant-Network-Message-Id: b6ad2f10-7438-43a0-fec1-08db725e95da X-MS-Exchange-CrossTenant-AuthSource: MN2PR19MB2782.namprd19.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2023 13:51:17.1968 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: afe33c34-7512-429f-b00e-f8690f958841 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mwMtNGpl8T0sBLiXg0KOAdU4q8ijtQLUWs7PzZqXj9jqWJ/A59uz7W8ykBpZx7/xDcLxZC7wOHAJHezLobloCw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR19MB8038 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk ----==_mimepart_64930053ecc48_381d1053722 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Yes, my question was essentially do I need to separately compress WAL files for storage/archiving even with wal_compression set. So, I think you've answered my question. Thanks! Sean On June 20, 2023, "Wetmore, Matthew (CTR)" wrote: > From my understanding, > > =C2=A0 > > Wal_compression will compress full page writes in the wal file.=C2=A0 T= he > size of your wal is configurable. > > =C2=A0 > > So for your 16MB wal, it=E2=80=99s already compressed into the wal, and= your > wal_size won=E2=80=99t change.=C2=A0 (just more data can fit into your = 16MB) > > =C2=A0 > > As far as archiving, I create a dir, then tar.gz the dir for storage. > (if that was your question) > > =C2=A0 > > =C2=A0 > > =C2=A0 > > From: Sean O'Grady > Sent: Tuesday, June 20, 2023 11:19 AM > To: pgsql-admin@lists.postgresql.org > Subject: [EXTERNAL] Question about wal_compression and what to expect > > =C2=A0 > > Hello, > > I've enabled wal_compression however I do not see any change in the > 16MB size for each WAL file. Through all my googling and searching the > mailing lists, I could not find a description if a change *should* be > expected even though many of the articles I've read indicate > compression can save disk space.=C2=A0 > > I am currently thinking that the records in the WAL file are > compressed for the purpose of writing less data to disk, but that the > file is still padded to 16MB. If so, does that mean to realize a space > savings for archiving that my archive_command should compress the file > (gzip, bz2, etc)? > > Thanks, > Sean O'Grady > ----==_mimepart_64930053ecc48_381d1053722 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =20
Yes, my question was essentially do I need to separately compress WA= L files for storage/archiving even with wal_compression set. So, I think yo= u've answered my question.

Thanks!
Sean

On June 20, 2023, = "Wetmore, Matthew (CTR)" <matthew.wetmore@express-scripts.com&= gt; wrote:

From my understanding,

 

Wal_compression will compress full pa= ge writes in the wal file.  The size of your wal is configurable.

 

So for your 16MB wal, it=E2=80=99s al= ready compressed into the wal, and your wal_size won=E2=80=99t change. = ; (just more data can fit into your 16MB)

 

As far as archiving, I create a dir, = then tar.gz the dir for storage. (if that was your question)

 

 

 

From: Sean O'Grady <sean@thingee.= com>
Sent: Tuesday, June 20, 2023 11:19 AM
To: pgsql-admin@lists.postgresql.org
Subject: [EXTERNAL] Question about wal_compression and what to expec= t

 

Hello,

I've enabled wal_compression however I do not see any change in the 16MB si= ze for each WAL file. Through all my googling and searching the mailing lis= ts, I could not find a description if a change *should* be expected even th= ough many of the articles I've read indicate compression can save disk space. 

I am currently thinking that the records in the WAL file are compressed for= the purpose of writing less data to disk, but that the file is still padde= d to 16MB. If so, does that mean to realize a space savings for archiving t= hat my archive_command should compress the file (gzip, bz2, etc)?

Thanks,
Sean O'Grady

----==_mimepart_64930053ecc48_381d1053722--