Feedback-ID: i76984098:Fastmail
Message-ID: <a1a6a993-454a-41bf-84d7-194418f791f8@aklaver.com>
Date: Wed, 17 Dec 2025 08:31:34 -0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: wal segment size
To: Colin 't Hart <colinthart@gmail.com>,
 Laurenz Albe <laurenz.albe@cybertec.at>
Cc: PostgreSQL General <pgsql-general@lists.postgresql.org>
References: <CAMon-aQ8X7H7jCC6ggKm9Uzcu3KfGVC34RRqKt8BCfpDn2tq9A@mail.gmail.com>
 <2a301859ab47c2e83ba5c8ebf75112d61246a8cd.camel@cybertec.at>
 <CAMon-aSkPs=N5GHJsOzLs5RKcrba3=ng_6HUEKZy1=1GJgCEqg@mail.gmail.com>
Content-Language: en-US
From: Adrian Klaver <adrian.klaver@aklaver.com>
In-Reply-To: <CAMon-aSkPs=N5GHJsOzLs5RKcrba3=ng_6HUEKZy1=1GJgCEqg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/a1a6a993-454a-41bf-84d7-194418f791f8%40aklaver.com>
Precedence: bulk

On 12/17/25 08:10, Colin 't Hart wrote:
> Thanks Laurenz, that confirms what I was assuming. Archiving is via 
> pgbackrest to a backup server, over SSH. Approx 750ms to archive each 
> segment is crazy -- I'll check compression parameters too.

How much of that time is network travel?

What are the configuration settings for the archiving portion of pgBackRest?

> 
> Any reason not to bump it up to 1GB? Or is that overkill?
> 
> /Colin
> 
> On Wed, 17 Dec 2025 at 16:25, Laurenz Albe <laurenz.albe@cybertec.at 
> <mailto:laurenz.albe@cybertec.at>> wrote:
> 
>     On Wed, 2025-12-17 at 16:13 +0100, Colin 't Hart wrote:
>      > I see very little advice on tuning WAL segment size.
>      >
>      > One of my clients has a few datawarehouses at around 8 - 16 TB
>      >
>      > On one of the nodes there are approx 15000 WAL segments of 16MB
>     each, totalling
>      > approx 230GB. The archiver is archiving approx one per second, so
>     approx 4 hours to clear.
>      >
>      > Would we gain anything by bumping the WAL segment size?
> 
>     Very likely yes, if the problem is the overhead of starting the
>     archive_command.
> 
>     Another thing that can slow down archiving is if you compress these
>     segments
>     too aggressively.
> 
>     Yours,
>     Laurenz Albe
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com