Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jeEj4-0004yh-N5 for pgsql-docs@arkaria.postgresql.org; Thu, 28 May 2020 09:21:23 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jeEj3-0005HS-FG for pgsql-docs@arkaria.postgresql.org; Thu, 28 May 2020 09:21:21 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jeEj3-0005HK-3P for pgsql-docs@lists.postgresql.org; Thu, 28 May 2020 09:21:21 +0000 Received: from mail.postgrespro.ru ([93.174.131.139]) by makus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jeEiz-0001qS-M0 for pgsql-docs@lists.postgresql.org; Thu, 28 May 2020 09:21:20 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id 04A7D21C6B82; Thu, 28 May 2020 12:21:15 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-99 required=4 WHITELISTED tests=[] autolearn=unavailable Received: from mail.postgrespro.ru (unknown [93.174.131.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id B1F5321C6B72; Thu, 28 May 2020 12:21:14 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1590657674; bh=Sw8BZK9Nv8yeU/A5MpOFNER13x/HFGQE57oEj6QFSlw=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=I581bd6lQKDaOU99lr4VfPrDvOBO6MGy2801LXMbD8wfsBoAYxbVcLZUmt9G/5E4a Ww8HyhUY2+FMQ0mZ5uZ/ZI9LcwAbhUNLxTdjhtkCJ4Zla17NEWtNfbv9qD2AyVmUqu bILJLe7of9Z7ODpgTTzZ2JUn2CarB4C6b1yW183U= MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_c66796b561e1d1b16196b468ace8cb53" Date: Thu, 28 May 2020 12:21:14 +0300 From: p.luzanov@postgrespro.ru To: "David G. Johnston" Cc: pluzanov@postgrespro.ru, pgsql-docs@lists.postgresql.org Subject: Re: max_wal_size In-Reply-To: References: <159057129673.7025.17016799863173731176@wrigleys.postgresql.org> <92491c6d07a3216ef2caee8ef55c02b5@postgrespro.ru> <8aac88e2a22cd84e50ee1a2bcf3da9b7@postgrespro.ru> User-Agent: Roundcube Webmail/1.4.0 Message-ID: X-Sender: p.luzanov@postgrespro.ru List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --=_c66796b561e1d1b16196b468ace8cb53 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed David, > I think part of the issue is your operating with a mental model of > this contrary to the one the documentation was written for. I made this assumption(checkpoints triggered by size, checkpoint_completion_target=1) only for simplifying example. (Although there is a practical task: calculate the possible approximate size of all WAL files. This is especially important for versions prior to 11, where WAL files were stored for the previous checkpoint.) > In any case "grow to" means that it doesn't matter whether at the end > of the previous checkpoint 1gb or 9gb were in use - once the usage > goes "up to" 10gp the automatic checkpoint will trigger. If the word > "to" wasn't there your interpretation would be correct. Its a fair > argument to say such an important distinction shouldn't be placed on > the word "to" but it isn't wrong. Thank you for an explanation. But give me another chance to explain. Now we have the glossary. According to the glossary, a checkpoint has two meanings. The first meaning is that this is a point: "A point in the WAL sequence at which it is guaranteed that the heap and index data files have been updated with all information from shared memory modified before that checkpoint; a checkpoint record is written and flushed to WAL to mark that point." Simple illustration: A B C WAL |-----------|-----------|-----------|-----------> Points A,B,C are records in the WAL. The second meaning is that this is an action (performed by checkpointer process or CP): "A checkpoint is also the act of carrying out all the actions that are necessary to reach a checkpoint as defined above." Add a process to the illustration. Checkpoint_completion_target here is the default value of 0.5: A B C WAL |-----------|-----+-----|-----+-----|-----+-----> a b c CP +-----+ +-----+ +-----+ b e b e b e Checkpointer process for point A starts at a(b) and ends at a(e). At the end of the work, CP makes a WAL record(plus symbol on WAL line). Back to the max_wal_size description: "Maximum size to let the WAL grow to between automatic WAL checkpoints." As I understand now, this is true when we talk about the checkpoint as a process. "Maximum size to let the WAL grow to between a(b) and b(e), between b(b) and c(e), etc". But this not true when we talk about the checkpoint as a point. "Maximum size to let the WAL grow to between A and B". My concerns that someone reading the description of the parameter may confuse the two meanings of checkpoint: as a point and as a process. After all, this happened to me. ----- Pavel Luzanov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company --=_c66796b561e1d1b16196b468ace8cb53 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8
= David,

I think part of the issue is your operating with a men= tal model of
this contrary to the one the documentation was written fo= r.

I made this assumption(checkpoints triggered by size, checkpoint_comp= letion_target=3D1) only for simplifying example.
(Although there is a = practical task: calculate the possible approximate size of all WAL files. T= his is especially important for versions prior to 11, where WAL files were = stored for the previous checkpoint.)

In any case "grow to" means that it doesn't matter whe= ther at the end
of the previous checkpoint 1gb or 9gb were in use - on= ce the usage
goes "up to" 10gp the automatic checkpoint will trigger= =2E  If the word
"to" wasn't there your interpretation would be c= orrect.  Its a fair
argument to say such an important distinction= shouldn't be placed on
the word "to" but it isn't wrong.

Thank you for an explanation. But give me another chance to explain= =2E

Now we have the glossary. According to the glossary, a check= point has two meanings.

The first meaning is that this is a poin= t:
"A point in the WAL sequence at which it is guaranteed that the hea= p and index data files have been updated with all information from shared m= emory modified before that checkpoint; a checkpoint record is written and f= lushed to WAL to mark that point."

Simple illustration:
&nb= sp;            =    A          &= nbsp;B           C
W= AL |-----------|-----------|-----------|----------->

Points A= ,B,C are records in the WAL.

The second meaning is that this is = an action (performed by checkpointer process or CP):
"A checkpoint is = also the act of carrying out all the actions that are necessary to reach a = checkpoint as defined above."

Add a process to the illustration= =2E Checkpoint_completion_target here is the default value of 0.5:
            =     A         &= nbsp; B           C<= br />WAL |-----------|-----+-----|-----+-----|-----+----->

&n= bsp;            = ;      a       =     b         &= nbsp; c  
CP         = ;     +-----+     +-----+ &nbs= p;   +-----+
       =          b    &= nbsp;e     b     e   &nbs= p; b     e

Checkpointer process for poi= nt A starts at a(b) and ends at a(e).
At the end of the work, CP makes= a WAL record(plus symbol on WAL line).

Back to the max_wal_size= description:
"Maximum size to let the WAL grow to between automatic W= AL checkpoints."

As I understand now, this is true when we talk = about the checkpoint as a process.
"Maximum size to let the WAL grow t= o between a(b) and b(e), between b(b) and c(e), etc".

But this n= ot true when we talk about the checkpoint as a point.
"Maximum size to= let the WAL grow to between A and B".

My concerns that someone = reading the description of the parameter may confuse the two meanings of ch= eckpoint: as a point and as a process. After all, this happened to me.

-----
Pavel Luzanov
Postgres Professional: http:= //www.postgrespro.com
The Russian Postgres Company
--=_c66796b561e1d1b16196b468ace8cb53--