pg.ddx.io pgsql-docs@postgresql.org mailing list archive
help / color / mirror / Atom feedformula about the number of WAL files
5+ messages / 3 participants
[nested] [flat]
* formula about the number of WAL files
@ 2010-10-14 03:00 Fujii Masao <masao.fujii@gmail.com>
2010-10-15 09:19 ` Re: formula about the number of WAL files Simon Riggs <simon@2ndQuadrant.com>
0 siblings, 1 reply; 5+ messages in thread
From: Fujii Masao @ 2010-10-14 03:00 UTC (permalink / raw)
To: pgsql-docs
Hi,
----------------
29.4. WAL Configuration
There will always be at least one WAL segment file, and will normally not
be more than (2 + checkpoint_completion_target) * checkpoint_segments + 1
or checkpoint_segments + wal_keep_segments + 1 files.
----------------
The above formula is wrong. The correct is
(2 + checkpoint_completion_target) * checkpoint_segments +
wal_keep_segments + 1
The attached patch fixes this fault.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Attachments:
num_of_wal_formula_v2.patch (application/octet-stream, 1.0K) download
*** a/doc/src/sgml/wal.sgml
--- b/doc/src/sgml/wal.sgml
***************
*** 450,457 ****
<para>
There will always be at least one WAL segment file, and will normally
! not be more than (2 + <varname>checkpoint_completion_target</varname>) * <varname>checkpoint_segments</varname> + 1
! or <varname>checkpoint_segments</> + <xref linkend="guc-wal-keep-segments"> + 1
files. Each segment file is normally 16 MB (though this size can be
altered when building the server). You can use this to estimate space
requirements for <acronym>WAL</acronym>.
--- 450,456 ----
<para>
There will always be at least one WAL segment file, and will normally
! not be more than (2 + <varname>checkpoint_completion_target</varname>) * <varname>checkpoint_segments</varname> + <xref linkend="guc-wal-keep-segments"> + 1
files. Each segment file is normally 16 MB (though this size can be
altered when building the server). You can use this to estimate space
requirements for <acronym>WAL</acronym>.
* Re: formula about the number of WAL files
2010-10-14 03:00 formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
@ 2010-10-15 09:19 ` Simon Riggs <simon@2ndQuadrant.com>
2010-10-15 10:44 ` Re: formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Simon Riggs @ 2010-10-15 09:19 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; +Cc: pgsql-docs
On Thu, 2010-10-14 at 12:00 +0900, Fujii Masao wrote:
> ----------------
> 29.4. WAL Configuration
>
> There will always be at least one WAL segment file, and will normally not
> be more than (2 + checkpoint_completion_target) * checkpoint_segments + 1
> or checkpoint_segments + wal_keep_segments + 1 files.
> ----------------
>
> The above formula is wrong. The correct is
>
> (2 + checkpoint_completion_target) * checkpoint_segments +
> wal_keep_segments + 1
>
> The attached patch fixes this fault.
The current docs are wrong, but your re-definition doesn't match the
actual code in CreateCheckPoint().
I've updated the docs to reflect how the code behaves.
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Training and Services
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: formula about the number of WAL files
2010-10-14 03:00 formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
2010-10-15 09:19 ` Re: formula about the number of WAL files Simon Riggs <simon@2ndQuadrant.com>
@ 2010-10-15 10:44 ` Fujii Masao <masao.fujii@gmail.com>
2010-10-15 13:48 ` Re: formula about the number of WAL files Simon Riggs <simon@2ndQuadrant.com>
0 siblings, 1 reply; 5+ messages in thread
From: Fujii Masao @ 2010-10-15 10:44 UTC (permalink / raw)
To: Simon Riggs <simon@2ndquadrant.com>; +Cc: pgsql-docs
On Fri, Oct 15, 2010 at 6:19 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> On Thu, 2010-10-14 at 12:00 +0900, Fujii Masao wrote:
>> ----------------
>> 29.4. WAL Configuration
>>
>> There will always be at least one WAL segment file, and will normally not
>> be more than (2 + checkpoint_completion_target) * checkpoint_segments + 1
>> or checkpoint_segments + wal_keep_segments + 1 files.
>> ----------------
>>
>> The above formula is wrong. The correct is
>>
>> (2 + checkpoint_completion_target) * checkpoint_segments +
>> wal_keep_segments + 1
>>
>> The attached patch fixes this fault.
>
> The current docs are wrong, but your re-definition doesn't match the
> actual code in CreateCheckPoint().
>
> I've updated the docs to reflect how the code behaves.
Hmm.. I seem to be very mixed up. Let me consider the formula again.
Basically checkpoint deletes the WAL files which are older than
the prior checkpoint's redo location. So at least WAL files which
were generated from the prior ckpt start to current ckpt end can
exist in pg_xlog directory. The number of those WAL files is
(1 + checkpoint_completion_target) * checkpoint_segments
But if this number is smaller than wal_keep_segments, checkpoint
leaves wal_keep_segments WAL files in pg_xlog instead. So, from
this aspect, the maximum number of WAL files in pg_xlog is
(1 + checkpoint_completion_target) * checkpoint_segments
or
wal_keep_segments
After checkpoint deletes old WAL files, the number of WAL files in
pg_xlog continues to increase until subsequent checkpoint will have
deleted WAL files. The increased number is
checkpoint_segments
Furthermore, one WAL file might be preallocated aside from the above.
As the result of the above, the maximum number of WAL files in
pg_xlog is
(2 + checkpoint_completion_target) * checkpoint_segments + 1
or
wal_keep_segments + checkpoint_segments + 1
So the original description seems to be correct. Am I missing
something?
# My previous proposal is definitely wrong. Sorry for noise.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: formula about the number of WAL files
2010-10-14 03:00 formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
2010-10-15 09:19 ` Re: formula about the number of WAL files Simon Riggs <simon@2ndQuadrant.com>
2010-10-15 10:44 ` Re: formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
@ 2010-10-15 13:48 ` Simon Riggs <simon@2ndQuadrant.com>
2010-10-28 01:30 ` Re: formula about the number of WAL files Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Simon Riggs @ 2010-10-15 13:48 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; +Cc: pgsql-docs
On Fri, 2010-10-15 at 19:44 +0900, Fujii Masao wrote:
> As the result of the above, the maximum number of WAL files in
> pg_xlog is
>
> (2 + checkpoint_completion_target) * checkpoint_segments + 1
>
> or
>
> wal_keep_segments + checkpoint_segments + 1
>
> So the original description seems to be correct. Am I missing
> something?
After some thought, I agree.
> # My previous proposal is definitely wrong. Sorry for noise.
I'd say the definition of wal_keep_segments isn't good, which is what is
causing the problem here. Will sort that out another day.
I will update the current code with some comments, so this doesn't
happen again. Will do this on Tuesday now, no time left here.
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Training and Services
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: formula about the number of WAL files
2010-10-14 03:00 formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
2010-10-15 09:19 ` Re: formula about the number of WAL files Simon Riggs <simon@2ndQuadrant.com>
2010-10-15 10:44 ` Re: formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
2010-10-15 13:48 ` Re: formula about the number of WAL files Simon Riggs <simon@2ndQuadrant.com>
@ 2010-10-28 01:30 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Robert Haas @ 2010-10-28 01:30 UTC (permalink / raw)
To: Simon Riggs <simon@2ndquadrant.com>; +Cc: Fujii Masao <masao.fujii@gmail.com>; pgsql-docs
On Fri, Oct 15, 2010 at 9:48 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
> On Fri, 2010-10-15 at 19:44 +0900, Fujii Masao wrote:
>
>> As the result of the above, the maximum number of WAL files in
>> pg_xlog is
>>
>> (2 + checkpoint_completion_target) * checkpoint_segments + 1
>>
>> or
>>
>> wal_keep_segments + checkpoint_segments + 1
>>
>> So the original description seems to be correct. Am I missing
>> something?
>
> After some thought, I agree.
I've put back the original formula.
>> # My previous proposal is definitely wrong. Sorry for noise.
>
> I'd say the definition of wal_keep_segments isn't good, which is what is
> causing the problem here. Will sort that out another day.
>
> I will update the current code with some comments, so this doesn't
> happen again. Will do this on Tuesday now, no time left here.
This is probably a good idea, because the comments in xlog.c are a bit
sparse on this point, but I haven't tried to do it.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2010-10-28 01:30 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-10-14 03:00 formula about the number of WAL files Fujii Masao <masao.fujii@gmail.com>
2010-10-15 09:19 ` Simon Riggs <simon@2ndQuadrant.com>
2010-10-15 10:44 ` Fujii Masao <masao.fujii@gmail.com>
2010-10-15 13:48 ` Simon Riggs <simon@2ndQuadrant.com>
2010-10-28 01:30 ` Robert Haas <robertmhaas@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox