pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedprobably cause (and fix) for floating-point assist faults on itanium
5+ messages / 3 participants
[nested] [flat]
* probably cause (and fix) for floating-point assist faults on itanium
@ 2011-11-18 01:07 Greg Matthews <gregory.a.matthews@nasa.gov>
0 siblings, 1 reply; 5+ messages in thread
From: Greg Matthews @ 2011-11-18 01:07 UTC (permalink / raw)
To: pgsql-performance
Hi folks,
I'm running PG 8.3.15 on an itanium box and was seeing lots of
floating-point assist faults by the kernel. Searched around, found a
couple references/discussions here and there:
http://archives.postgresql.org/pgsql-general/2008-08/msg00244.php
http://archives.postgresql.org/pgsql-performance/2011-06/msg00093.php
http://archives.postgresql.org/pgsql-performance/2011-06/msg00102.php
I took up Tom's challenge and found that the buffer allocation prediction
code in BgBufferSync() is the likely culprit:
if (smoothed_alloc <= (float) recent_alloc)
smoothed_alloc = recent_alloc;
else
smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
smoothing_samples;
smoothed_alloc (float) is moving towards 0 during any extended period of
time when recent_alloc (uint32) remains 0. In my case it takes just a
minute or two before it becomes small enough to start triggering the
fault.
Given how smoothed_alloc is used just after this place in the code it
seems overkill to allow it to continue to shrink so small, so I made a
little mod:
if (smoothed_alloc <= (float) recent_alloc)
smoothed_alloc = recent_alloc;
else if (smoothed_alloc >= 0.00001)
smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
smoothing_samples;
This seems to have done the trick. From what I can tell this section of
code is unchanged in 9.1.1 - perhaps in a future version a similar mod
could be made?
FWIW, I don't think it's really much of a performance impact for the
database, because if recent_alloc remains 0 for a long while it probably
means the DB isn't doing much anyway. However it is annoying when system
logs fill up, and the extra floating point handling may affect some other
process(es).
-Greg
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: probably cause (and fix) for floating-point assist faults on itanium
@ 2011-11-18 14:21 Claudio Freire <klaussfreire@gmail.com>
parent: Greg Matthews <gregory.a.matthews@nasa.gov>
0 siblings, 1 reply; 5+ messages in thread
From: Claudio Freire @ 2011-11-18 14:21 UTC (permalink / raw)
To: Greg Matthews <gregory.a.matthews@nasa.gov>; +Cc: pgsql-performance
On Thu, Nov 17, 2011 at 10:07 PM, Greg Matthews
<gregory.a.matthews@nasa.gov> wrote:
> if (smoothed_alloc <= (float) recent_alloc)
> smoothed_alloc = recent_alloc;
> else if (smoothed_alloc >= 0.00001)
> smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
> smoothing_samples;
>
I don't think that logic is sound.
Rather,
if (smoothed_alloc <= (float) recent_alloc) {
smoothed_alloc = recent_alloc;
} else {
if (smoothed_alloc < 0.000001)
smoothed_alloc = 0;
smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
smoothing_samples;
}
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: probably cause (and fix) for floating-point assist faults on itanium
@ 2011-11-18 16:11 Tom Lane <tgl@sss.pgh.pa.us>
parent: Claudio Freire <klaussfreire@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2011-11-18 16:11 UTC (permalink / raw)
To: Claudio Freire <klaussfreire@gmail.com>; +Cc: Greg Matthews <gregory.a.matthews@nasa.gov>; pgsql-performance
Claudio Freire <klaussfreire@gmail.com> writes:
> On Thu, Nov 17, 2011 at 10:07 PM, Greg Matthews
> <gregory.a.matthews@nasa.gov> wrote:
>> if (smoothed_alloc <= (float) recent_alloc)
>> smoothed_alloc = recent_alloc;
>> else if (smoothed_alloc >= 0.00001)
>> smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
>> smoothing_samples;
>>
> I don't think that logic is sound.
> Rather,
> if (smoothed_alloc <= (float) recent_alloc) {
> smoothed_alloc = recent_alloc;
> } else {
> if (smoothed_alloc < 0.000001)
> smoothed_alloc = 0;
> smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
> smoothing_samples;
> }
The real problem with either of these is the cutoff number is totally
arbitrary. I'm thinking of something like this:
/*
* Track a moving average of recent buffer allocations. Here, rather than
* a true average we want a fast-attack, slow-decline behavior: we
* immediately follow any increase.
*/
if (smoothed_alloc <= (float) recent_alloc)
smoothed_alloc = recent_alloc;
else
smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
smoothing_samples;
/* Scale the estimate by a GUC to allow more aggressive tuning. */
upcoming_alloc_est = smoothed_alloc * bgwriter_lru_multiplier;
+ /*
+ * If recent_alloc remains at zero for many cycles,
+ * smoothed_alloc will eventually underflow to zero, and the
+ * underflows produce annoying kernel warnings on some platforms.
+ * Once upcoming_alloc_est has gone to zero, there's no point in
+ * tracking smaller and smaller values of smoothed_alloc, so just
+ * reset it to exactly zero to avoid this syndrome.
+ */
+ if (upcoming_alloc_est == 0)
+ smoothed_alloc = 0;
/*
* Even in cases where there's been little or no buffer allocation
* activity, we want to make a small amount of progress through the buffer
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: probably cause (and fix) for floating-point assist faults on itanium
@ 2011-11-18 16:37 Greg Matthews <gregory.a.matthews@nasa.gov>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 5+ messages in thread
From: Greg Matthews @ 2011-11-18 16:37 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Claudio Freire <klaussfreire@gmail.com>; pgsql-performance
Looks good to me. I built PG with this change, no kernel warnings after
~10 minutes of running. I'll continue to monitor but I think this fixes
the syndrome. Thanks Tom.
-Greg
On Fri, 18 Nov 2011, Tom Lane wrote:
> Claudio Freire <klaussfreire@gmail.com> writes:
>> On Thu, Nov 17, 2011 at 10:07 PM, Greg Matthews
>> <gregory.a.matthews@nasa.gov> wrote:
>>> if (smoothed_alloc <= (float) recent_alloc)
>>> smoothed_alloc = recent_alloc;
>>> else if (smoothed_alloc >= 0.00001)
>>> smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
>>> smoothing_samples;
>>>
>
>> I don't think that logic is sound.
>
>> Rather,
>
>> if (smoothed_alloc <= (float) recent_alloc) {
>> smoothed_alloc = recent_alloc;
>> } else {
>> if (smoothed_alloc < 0.000001)
>> smoothed_alloc = 0;
>> smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
>> smoothing_samples;
>> }
>
> The real problem with either of these is the cutoff number is totally
> arbitrary. I'm thinking of something like this:
>
> /*
> * Track a moving average of recent buffer allocations. Here, rather than
> * a true average we want a fast-attack, slow-decline behavior: we
> * immediately follow any increase.
> */
> if (smoothed_alloc <= (float) recent_alloc)
> smoothed_alloc = recent_alloc;
> else
> smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
> smoothing_samples;
>
> /* Scale the estimate by a GUC to allow more aggressive tuning. */
> upcoming_alloc_est = smoothed_alloc * bgwriter_lru_multiplier;
>
> + /*
> + * If recent_alloc remains at zero for many cycles,
> + * smoothed_alloc will eventually underflow to zero, and the
> + * underflows produce annoying kernel warnings on some platforms.
> + * Once upcoming_alloc_est has gone to zero, there's no point in
> + * tracking smaller and smaller values of smoothed_alloc, so just
> + * reset it to exactly zero to avoid this syndrome.
> + */
> + if (upcoming_alloc_est == 0)
> + smoothed_alloc = 0;
>
> /*
> * Even in cases where there's been little or no buffer allocation
> * activity, we want to make a small amount of progress through the buffer
>
>
> regards, tom lane
>
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: probably cause (and fix) for floating-point assist faults on itanium
@ 2011-11-19 05:39 Tom Lane <tgl@sss.pgh.pa.us>
parent: Greg Matthews <gregory.a.matthews@nasa.gov>
0 siblings, 0 replies; 5+ messages in thread
From: Tom Lane @ 2011-11-19 05:39 UTC (permalink / raw)
To: Greg Matthews <gregory.a.matthews@nasa.gov>; +Cc: Claudio Freire <klaussfreire@gmail.com>; pgsql-performance
Greg Matthews <gregory.a.matthews@nasa.gov> writes:
> Looks good to me. I built PG with this change, no kernel warnings after
> ~10 minutes of running. I'll continue to monitor but I think this fixes
> the syndrome. Thanks Tom.
Patch committed -- thanks for checking it.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2011-11-19 05:39 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2011-11-18 01:07 probably cause (and fix) for floating-point assist faults on itanium Greg Matthews <gregory.a.matthews@nasa.gov>
2011-11-18 14:21 ` Claudio Freire <klaussfreire@gmail.com>
2011-11-18 16:11 ` Tom Lane <tgl@sss.pgh.pa.us>
2011-11-18 16:37 ` Greg Matthews <gregory.a.matthews@nasa.gov>
2011-11-19 05:39 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox