public inbox for [email protected]
help / color / mirror / Atom feedEmail search failure
9+ messages / 6 participants
[nested] [flat]
* Email search failure
@ 2008-07-31 20:19 Bruce Momjian <[email protected]>
2008-07-31 20:44 ` Re: Email search failure Alvaro Herrera <[email protected]>
2008-07-31 20:46 ` Re: Email search failure Alvaro Herrera <[email protected]>
2008-07-31 21:26 ` Re: Email search failure Tom Lane <[email protected]>
0 siblings, 3 replies; 9+ messages in thread
From: Bruce Momjian @ 2008-07-31 20:19 UTC (permalink / raw)
To: pgsql-www
Why is the email below now appearing in a search?
http://search.postgresql.org/search?q=As+you+yourself+said%2C+this+patch+mostly+gets+the+machinery+t...
---------------------------------------------------------------------------
----- Forwarded message from Pavan Deolasee -----
On Tue, Jul 1, 2008 at 4:13 AM, Simon Riggs <[email protected]> wrote:
>
>
> The first "half" is actually quite large, but that makes it even more
> sensible to commit this part now.
>
> The enclosed patch introduces the machinery by which we might later
> optimise hint bit setting. It differentiates between hint bit setting
> and block dirtying, when the distinction can safely be made. It acts
> safely during VACUUM and correctly during checkpoint. In all other
> respects it emulates current behaviour.
>
As you yourself said, this patch mostly gets the machinery to count
hint bits in place and leaves the actual optimization for future. But
I think we should try at least one or two possible optimizations and
get some numbers before we jump and make substantial changes to the
code. Also that would help us in testing the patch for correctness and
performance.
For example, the following hunk seems buggy to me:
Index: src/backend/storage/buffer/bufmgr.c
===================================================================
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/storage/buffer/bufmgr.c,v
retrieving revision 1.232
diff -c -r1.232 bufmgr.c
*** src/backend/storage/buffer/bufmgr.c 12 Jun 2008 09:12:31 -0000 1.232
--- src/backend/storage/buffer/bufmgr.c 30 Jun 2008 22:17:20 -0000
***************
*** 1460,1473 ****
if (bufHdr->refcount == 0 && bufHdr->usage_count == 0)
result |= BUF_REUSABLE;
! else if (skip_recently_used)
{
/* Caller told us not to write recently-used buffers */
UnlockBufHdr(bufHdr);
return result;
}
! if (!(bufHdr->flags & BM_VALID) || !(bufHdr->flags & BM_DIRTY))
{
/* It's clean, so nothing to do */
UnlockBufHdr(bufHdr);
--- 1462,1477 ----
if (bufHdr->refcount == 0 && bufHdr->usage_count == 0)
result |= BUF_REUSABLE;
! else if (LRU_scan)
{
/* Caller told us not to write recently-used buffers */
UnlockBufHdr(bufHdr);
return result;
}
! if (!(bufHdr->flags & BM_VALID) ||
! !(bufHdr->flags & BM_DIRTY ||
! (LRU_scan && bufHdr->hint_count > 0)))
{
/* It's clean, so nothing to do */
UnlockBufHdr(bufHdr);
In the "if" condition above, we would throw away a buffer if the
hint_count is greater than zero, even if the buffer is dirty. This
doesn't seem correct to me, unless I am missing something obvious.
> The actual tuning patch can be discussed later, probably at length.
> Later patches will be fairly small in comparison and so various people
> can fairly easily come up with their own favoured modifications for
> testing.
>
>
I would suggest, let's have at least one tuning patch along with some
tests and numbers, before we go ahead and commit anything.
Thanks,
Pavan
--
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-patches mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches
----- End of forwarded message from Pavan Deolasee -----
--
Bruce Momjian <[email protected]> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
@ 2008-07-31 20:44 ` Alvaro Herrera <[email protected]>
2 siblings, 0 replies; 9+ messages in thread
From: Alvaro Herrera @ 2008-07-31 20:44 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: pgsql-www
Bruce Momjian wrote:
>
> Why is the email below now appearing in a search?
Can we please see the email headers? From, To, Date, Message-Id, Cc.
--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
@ 2008-07-31 20:46 ` Alvaro Herrera <[email protected]>
2008-07-31 20:52 ` Re: Email search failure Bruce Momjian <[email protected]>
2 siblings, 1 reply; 9+ messages in thread
From: Alvaro Herrera @ 2008-07-31 20:46 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: pgsql-www
Bruce Momjian wrote:
>
> Why is the email below now appearing in a search?
>
> http://search.postgresql.org/search?q=As+you+yourself+said%2C+this+patch+mostly+gets+the+machinery+t...
FWIW, this patch is in the archives:
http://archives.postgresql.org/message-id/[email protected]...
and it's referenced on the commitfest page.
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
2008-07-31 20:46 ` Re: Email search failure Alvaro Herrera <[email protected]>
@ 2008-07-31 20:52 ` Bruce Momjian <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Bruce Momjian @ 2008-07-31 20:52 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: pgsql-www
Alvaro Herrera wrote:
> Bruce Momjian wrote:
> >
> > Why is the email below now appearing in a search?
> >
> > http://search.postgresql.org/search?q=As+you+yourself+said%2C+this+patch+mostly+gets+the+machinery+t...
>
> FWIW, this patch is in the archives:
>
> http://archives.postgresql.org/message-id/[email protected]...
>
> and it's referenced on the commitfest page.
Yep, here it is in the archives:
http://archives.postgresql.org/pgsql-patches/2008-07/msg00199.php
but it does not show up in a search:
http://search.postgresql.org/search?q=Hint+Bits+and+Write+&m=1&l=&d=365&s=d
--
Bruce Momjian <[email protected]> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
@ 2008-07-31 21:26 ` Tom Lane <[email protected]>
2008-08-01 03:15 ` Re: Email search failure Joshua D. Drake <[email protected]>
2 siblings, 1 reply; 9+ messages in thread
From: Tom Lane @ 2008-07-31 21:26 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: pgsql-www
Bruce Momjian <[email protected]> writes:
> Why is the email below now appearing in a search?
Probably because nothing has gotten indexed for a month or more.
Whoever is supposed to maintain the archive indexer has been
on vacation since it broke ...
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
2008-07-31 21:26 ` Re: Email search failure Tom Lane <[email protected]>
@ 2008-08-01 03:15 ` Joshua D. Drake <[email protected]>
2008-08-01 12:39 ` Re: Email search failure Magnus Hagander <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Joshua D. Drake @ 2008-08-01 03:15 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Bruce Momjian <[email protected]>; pgsql-www
Tom Lane wrote:
> Bruce Momjian <[email protected]> writes:
>> Why is the email below now appearing in a search?
>
> Probably because nothing has gotten indexed for a month or more.
> Whoever is supposed to maintain the archive indexer has been
> on vacation since it broke ...
That would be Magnus and you are correct. He just got back. The problem
(last I checked) is an issue with Russian emails.
Sincerely,
Joshua D. Drake
>
> regards, tom lane
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
2008-07-31 21:26 ` Re: Email search failure Tom Lane <[email protected]>
2008-08-01 03:15 ` Re: Email search failure Joshua D. Drake <[email protected]>
@ 2008-08-01 12:39 ` Magnus Hagander <[email protected]>
2008-08-01 13:23 ` Re: Email search failure [email protected]
0 siblings, 1 reply; 9+ messages in thread
From: Magnus Hagander @ 2008-08-01 12:39 UTC (permalink / raw)
To: Joshua D. Drake <[email protected]>; +Cc: Tom Lane <[email protected]>; Bruce Momjian <[email protected]>; pgsql-www
Joshua D. Drake wrote:
> Tom Lane wrote:
>> Bruce Momjian <[email protected]> writes:
>>> Why is the email below now appearing in a search?
>>
>> Probably because nothing has gotten indexed for a month or more.
>> Whoever is supposed to maintain the archive indexer has been
>> on vacation since it broke ...
>
> That would be Magnus and you are correct. He just got back. The problem
> (last I checked) is an issue with Russian emails.
Looking at it now. That clearly wasn't the only problem, because there
was a "sleep 1800" process that had been running since July 3. Logfiles
weren't touched etc. Just restarting it fixed that part, which clearly
somebody else could've done as well ;)
I found the bug with the Russian emails, btw. It seems mhonarc encoded
the invalid UTF8 sequences inside valid HTML escape entities And the
code applied the "fix broken UTF8" logic *before* it unescaped the HTML
entities. Now it does it both before and after..
Oh, and this should never have affected messages on -hackers for
example, because it was always processed before ru-general. It would hit
the PUG lists, -www, -patches and a few others.
//Magnus
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
2008-07-31 21:26 ` Re: Email search failure Tom Lane <[email protected]>
2008-08-01 03:15 ` Re: Email search failure Joshua D. Drake <[email protected]>
2008-08-01 12:39 ` Re: Email search failure Magnus Hagander <[email protected]>
@ 2008-08-01 13:23 ` [email protected]
2008-08-01 13:36 ` Re: Email search failure Magnus Hagander <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: [email protected] @ 2008-08-01 13:23 UTC (permalink / raw)
To: Magnus Hagander <[email protected]>; +Cc: Joshua D. Drake <[email protected]>; Tom Lane <[email protected]>; Bruce Momjian <[email protected]>; pgsql-www
Any idea why there were no alerts? Ae we monitoring pgsql-aardvarks
instead of pgsql-zebras?
On 8/1/08, Magnus Hagander <[email protected]> wrote:
> Joshua D. Drake wrote:
>> Tom Lane wrote:
>>> Bruce Momjian <[email protected]> writes:
>>>> Why is the email below now appearing in a search?
>>>
>>> Probably because nothing has gotten indexed for a month or more.
>>> Whoever is supposed to maintain the archive indexer has been
>>> on vacation since it broke ...
>>
>> That would be Magnus and you are correct. He just got back. The problem
>> (last I checked) is an issue with Russian emails.
>
> Looking at it now. That clearly wasn't the only problem, because there
> was a "sleep 1800" process that had been running since July 3. Logfiles
> weren't touched etc. Just restarting it fixed that part, which clearly
> somebody else could've done as well ;)
>
> I found the bug with the Russian emails, btw. It seems mhonarc encoded
> the invalid UTF8 sequences inside valid HTML escape entities And the
> code applied the "fix broken UTF8" logic *before* it unescaped the HTML
> entities. Now it does it both before and after..
>
> Oh, and this should never have affected messages on -hackers for
> example, because it was always processed before ru-general. It would hit
> the PUG lists, -www, -patches and a few others.
>
> //Magnus
>
> --
> Sent via pgsql-www mailing list ([email protected])
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-www
>
--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Email search failure
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
2008-07-31 21:26 ` Re: Email search failure Tom Lane <[email protected]>
2008-08-01 03:15 ` Re: Email search failure Joshua D. Drake <[email protected]>
2008-08-01 12:39 ` Re: Email search failure Magnus Hagander <[email protected]>
2008-08-01 13:23 ` Re: Email search failure [email protected]
@ 2008-08-01 13:36 ` Magnus Hagander <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Magnus Hagander @ 2008-08-01 13:36 UTC (permalink / raw)
To: [email protected]; +Cc: Joshua D. Drake <[email protected]>; Tom Lane <[email protected]>; Bruce Momjian <[email protected]>; pgsql-www
Stefan acknowledged the alert on 2008-07-04 07:36:15 (per nagios
history), which makes it stop complaining (with the comment that I
should be looking at it - doesn't have any effect if I don't actually
have time to do that before I go on vacation). It still showed up under
services that were down if you went to the nagios website.
//Magnus
[email protected] wrote:
> Any idea why there were no alerts? Ae we monitoring pgsql-aardvarks
> instead of pgsql-zebras?
>
> On 8/1/08, Magnus Hagander <[email protected]> wrote:
>> Joshua D. Drake wrote:
>>> Tom Lane wrote:
>>>> Bruce Momjian <[email protected]> writes:
>>>>> Why is the email below now appearing in a search?
>>>> Probably because nothing has gotten indexed for a month or more.
>>>> Whoever is supposed to maintain the archive indexer has been
>>>> on vacation since it broke ...
>>> That would be Magnus and you are correct. He just got back. The problem
>>> (last I checked) is an issue with Russian emails.
>> Looking at it now. That clearly wasn't the only problem, because there
>> was a "sleep 1800" process that had been running since July 3. Logfiles
>> weren't touched etc. Just restarting it fixed that part, which clearly
>> somebody else could've done as well ;)
>>
>> I found the bug with the Russian emails, btw. It seems mhonarc encoded
>> the invalid UTF8 sequences inside valid HTML escape entities And the
>> code applied the "fix broken UTF8" logic *before* it unescaped the HTML
>> entities. Now it does it both before and after..
>>
>> Oh, and this should never have affected messages on -hackers for
>> example, because it was always processed before ru-general. It would hit
>> the PUG lists, -www, -patches and a few others.
>>
>> //Magnus
>>
>> --
>> Sent via pgsql-www mailing list ([email protected])
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-www
>>
>
>
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2008-08-01 13:36 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2008-07-31 20:19 Email search failure Bruce Momjian <[email protected]>
2008-07-31 20:44 ` Alvaro Herrera <[email protected]>
2008-07-31 20:46 ` Alvaro Herrera <[email protected]>
2008-07-31 20:52 ` Bruce Momjian <[email protected]>
2008-07-31 21:26 ` Tom Lane <[email protected]>
2008-08-01 03:15 ` Joshua D. Drake <[email protected]>
2008-08-01 12:39 ` Magnus Hagander <[email protected]>
2008-08-01 13:23 ` [email protected]
2008-08-01 13:36 ` Magnus Hagander <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox