Subject: Mail archive indexes are broken, URLs too

From: Tom Lane <[email-masked]>
Date: Sun, 16 Jul 2006 14:43:09 -0400

When Marc fixed the message-boundary pattern and regenerated the
archives, many of the existing messages changed URLs because they
got assigned slightly different numbers.  I notice that the archive
search engine hasn't yet tracked this change --- if you do a search
and click on a link to a message, you'll arrive at a message close
to the one you want but probably not quite it.

Regenerating the archive indexes is presumably not hard, but there's
a bigger problem: for awhile now many of us have been in the habit
of citing old discussions by archive URLs.  All those links are now
broken too, and I can't think of any easy way to fix them.  And then
there's Google etc.

I wonder if it'd be better to revert the regeneration of the archives,
and only apply the new message-boundary pattern to future messages.

			regards, tom lane

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sun, 16 Jul 2006 16:43:43 -0300 (ADT)

> and only apply the new message-boundary pattern to future messages.

Nope, for one simple reason ... if, for some reason, at some point in the 
future, we have to regenerate everything anyway (ie. the last time we did 
a major template change for the archives), all the #'ng is going to end up 
reverting back to what it is now ... so we'd only be 'delaying the 
inevitable' ...

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . [email-masked]                              MSN . [email-masked]
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664

---

From: Jim Nasby <[email-masked]>
Date: Wed, 19 Jul 2006 13:12:06 -0500

On Jul 16, 2006, at 2:43 PM, Marc G. Fournier wrote:
> Nope, for one simple reason ... if, for some reason, at some point  
> in the future, we have to regenerate everything anyway (ie. the  
> last time we did a major template change for the archives), all the  
> #'ng is going to end up reverting back to what it is now ... so  
> we'd only be 'delaying the inevitable' ...

This is a problem for most mailing lists, but I think it's a critical  
one for us since we depend very, very heavily on the archives.

Can we change the lists so that they will generate a UUID and add it  
to message headers, and then allow the archive software to key off of  
that?
--
Jim C. Nasby, Sr. Engineering Consultant      [email-masked]
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 29 Jul 2006 11:39:00 -0400 (EDT)

> and only apply the new message-boundary pattern to future messages.

Agreed.  There have been no changes since we discussed this.

The best proposal was to renumber the newly-found items to the end of
the numeric range for the pre-July 2006 archives, and to properly number
July 2006 and later archives.  And this date range has to be enbedded in
the archive script so if it is ever run again, this behavior continues
to happen.

The longer we take to fix this, the more likely that people are creating
URL's that refer to the existing pre-July 2006 numbering which should
change.  It needs to be fixed quickly.

And we can't just leave it alone because old archive emails have URLs
that point to now-incorrect numbers, and there is no good way to fix
that everywhere are emails are archived.

-- 
  Bruce Momjian   [email-masked]
  EnterpriseDB    http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 1 Aug 2006 15:11:26 -0400 (EDT)

Is anyone working on this?  Marc?  If not, who can make these
modifications to the archive numbering?

---------------------------------------------------------------------------

Bruce Momjian wrote:
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
> 
>                http://archives.postgresql.org

---

From: "Joshua D. Drake" <[email-masked]>
Date: Tue, 01 Aug 2006 13:06:02 -0700

> modifications to the archive numbering?

I believe Marc is the only one that can at last I heard on this, he 
disagreed with rolling back the change.

Joshua D. Drake

-- 

    === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
    Providing the most comprehensive  PostgreSQL solutions since 1997
              http://www.commandprompt.com/

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 1 Aug 2006 17:36:29 -0400 (EDT)

> disagreed with rolling back the change.

I have heard no reason he doesn't like the change, and unless he can
convince most of us, it is time to make the change.

---

From: "Joshua D. Drake" <[email-masked]>
Date: Tue, 01 Aug 2006 14:39:26 -0700

> convince most of us, it is time to make the change.

Marc wrote:

Nope, for one simple reason ... if, for some reason, at some point in 
the future, we have to regenerate everything anyway (ie. the last time 
we did a major template change for the archives), all the #'ng is going 
to end up reverting back to what it is now ... so we'd only be 'delaying 
the inevitable' ...

On July 17th.

Joshua D. Drake

--

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 1 Aug 2006 21:44:26 -0400 (EDT)

> to end up reverting back to what it is now ... so we'd only be 'delaying

If you look below you will see my idea was to hack the script to always
use the method of putting newly found items numerically at the end for
pre-July 2006 dumps.  That addresses Marc's concern.

Marc hasn't responded so I assume he is busy and will hack on this when
he gets back.

---

From: "Marc G. Fournier" <[email-masked]>
Date: Tue, 1 Aug 2006 23:09:11 -0300 (ADT)

> Marc hasn't responded so I assume he is busy and will hack on this when

Yup, been busy dealing with an Adaptec driver issue, will try and get 
something hacked up over the coming weekend, sorry for the delay ...

----

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 1 Aug 2006 23:06:35 -0400 (EDT)

> something hacked up over the coming weekend, sorry for the delay ...

Thanks.  When talking via IM I didn't get the sense whether you agreed
that this was a good idea or not, so I figured I should ask on the lists
so others know it is in process.

---

From: "Marc G. Fournier" <[email-masked]>
Date: Wed, 2 Aug 2006 00:25:24 -0300 (ADT)

> so others know it is in process.

Oh, I still don't think its a good idea, but understand why ...

---

From: "Marc G. Fournier" <[email-masked]>
Date: Wed, 9 Aug 2006 14:18:14 -0300 (ADT)

Just shutdown rsync while I rebuild the archives for the 'old/new' scheme, 
where old is pre-July 2006 ...

will post once its been all rebuilt ...

----

---

From: "Marc G. Fournier" <[email-masked]>
Date: Wed, 9 Aug 2006 16:27:38 -0300 (ADT)

'k, rsync is back up ... for a short period, part of the archives will 
disappear, but a large portion of it is re-generated, and figured may as 
well let the 'feed server' start downloading now :)

----

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 9 Aug 2006 15:45:13 -0400 (EDT)

Nice, thanks.