Subject: mailing list offer

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 18 Oct 2004 18:28:39 -0700

Hello,

Now that CMD is hosting the archives I thought that I would offer some
more resources. I would be willing to purchase a dedicated machine to 
host the mailing lists at our location. We would use this machine for 
the archives and lists both.

As we already relay a great deal of the archive traffic through two of
our machines I do not see our bandwidth (3 DS3s) as being an issue. In 
terms of reliability the machine would also be placed on a natural gas 
fired generator and of course the network is switched and firewalled.

I would also be willing to dedicate some resources to some other
projects that people have brought up, including direct integration of 
the mail archives and lists into PostgreSQL.

What are the thoughts on this?

Sincerely,

Joshua D. Drake

-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - [email-masked] - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL

---

From: "Marc G. Fournier" <[email-masked]>
Date: Mon, 18 Oct 2004 23:07:24 -0300 (ADT)

Thanks, but the mailing lists are fine where they currently are ... there 
are no current plans to move them ...

On Mon, 18 Oct 2004, Joshua D. Drake wrote:

> more resources. I would be willing to purchase a dedicated machine to host 
> the mailing lists at our location. We would use this machine for the archives 
> and lists both.
>
> our machines I do not see our bandwidth (3 DS3s) as being an issue. In terms 
> of reliability the machine would also be placed on a natural gas fired 
> generator and of course the network is switched and firewalled.
>
> projects that people have brought up, including direct integration of the 
> mail archives and lists into PostgreSQL.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>   (send "unregister YourEmailAddressHere" to [email-masked])
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: [email-masked]           Yahoo!: yscrappy              ICQ: 7615664

---

From: Justin Clift <[email-masked]>
Date: Tue, 19 Oct 2004 12:22:53 +1000

Marc G. Fournier wrote:
> 
> Thanks, but the mailing lists are fine where they currently are ... 
> there are no current plans to move them ...

Actually, it's an interesting thought and once in place it would free up 
people's time for other stuff.

Joshua, what hardware would you be looking at getting for it, and what 
kind of software do you guys generally use for mailing lists?

The other initial main thought is... what kind of hours of support can 
we expect?  Like, if something fails at 1am, can we ring someone who'll 
be able to look into it in a decent timeframe and have the ability to 
get it resolved (within reason of course).

:)

Regards and best wishes,

Justin Clift

---

From: John Hansen <[email-masked]>
Date: Tue, 19 Oct 2004 13:11:35 +1000

> the mail archives and lists into PostgreSQL.

I can't see anything that would stop you from doing this now...
Marc should be able to provide you with the old mbox archives, and
following them, simply subscribe to all of the lists, and you have
yourself an up to date archive.

I had the forums going, and have been playing with getting the archives
integrated, but the importer is way too buggy, and needs a lot of work
before they can be brought to life.

A fully integrated solution would be sweet. One that combines the
features of the archives, the search engine, and the lists themselves.

You should be aware tho, that iirc the current tally is closing in on
300.000 emails. Does anyone have any installations with that much data
indexed by tsearch2, who would be able to give us some indicative
reports on query time?

Mnogosearch was scrapped due to it's speed (or lack thereof), and iirc,
tsearch2 closely resembles the same type of mechanism for full text
searches. Just something to think about.

Btw, With regards to the forums, I imagine they'll come back over the
weekend...

> What are the thoughts on this?

... John

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 18 Oct 2004 20:12:50 -0700

Justin Clift wrote:

> Actually, it's an interesting thought and once in place it would free 
> up people's time for other stuff.
>

I figured I would put together a small dual opteron with an SATA array...

We are fond of mailman but will use anything the project desires.

> we expect?  Like, if something fails at 1am, can we ring someone 
> who'll be able to look into it in a decent timeframe and have the 
> ability to get it resolved (within reason of course).

I would provide my cell phone to appropriate responsible parties and 
other on-call numbers when needed.

Sincerely,

Joshua D. Drake

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 18 Oct 2004 20:24:43 -0700

>indexed by tsearch2, who would be able to give us some indicative
I have a database now with 1.77 million entries using Tsearch. The 
entries are book titles. It runs
very well. There are some tricks to it though. For example what we had 
to was have a single
table that has nothing but a product_id, title, title_vectors.

Then on a search, we return the product_ids and grab the titles from the 
detail table via the product_ids
that were pulled from the search.

Sincerely,

Joshua D. Drake

---

From: "Marc G. Fournier" <[email-masked]>
Date: Tue, 19 Oct 2004 00:33:32 -0300 (ADT)

On Tue, 19 Oct 2004, John Hansen wrote:

> I can't see anything that would stop you from doing this now... Marc 
> should be able to provide you with the old mbox archives, and following 
> them, simply subscribe to all of the lists, and you have yourself an up 
> to date archive.

I've also just added in a 'mbox' link to the archives ... so you can 
download the raw files from the web site now too ...

----

---

From: Oleg Bartunov <[email-masked]>
Date: Tue, 19 Oct 2004 07:35:44 +0400 (MSD)

> searches. Just something to think about.

in principle, www.pgsql.ru/db/mw (fts.postgresql.org in past) was designed
to be an integrated solution, but time past and we have no spare time
to continue developing (we only reinstalled fts.postgresql.org).
As for tsearch2 performance, then it's slow on full archive, but reasonably
fast searching last month (default) or last year archive. 
We have tested our idea using read only archive and online news archive,
and it worked very-very fast. Main idea is to have common infrastructure
(parsers, dictionaries, ranking) for inverted index (read only part) and
tsearch2 index for changing part. Running daemon (tsearchd) accepts
text for indexing, all metadata are stored in posgresql, so searching
could be constrained using metadata. Tsearchd is just a working prototype
and much work is needed.

> TIP 6: Have you searched our list archives?
>
>               http://archives.postgresql.org
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: [email-masked], http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

---

From: John Hansen <[email-masked]>
Date: Tue, 19 Oct 2004 13:41:30 +1000

> table that has nothing but a product_id, title, title_vectors.

Ok, i'm guessing none of the titles are as long as an email... what's
the speed like?

... John

---

From: "Joshua D. Drake" <[email-masked]>
Date: Mon, 18 Oct 2004 21:09:20 -0700

>Ok, i'm guessing none of the titles are as long as an email... what's
1.4 seconds for the search, 4.5ms for the pull of the full details..

So no -- that may not scale that well. We would have to test it out.

Sincerely,

Joshua D. Drake

---

From: "Joshua D. Drake" <[email-masked]>
Date: Tue, 19 Oct 2004 10:50:13 -0700

> So no -- that may not scale that well. We would have to test it out.

There are also other items that can be done, like keeping the tables
per month. And allow searching based on the month. If they want to
search the entire archives I think it is pretty much a safe bet that
it will take time.

Of course, we can always use the Google API as well.

Sincerely,

Joshua D. Drake

-- 
Command Prompt, Inc., home of PostgreSQL Replication, and plPHP.
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - [email-masked] - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL

---

From: "Joshua D. Drake" <[email-masked]>
Date: Tue, 19 Oct 2004 10:52:56 -0700

> We are fond of mailman but will use anything the project desires.

Would mailman be sufficient? It seems to offer the most versatility. 
Besides it uses our preferred language (python) :)

> other on-call numbers when needed.

To be more clear on this. If there was a problem during business hours
you can always call the office. We are PST and a tech is always 
answering the phone (we don't have an answering service or anything).

For after hours stuff, I would provide an after hours call. Basically I 
would treat this just like I treat our managed services, which means you
get 24x7 support.

Sincerely,

Joshua D. Drake

---

From: Justin Clift <[email-masked]>
Date: Wed, 20 Oct 2004 08:30:49 +1000

> people's time for other stuff.

I'm interested in what other people's opinion on this is.

Personally, I feel this is worth considering because it gives us more 
practice at getting existing PG companies further involved, and Command 
Prompt is a relatively safe bet here.

That's a good thing I feel, because we'll likely get further offers as 
PG gains wider usage.

?

Regards and best wishes,

Justin Clift

---

From: Josh Berkus <[email-masked]>
Date: Wed, 20 Oct 2004 11:53:55 -0700

Guys,

Hmmm ... from my perspective, what we need is not so much to move the mailing 
lists, is to have a fallback to another mailing list server in the event that 
Hub.org becomes unavailable for some reason, such as the trunk line in Costa 
Rica going down (as happened last fall).   The mailing lists are the backbone 
of our community; we really can't afford for them to go offline for more than 
a few hours.

-- 
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco

---

From: "Joshua D. Drake" <[email-masked]>
Date: Wed, 20 Oct 2004 12:08:39 -0700

> of our community; we really can't afford for them to go offline for more than 

Well although that is obviously a possibility (Backhoe comes to mind) 
our connectivity comes from three different providers:

1. ELI
2. Sprint
3. TimeWarner

Also our connectivity is physically accessible my half a dozen people 
within 30 minutes, even at 3:00 AM. If something goes down, the whole
freaking world is notified to make sure it is getting worked on. Did I 
mention the natural gas fired generator? ;)

I just want to help make the parts of the project that we can, as 
reliable as possible.

Sincerely,

Joshua D. Drake

---

From: "Joshua D. Drake" <[email-masked]>
Date: Wed, 08 Dec 2004 10:20:38 -0800

Hello,

I am sure that everyone is doing their best to get the mailing lists 
back up. However, I would like to re-extend my offer to host the mailing 
lists.
In the time that the lists have been down the last two days I could have
built a brand new machine, restored from backup, had the lists back up
and running with 16 hours to spare.

I am not trying to step on any toes here but the main source of community
communication is down right now and has been for 24 hours. This is really
not good.

Sincerely,

Joshua D. Drake

Joshua D. Drake wrote:

> Josh Berkus wrote:
>
>> Guys,
>>
>> Hmmm ... from my perspective, what we need is not so much to move the 
>> mailing lists, is to have a fallback to another mailing list server 
>> in the event that Hub.org becomes unavailable for some reason, such 
>> as the trunk line in Costa Rica going down (as happened last fall).   
>> The mailing lists are the backbone of our community; we really can't 
>> afford for them to go offline for more than a few hours.
>
>
>------------------------------------------------------------------------
>
>

---

From: Josh Berkus <[email-masked]>
Date: Wed, 8 Dec 2004 10:25:47 -0800

Josh,

> back up. However, I would like to re-extend my offer to host the mailing

Apparently the problem is that the network we use to replicate the mailing 
lists failed.   So we can't get at them at the moment, for any purpose: the 
machine is there and running, but inaccessable, and our backup of the lists 
is not reliable.  Marc and I have been chatting about other places to 
host/move stuff.   I'm sure he'll have more news when it warrants.

-- 
Josh Berkus
Aglio Database Solutions
San Francisco

---

From: "Joshua D. Drake" <[email-masked]>
Date: Wed, 08 Dec 2004 10:32:34 -0800

>host/move stuff.   I'm sure he'll have more news when it warrants.
O.k. well we haven't been down for more than 30 minutes (network wise)
in 4 years. Let me know if I can help.

Sincerely,

Joshua D. Drake

---

From: "Marc G. Fournier" <[email-masked]>
Date: Fri, 10 Dec 2004 18:51:04 -0400 (AST)

Thank you for your offer, but we have taken steps towards setting up a hot 
failover mirror of that VM to a server in Austria, which we will be 
working on more next week once the server is ready ...

On Wed, 8 Dec 2004, Joshua D. Drake wrote:

> Hello,
>
> I am sure that everyone is doing their best to get the mailing lists back up. 
> However, I would like to re-extend my offer to host the mailing lists.
>>> mailing lists, is to have a fallback to another mailing list server in the 
>>> event that Hub.org becomes unavailable for some reason, such as the trunk 
>>> line in Costa Rica going down (as happened last fall).   The mailing lists 
>>> are the backbone of our community; we really can't afford for them to go 
>>> offline for more than a few hours.
>> 
>> 
>> Well although that is obviously a possibility (Backhoe comes to mind) our 
>> connectivity comes from three different providers:
>> 
>> 1. ELI
>> 2. Sprint
>> 3. TimeWarner
>> 
>> I just want to help make the parts of the project that we can, as reliable 
>> as possible.
>> 
>> Sincerely,
>> 
>> Joshua D. Drake
>> 
>> 
>>> 
>> 
>> 

----

---

From: Oleg Bartunov <[email-masked]>
Date: Sat, 11 Dec 2004 11:24:39 +0300 (MSK)

Josh,

the problem is not with mirroring of mailing lists, there is google,
www.pgsql.ru, for example. I think, the problem is with reliable
mailing list gateway.

 	Oleg

 	Regards,
 		Oleg

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sat, 11 Dec 2004 09:19:04 -0400 (AST)

> www.pgsql.ru, for example. I think, the problem is with reliable

the problem was actually our provider not paying their provider and 
their provider finally sayin 'enough is enough' ...

----

---

From: Josh Berkus <[email-masked]>
Date: Sat, 11 Dec 2004 09:58:54 -0800

Oleg,

> www.pgsql.ru, for example. I think, the problem is with reliable

When I said "mailing lists" I was referring to majordomo and the subscriber 
lists, not to the archives.

-- 
Josh Berkus
Aglio Database Solutions
San Francisco