Subject: Re: Soooooo far ahead of scheduale ...

From: "John Hansen" <[email-masked]>
Date: Tue, 10 Aug 2004 07:08:17 +1000

> 
> I can put them there, if someone can point to me either a 
> format for, or a sample file of, the robots.txt file?
> 

http://www.robotstxt.org/wc/norobots.html

Regards,

John

---

From: "Marc G. Fournier" <[email-masked]>
Date: Tue, 10 Aug 2004 00:58:24 -0300 (ADT)

k, created ... have disallow set for cvsweb.cgi, regress and docs ...

> http://www.robotstxt.org/wc/norobots.html

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: [email-masked]           Yahoo!: yscrappy              ICQ: 7615664

---

From: "Marc G. Fournier" <[email-masked]>
Date: Tue, 17 Aug 2004 18:29:14 -0300 (ADT)

just changed the cron job to be every 6 hours instead of 24 ...

----