Date: Tue, 10 Apr 2007 01:36:02 +0200
To: "Andrew Hammond" <andrew.george.hammond@gmail.com>,
	"CAJ CAJ" <pguser@gmail.com>
Subject: Re: programmatic way to fetch latest release for a given major.minor
	version
From: Listmail <lists@peufeu.com>
Cc: pgsql-general@postgresql.org
Content-Type: text/plain; format=flowed; delsp=yes; charset=utf-8
MIME-Version: 1.0
References: <1176155240.221125.319880@y80g2000hsf.googlegroups.com>
	<467669b30704091526v79ae044boe7217cc0777dce1f@mail.gmail.com>
	<5a0a9d6f0704091534i4e4af90co2dad0e9e0b60efee@mail.gmail.com>
Content-Transfer-Encoding: Quoted-Printable
Message-ID: <op.tqjiicjuzcizji@apollo13>
In-Reply-To: <5a0a9d6f0704091534i4e4af90co2dad0e9e0b60efee@mail.gmail.com>
User-Agent: Opera Mail/9.10 (Linux)

	Here you go.

	Fetches versions and prints most recent minor for each major
	Tests all mirrors for speed and prints out the 4 fastest (takes some ti=
me)
	http://www.crummy.com/software/BeautifulSoup/

	Have a nice day !

#! /bin/env python
# -*- coding: utf-8  -*-

import urllib, BeautifulSoup, re, time, sys


def get_all_versions():
	soup =3D  =

BeautifulSoup.BeautifulSoup( urllib.urlopen( "http://ftp3.fr.postgresql.=
org/pub/postgresql/source/"  =

).read() )
	for a in soup( 'a', {'href': re.compile( r"v\d+.\d+.\d+" ) } ):
		yield map( int, re.search( r"v(\d+)\.(\d+)\.(\d+)*", a['href']  =

).groups() )

def get_latest_versions():
	lastversions =3D {}
	for a,b,c in sorted( get_all_versions() ):
		lastversions[ (a,b) ] =3D c
	return sorted( lastversions.items() )

def parse_query_string( url ):
	return dict( map( urllib.unquote_plus, pair.split('=3D',1) ) for pair i=
n  =

re.split( "&(?:amp;|)", urllib.splitquery( url )[1] ) )

def get_mirrors():
	soup =3D  =

BeautifulSoup.BeautifulSoup( urllib.urlopen( "http://wwwmaster.postgresq=
l.org/download/mirrors-ftp"  =

).read() )
	for a in soup( 'a', {'href': re.compile( r"\?setmir=3D.*url=3D" ) } ):
		yield parse_query_string( a['href'] )['url']

def get_fastest_mirrors( urls, filename ):
	for url in urls:
		sys.stdout.write( "		%s\r" % url )
		t =3D time.time()
		try:
			urllib.urlopen( url + filename )
		except:
			pass
		d =3D time.time()-t
		print "%.02f s" % d
		yield d, url

for major, minor in get_latest_versions():
	print "%d.%d.%d" % (major[0], major[1], minor)

mirrors =3D get_mirrors()
fastest =3D sorted( get_fastest_mirrors( mirrors, "sync_timestamp" ))[:4=
]
for d, mirror in fastest:
	print "%.02f s	%s" % (d,mirror)


On Tue, 10 Apr 2007 00:34:02 +0200, Andrew Hammond  =

<andrew.george.hammond@gmail.com> wrote:

> On 4/9/07, CAJ CAJ <pguser@gmail.com> wrote:
>> On 9 Apr 2007 14:47:20 -0700, Andrew Hammond
>> <andrew.george.hammond@gmail.com> wrote:
>> > I'm writing a script that wants to know the latest release for a gi=
ven
>> > major.minor version. Is there some better way than parsing
>> > http://www.postgresql.org/ftp/source/ or trying to
>> connect to ftp
>> > (which is invariably timing out on me today. Is that box getting
>> > hammered or something?) and doing the parsing that? Both approaches=

>> > feel quite awkward to me.
>>
>> Use wget to download via  HTTP (added recently).  Probably wise to ad=
d a
>> couple mirrors in your script.
>
> I'm not asking how to download stuff. I'm asking how to figure out the=

> current release number for a given major.minor. I thought that was
> clear in my original post, but I guess not. For example, how do I
> determine (programmatically) the lastest version of 8.1.
>
> I'm also interested in a clever way to select one of the "close"
> mirrors at random for downloading via http. However I had planned to
> hold that question until I'd solved the first issue.
>
> Andrew
>
> ---------------------------(end of broadcast)-------------------------=
--
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq