public inbox for [email protected]  

help / color / mirror / Atom feed

From: Listmail <[email protected]>
To: Andrew Hammond <[email protected]>
To: CAJ CAJ <[email protected]>
Cc: [email protected]
Subject: Re: programmatic way to fetch latest release for a given major.minor version
Date: Tue, 10 Apr 2007 01:36:02 +0200
Message-ID: <op.tqjiicjuzcizji@apollo13> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>
	<[email protected]>

	Here you go.

	Fetches versions and prints most recent minor for each major
	Tests all mirrors for speed and prints out the 4 fastest (takes some time)
	http://www.crummy.com/software/BeautifulSoup/

	Have a nice day !

#! /bin/env python
# -*- coding: utf-8  -*-

import urllib, BeautifulSoup, re, time, sys


def get_all_versions():
	soup =  
BeautifulSoup.BeautifulSoup( urllib.urlopen( "http://ftp3.fr.postgresql.org/pub/postgresql/source/";  
).read() )
	for a in soup( 'a', {'href': re.compile( r"v\d+.\d+.\d+" ) } ):
		yield map( int, re.search( r"v(\d+)\.(\d+)\.(\d+)*", a['href']  
).groups() )

def get_latest_versions():
	lastversions = {}
	for a,b,c in sorted( get_all_versions() ):
		lastversions[ (a,b) ] = c
	return sorted( lastversions.items() )

def parse_query_string( url ):
	return dict( map( urllib.unquote_plus, pair.split('=',1) ) for pair in  
re.split( "&(?:amp;|)", urllib.splitquery( url )[1] ) )

def get_mirrors():
	soup =  
BeautifulSoup.BeautifulSoup( urllib.urlopen( "http://wwwmaster.postgresql.org/download/mirrors-ftp";  
).read() )
	for a in soup( 'a', {'href': re.compile( r"\?setmir=.*url=" ) } ):
		yield parse_query_string( a['href'] )['url']

def get_fastest_mirrors( urls, filename ):
	for url in urls:
		sys.stdout.write( "		%s\r" % url )
		t = time.time()
		try:
			urllib.urlopen( url + filename )
		except:
			pass
		d = time.time()-t
		print "%.02f s" % d
		yield d, url

for major, minor in get_latest_versions():
	print "%d.%d.%d" % (major[0], major[1], minor)

mirrors = get_mirrors()
fastest = sorted( get_fastest_mirrors( mirrors, "sync_timestamp" ))[:4]
for d, mirror in fastest:
	print "%.02f s	%s" % (d,mirror)









On Tue, 10 Apr 2007 00:34:02 +0200, Andrew Hammond  
<[email protected]> wrote:

> On 4/9/07, CAJ CAJ <[email protected]> wrote:
>> On 9 Apr 2007 14:47:20 -0700, Andrew Hammond
>> <[email protected]> wrote:
>> > I'm writing a script that wants to know the latest release for a given
>> > major.minor version. Is there some better way than parsing
>> > http://www.postgresql.org/ftp/source/ or trying to
>> connect to ftp
>> > (which is invariably timing out on me today. Is that box getting
>> > hammered or something?) and doing the parsing that? Both approaches
>> > feel quite awkward to me.
>>
>> Use wget to download via  HTTP (added recently).  Probably wise to add a
>> couple mirrors in your script.
>
> I'm not asking how to download stuff. I'm asking how to figure out the
> current release number for a given major.minor. I thought that was
> clear in my original post, but I guess not. For example, how do I
> determine (programmatically) the lastest version of 8.1.
>
> I'm also interested in a clever way to select one of the "close"
> mirrors at random for downloading via http. However I had planned to
> hold that question until I'd solved the first issue.
>
> Andrew
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq

view thread (30+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected]
  Subject: Re: programmatic way to fetch latest release for a given major.minor version
  In-Reply-To: <op.tqjiicjuzcizji@apollo13>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox