From: Markus Schatten <markus.schatten@foi.hr>
To: pgsql-sql@postgresql.org
Subject: plpythonu + os.spawnv
Date: Sat, 8 Mar 2008 02:12:55 +0100
Message-ID: <200803080212.55150.markus.schatten@foi.hr> (raw)
Dear all,
I'm not sure if I'm at the right place to ask my question so please excuse me
if I'm not.
I'm trying to spawn a process from plpythonu using psql 8.1.11. The code is
working fine from Python shell but if I put it into a plpythonu function it
seems like the process isn't starting at all.
The code I'm using is (with additional comment on the important parts):
CREATE OR REPLACE FUNCTION top_inline_query( VARCHAR( 50 ), TEXT )
RETURNS TEXT
AS $$
import os
import re
import sets
import random
def f2vars( string ):
vars_re = re.compile( r'[?][^-:,\]\[]*' )
all_vars = vars_re.findall( string )
set_vars = sets.Set( all_vars )
for i in all_vars:
if i.startswith( '?_' ):
try:
set_vars.remove( i )
except:
pass
return list( set_vars )
''' THE SPAWN FUNCTION '''
def run(program, *args):
return os.spawnv(os.P_WAIT, program, (program,) + args)
def f2payser( query, module, path ):
''' THE PROCESS TO BE RUN '''
florahome = '/path/to/flora2/runflora' #where runflora is a shell script
that runs a shell-like environment (take a look at http://flora.sf.net)
quote_re = re.compile( r"['][^']*[']" )
quotes = quote_re.findall( query )
for i in quotes:
cp = i.replace( ' ', r'\s' )
query = query.replace( i, cp )
query = query[ :-1 ].replace( ' ', '' )
query_vars = f2vars( query )
#print query_vars
vars_print = ''
for i in query_vars:
vars_print += ",%write('" + i + "=')@_io,%write(" + i + ")@_io,%nl@_io"
randfilename = path + str( random.random() )
while os.path.exists( randfilename ):
randfilename = path + str( random.random() )
'''THE QUERY STRING TO BE PASSED TO FLORA'''
query_str = "-e \"['" + module + "'].\" -e \"%tell('" + randfilename
+ "')@_io," + query + vars_print + ",%nl@_io.\" -e \"_halt.\""
plpy.info( florahome + ' ' + str( query_str ) )
run(florahome, query_str)
vars_re = re.compile( '([?].*)[=](.*)')
result = {}
results = []
warning = False
try:
file = open( randfilename, 'r' )
except:
file = []
for i in file:
val_pair = vars_re.findall( i )
if len( val_pair ) == 1:
if not ( val_pair[ 0 ][ 1 ].startswith( '(' ) and val_pair[ 0 ][
1 ].endswith( ')' ) and val_pair[ 0 ][ 1 ].find( '?_' ) != -1 ):
result[ val_pair[ 0 ][ 0 ] ] = val_pair[ 0 ][ 1 ]
else:
warning = True
elif val_pair == []:
if result != {}:
if not warning:
results.append( result )
result = {}
warning = False
else:
raise ValueError,'Syntax error!'
try:
os.remove( randfilename )
except:
pass
res_copy = results[ : ]
for i in res_copy:
results.remove( i )
if not i in results:
results.append( i )
return results
project = args[ 0 ]
query = args[ 1 ]
'''THE ORIGINAL QUERY COMMENTED OUT '''
#onto_query = 'SELECT top_export_flora2_ontology( \'%s\' ) AS ontology;' %
project
#ontology = plpy.execute( onto_query )[ 0 ][ 'ontology' ]
ontology = 'markus:person[ name->markus, surename->schatten ].\n'
path = '/path/to/chmoded/directory/to/store/files/in'
randfilename = path + str( int( random.random() * 1000000 ) ) + '.flr'
while os.path.exists( randfilename ):
randfilename = path + str( int( random.random() * 1000000 ) ) + '.flr'
onto_file = open( randfilename, 'w' )
onto_file.write( ontology )
onto_file.close()
os.chmod( randfilename, 0755 )
module = randfilename[ :-4 ]
results = f2payser( query, module, path )
'''CLEAN UP'''
os.remove( randfilename )
return str( results )
$$
LANGUAGE plpythonu;
The funny thing is that the same code (as said before) works fine when called
from a Python environment so my question is (if I'm not missing something in
the code) is there a difference in handling spawned processes in plpthonu
and "normal" python?
The query passed to FLORA reads an ontology file queries it and writes the
results to another file that is than read by the plpythonu function. I tried
a lot of different approaches but none of them seems to be starting the FLORA
process at all. It seem like plpythonu just runs over the spawn function
doing nothing continuing the execution of the rest of the code.
Any help would be very appreciated because I'm running out of ideas ;)
Best regards,
--
Markus Schatten, MSc
Faculty of Organization and Informatics
Varaždin, Croatia
e-mail: markus.schatten@foi.hr
http://www.foi.hr
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: markus.schatten@foi.hr
Subject: Re: plpythonu + os.spawnv
In-Reply-To: <200803080212.55150.markus.schatten@foi.hr>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox