pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  

help / color / mirror / Atom feed

plpythonu + os.spawnv
2+ messages / 1 participants
[nested] [flat]

* plpythonu + os.spawnv
@ 2008-03-08 01:12  Markus Schatten <markus.schatten@foi.hr>
  0 siblings, 1 reply; 2+ messages in thread

From: Markus Schatten @ 2008-03-08 01:12 UTC (permalink / raw)
  To: pgsql-sql

Dear all,

I'm not sure if I'm at the right place to ask my question so please excuse me 
if I'm not.

I'm trying to spawn a process from plpythonu using psql  8.1.11. The code is 
working fine from Python shell but if I put it into a plpythonu function it 
seems like the process isn't starting at all.

The code I'm using is (with additional comment on the important parts):

CREATE OR REPLACE FUNCTION top_inline_query( VARCHAR( 50 ), TEXT )
	RETURNS TEXT
	AS $$
	
import os
import re
import sets
import random

def f2vars( string ):
  vars_re = re.compile( r'[?][^-:,\]\[]*' )
  all_vars = vars_re.findall( string )
  set_vars = sets.Set( all_vars )
  for i in all_vars:
    if i.startswith( '?_' ):
      try:
        set_vars.remove( i )
      except:
        pass
  
  return list( set_vars )

''' THE SPAWN FUNCTION '''
def run(program, *args):
    return os.spawnv(os.P_WAIT, program, (program,) + args)


def f2payser( query, module, path ):
    ''' THE PROCESS TO BE RUN '''
    florahome = '/path/to/flora2/runflora' #where runflora is a shell script 
that runs a shell-like environment (take a look at http://flora.sf.net)

  
  quote_re = re.compile( r"['][^']*[']" )
  quotes = quote_re.findall( query )
  
  for i in quotes:
    cp = i.replace( ' ', r'\s' )
    query = query.replace( i, cp )  
  
  query = query[ :-1 ].replace( ' ', '' )
  
  query_vars = f2vars( query )
  #print query_vars
  
  
  vars_print = ''
  for i in query_vars:
    vars_print += ",%write('" + i + "=')@_io,%write(" + i + ")@_io,%nl@_io"
  
  
  randfilename = path + str( random.random() )
  while os.path.exists( randfilename ):
    randfilename = path + str( random.random() )
  
  '''THE QUERY STRING TO BE PASSED TO FLORA'''
  query_str = "-e \"['" + module + "'].\" -e \"%tell('" + randfilename 
+ "')@_io," + query + vars_print + ",%nl@_io.\" -e \"_halt.\""
  
  plpy.info( florahome + ' ' + str( query_str ) )
  run(florahome, query_str)
  
  vars_re = re.compile( '([?].*)[=](.*)')
  
  result = {}
  results = []
  
  warning = False
  try:
    file = open( randfilename, 'r' )
  except:
    file = []

  for i in file:
    val_pair = vars_re.findall( i )
    if len( val_pair ) == 1:
      if not ( val_pair[ 0 ][ 1 ].startswith( '(' ) and val_pair[ 0 ][ 
1 ].endswith( ')' ) and val_pair[ 0 ][ 1 ].find( '?_' ) != -1 ):
        result[ val_pair[ 0 ][ 0 ] ] = val_pair[ 0 ][ 1 ]
      else:
        warning = True
    elif val_pair == []:
      if result != {}:
        if not warning:
          results.append( result )
          result = {}
        warning = False
        
    else:
      raise ValueError,'Syntax error!'
  try:
    os.remove( randfilename )
  except:
    pass
  
  res_copy = results[ : ]
  for i in res_copy:
    results.remove( i )
    if not i in results:
      results.append( i )
  return results

project = args[ 0 ]
query = args[ 1 ]

'''THE ORIGINAL QUERY COMMENTED OUT '''
#onto_query = 'SELECT top_export_flora2_ontology( \'%s\' ) AS ontology;' % 
project

#ontology = plpy.execute( onto_query )[ 0 ][ 'ontology' ]

ontology = 'markus:person[ name->markus, surename->schatten ].\n'


path = '/path/to/chmoded/directory/to/store/files/in'
randfilename = path + str( int( random.random() * 1000000 ) ) + '.flr'
while os.path.exists( randfilename ):
  randfilename = path + str( int( random.random() * 1000000 ) ) + '.flr'

onto_file = open( randfilename, 'w' )
onto_file.write( ontology )
onto_file.close()
os.chmod( randfilename, 0755 )

module = randfilename[ :-4 ]


results = f2payser( query, module, path )

'''CLEAN UP'''
os.remove( randfilename )

return str( results )

  $$
  LANGUAGE plpythonu;

The funny thing is that the same code (as said before) works fine when called 
from a Python environment so my question is (if I'm not missing something in 
the code) is there a difference in handling spawned processes in plpthonu 
and "normal" python?

The query passed to FLORA reads an ontology file queries it and writes the 
results to another file that is than read by the plpythonu function. I tried 
a lot of different approaches but none of them seems to be starting the FLORA 
process at all. It seem like plpythonu just runs over the spawn function 
doing nothing continuing the execution of the rest of the code.

Any help would be very appreciated because I'm running out of ideas ;)

Best regards,

-- 
Markus Schatten, MSc
Faculty of Organization and Informatics
Varaždin, Croatia
e-mail: markus.schatten@foi.hr
http://www.foi.hr



^ permalink  raw  reply  [nested|flat] 2+ messages in thread

* Re: plpythonu + os.spawnv
@ 2008-03-08 14:07  Markus Schatten <markus.schatten@foi.hr>
  parent: Markus Schatten <markus.schatten@foi.hr>
  0 siblings, 0 replies; 2+ messages in thread

From: Markus Schatten @ 2008-03-08 14:07 UTC (permalink / raw)
  To: pgsql-sql

Dear all,

I was right about posting to the wrong place ;) The error was that a folder in 
the path to XSB (that is called from FLORA) wasn't chmoded in the right way.

Thanks anyway ;)

Best regards
-- 
Markus Schatten, MSc
Faculty of Organization and Informatics
Varaždin, Croatia
e-mail: markus.schatten@foi.hr
http://www.foi.hr



^ permalink  raw  reply  [nested|flat] 2+ messages in thread

end of thread, other threads:[~2008-03-08 14:07 UTC | newest]

Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2008-03-08 01:12 plpythonu + os.spawnv Markus Schatten <markus.schatten@foi.hr>
2008-03-08 14:07 ` Markus Schatten <markus.schatten@foi.hr>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox