From: Jan Urbański <wulczer@wulczer.org>
To: Postgres - Hackers <pgsql-hackers@postgresql.org>
Subject: plpython SPI cursors
Date: Sun, 16 Oct 2011 01:28:56 +0200
Message-ID: <4E9A1738.6090108@wulczer.org> (raw)
Hi,
attached is a patch implementing the usage of SPI cursors in PL/Python.
Currently when trying to process a large table in PL/Python you have
slurp it all into memory (that's what plpy.execute does).
This patch allows reading the result set in smaller chunks, using a SPI
cursor behind the scenes.
Example usage:
cursor = plpy.cursor("select a, b from hugetable")
for row in cursor:
plpy.info("a is %s and b is %s" % (row['a'], row['b']))
The patch itself is simple, but there's a lot of boilerplate dedicated
to opening a subtransaction and handling prepared plans. I'd like to do
some refactoring of they way PL/Python uses SPI to reduce the amount of
boilerplate needed, but that'll come as a separate patch (just before
the patch to split plpython.c in smaller chunks).
This feature has been sponsored by Nomao.
Cheers,
Jan
PS: I already added it to the November CF.
J
Attachments:
[text/x-diff] 0001-Add-cursor-support-to-plpythonu.patch (0B, ../4E9A1738.6090108@wulczer.org/2-0001-Add-cursor-support-to-plpythonu.patch)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: wulczer@wulczer.org
Subject: Re: plpython SPI cursors
In-Reply-To: <4E9A1738.6090108@wulczer.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox