pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Transforms for jsonb to PL/Python
9+ messages / 4 participants
[nested] [flat]
Commits from this thread:
2018-04-02 6278a2a262 Remove contrib/jsonb_plpython's tests for infinity and NaN conversions.
2018-04-02 beff4bb9c7 Teach configure --with-python to report the Python version number.
* pgsql: Transforms for jsonb to PL/Python
@ 2018-03-28 12:39 Peter Eisentraut <peter_e@gmx.net>
0 siblings, 1 reply; 9+ messages in thread
From: Peter Eisentraut @ 2018-03-28 12:39 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Transforms for jsonb to PL/Python
Add a new contrib module jsonb_plpython that provide a transform between
jsonb and PL/Python. jsonb values are converted to appropriate Python
types such as dicts and lists, and vice versa.
Author: Anthony Bykov <a.bykov@postgrespro.ru>
Reviewed-by: Aleksander Alekseev <a.alekseev@postgrespro.ru>
Reviewed-by: Nikita Glukhov <n.gluhov@postgrespro.ru>
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/3f44e3db72ad4097aae078c075a9b3cb3d6b761b
Modified Files
--------------
contrib/Makefile | 4 +-
contrib/jsonb_plpython/.gitignore | 6 +
contrib/jsonb_plpython/Makefile | 39 ++
contrib/jsonb_plpython/expected/jsonb_plpython.out | 347 ++++++++++++++++
contrib/jsonb_plpython/jsonb_plpython.c | 453 +++++++++++++++++++++
contrib/jsonb_plpython/jsonb_plpython2u--1.0.sql | 19 +
contrib/jsonb_plpython/jsonb_plpython2u.control | 6 +
contrib/jsonb_plpython/jsonb_plpython3u--1.0.sql | 19 +
contrib/jsonb_plpython/jsonb_plpython3u.control | 6 +
contrib/jsonb_plpython/jsonb_plpythonu--1.0.sql | 19 +
contrib/jsonb_plpython/jsonb_plpythonu.control | 6 +
contrib/jsonb_plpython/sql/jsonb_plpython.sql | 218 ++++++++++
doc/src/sgml/json.sgml | 15 +
13 files changed, 1155 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-03-28 14:19 Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
parent: Peter Eisentraut <peter_e@gmx.net>
0 siblings, 1 reply; 9+ messages in thread
From: Peter Eisentraut @ 2018-03-28 14:19 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
On 3/28/18 08:39, Peter Eisentraut wrote:
> Transforms for jsonb to PL/Python
Needs fixes for Windows build. I can give it a blind attempt, but help
would be welcome.
I'm looking into the regression test failures that appear to be minor
output differences.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-03-31 05:15 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 1 reply; 9+ messages in thread
From: Tom Lane @ 2018-03-31 05:15 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: pgsql-committers@lists.postgresql.org
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> I'm looking into the regression test failures that appear to be minor
> output differences.
gaur (python 2.5) is showing a different output pathology:
*** 246,253 ****
return x
$$;
SELECT test1inf();
! ERROR: could not convert value "inf" to jsonb
! CONTEXT: while creating return value
PL/Python function "test1inf"
-- test python -infinity -> jsonb
CREATE FUNCTION test2inf() RETURNS jsonb
--- 246,255 ----
return x
$$;
SELECT test1inf();
! ERROR: ValueError: invalid literal for float(): inf
! CONTEXT: Traceback (most recent call last):
! PL/Python function "test1inf", line 2, in <module>
! x = float('inf')
PL/Python function "test1inf"
-- test python -infinity -> jsonb
CREATE FUNCTION test2inf() RETURNS jsonb
***************
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=gaur&dt=2018-03-31%2002%3A50%3A13
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-04-01 22:07 Tom Lane <tgl@sss.pgh.pa.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 9+ messages in thread
From: Tom Lane @ 2018-04-01 22:07 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: pgsql-committers@lists.postgresql.org
I wrote:
> gaur (python 2.5) is showing a different output pathology:
A bit of google research indicates that Python did not have reliable
handling of infinity or NaN floats until 2.6; before that, you could
get failures or platform-specific results from code like float('inf').
So this test ain't gonna work as-is on old versions.
I see a few options:
1. Just remove the tests involving inf/nan. As far as I can see,
there's no explicit handling of these cases in jsonb_plpython,
so we're not really buying any test coverage of our own code anyway.
2. Maintain additional expected-output files. One problem with this
is it's not clear how many we'd need.
3. Somehow tweak the test cases in a different way, so that they're
not exercised before 2.6.
4. Move our minimum supported Python version up to 2.6 (from 2.4
where it is now).
#4 seems like an overreaction at first sight, but on the other
hand, I wonder how many people really care about pre-2.6 anymore
anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only
reason I'm running 2.5 on gaur/pademelon is we claim to support it.
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-04-02 00:16 Devrim Gündüz <devrim@gunduz.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 9+ messages in thread
From: Devrim Gündüz @ 2018-04-02 00:16 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: pgsql-committers@lists.postgresql.org
Hi,
On Sun, 2018-04-01 at 18:07 -0400, Tom Lane wrote:
> 4. Move our minimum supported Python version up to 2.6 (from 2.4
> where it is now).
>
> #4 seems like an overreaction at first sight, but on the other
> hand, I wonder how many people really care about pre-2.6 anymore
> anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only
> reason I'm running 2.5 on gaur/pademelon is we claim to support it.
FWIW, 2.6 is also the version that RHEL 6 has, so we still can support it.
Regards,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-04-02 14:59 Tom Lane <tgl@sss.pgh.pa.us>
parent: Devrim Gündüz <devrim@gunduz.org>
0 siblings, 1 reply; 9+ messages in thread
From: Tom Lane @ 2018-04-02 14:59 UTC (permalink / raw)
To: Devrim Gündüz <devrim@gunduz.org>; +Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; pgsql-committers@lists.postgresql.org
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes:
> On Sun, 2018-04-01 at 18:07 -0400, Tom Lane wrote:
>> 4. Move our minimum supported Python version up to 2.6 (from 2.4
>> where it is now).
>>
>> #4 seems like an overreaction at first sight, but on the other
>> hand, I wonder how many people really care about pre-2.6 anymore
>> anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only
>> reason I'm running 2.5 on gaur/pademelon is we claim to support it.
> FWIW, 2.6 is also the version that RHEL 6 has, so we still can support it.
I poked around in the buildfarm logs to see what versions we're actually
testing. It's hard to tell in some cases. (I notice that configure does
actually extract python_version, but doesn't display it ... should we make
it do so?) However, it's clear from version numbers embedded in places
like library paths that we are testing with python 2.4.x on aholehole,
arapaima, and prairiedog, and with 2.5.x on gaur/pademelon and locust.
aholehole and arapaima are evidently using the platform-supplied version
on RHEL 5, and locust the platform-supplied version on macOS 10.5.8.
So while those are surely trailing-edge platforms, I'm no longer thinking
that we can realistically claim there's zero remaining real-world interest
in these Python versions, and so option #4 doesn't seem workable.
I think option #1 (drop the tests with inf/nan) is the most attractive
of the remaining choices. It's surely the least work, and AFAICS
we aren't actually buying any code coverage with these test cases.
(It is interesting though that we have four other pre-2.6 platforms that
are passing these tests; apparently the pre-2.6 behavior wasn't broken
everywhere.)
Unless someone volunteers to undertake one of the other options,
I'll go remove these tests shortly.
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-04-02 15:25 Devrim Gündüz <devrim@gunduz.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 9+ messages in thread
From: Devrim Gündüz @ 2018-04-02 15:25 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; pgsql-committers@lists.postgresql.org
Hi,
On Mon, 2018-04-02 at 10:59 -0400, Tom Lane wrote:
> aholehole and arapaima are evidently using the platform-supplied version
> on RHEL 5, and locust the platform-supplied version on macOS 10.5.8.
Yeah, I'm running arapaima and aholehole, they have stock RHEL 5 packages
(Python 2.4)
> So while those are surely trailing-edge platforms, I'm no longer thinking
> that we can realistically claim there's zero remaining real-world interest
> in these Python versions, and so option #4 doesn't seem workable.
RHEL 5 is already EOL, last supported (community) RPM release was 9.6, so it
won't hurt to drop Python 2.4 support from RPM point of view.
Regards,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-04-02 15:42 Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 9+ messages in thread
From: Peter Eisentraut @ 2018-04-02 15:42 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org
On 4/1/18 18:07, Tom Lane wrote:
> 1. Just remove the tests involving inf/nan. As far as I can see,
> there's no explicit handling of these cases in jsonb_plpython,
> so we're not really buying any test coverage of our own code anyway.
That makes sense.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Transforms for jsonb to PL/Python
@ 2018-04-02 16:36 Tom Lane <tgl@sss.pgh.pa.us>
parent: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2018-04-02 16:36 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: pgsql-committers@lists.postgresql.org
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 4/1/18 18:07, Tom Lane wrote:
>> 1. Just remove the tests involving inf/nan. As far as I can see,
>> there's no explicit handling of these cases in jsonb_plpython,
>> so we're not really buying any test coverage of our own code anyway.
> That makes sense.
OK, done.
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2018-04-02 16:36 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-03-28 12:39 pgsql: Transforms for jsonb to PL/Python Peter Eisentraut <peter_e@gmx.net>
2018-03-28 14:19 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2018-03-31 05:15 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-04-01 22:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-04-02 00:16 ` Devrim Gündüz <devrim@gunduz.org>
2018-04-02 14:59 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-04-02 15:25 ` Devrim Gündüz <devrim@gunduz.org>
2018-04-02 15:42 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2018-04-02 16:36 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox