Subject: Regarding useObjects

From: Achilleas Mantzios - cloud <[email-masked]>
Date: Tue, 26 Sep 2023 18:38:55 +0300

Dear All

I see support for useObjects was dropped some time in 2020, with this 
commit : 529e5dc3a8f071ad4946daf3bde1422c21a99524 .

Desperate mode question : was it replaced by smth else ? Any workaround ?

Serious mode question : Since eventually we'll have to bite the bullet 
and do an upgrade, how would you propose dealing with 1000s lines of 
java code using arrays of primitives ?

thank you

---

From: Vladimir Sitnikov <[email-masked]>
Date: Tue, 26 Sep 2023 19:07:12 +0300

> I see support for useObjects was dropped some time in 2020, with this commit : 529e5dc3a8f071ad4946daf3bde1422c21a99524 .

Could you provide a test case that demonstrates the failure?
It is hard to understand what you mean by "useObjects was dropped"

Vladimir

---

From: Achilleas Mantzios <[email-masked]>
Date: Tue, 26 Sep 2023 19:58:49 +0300

> It is hard to understand what you mean by "useObjects was dropped"

There used to be a variable inside : 
src/main/java/org/postgresql/jdbc/PgArray.java which was like :

private final boolean useObjects;
this.useObjects = true;

so there was the option to change this to false and recompile. This 
variable was removed between REL42.2.0 and REL42.3.0 . So not quite 
support, sorry for my wording.

-- 
Achilleas Mantzios
  IT DEV - HEAD
  IT DEPT
  Dynacom Tankers Mgmt

---

From: Vladimir Sitnikov <[email-masked]>
Date: Wed, 27 Sep 2023 09:03:11 +0300

> so there was the option to change this to false and recompile

We try doing our best to keep backward compatibility both
compilation-wise and behaviour-wise.
However, we treat "private fields and methods" as private, and we do
not expect people would depend on them.
If you have to modify sources or depend on private APIs I would
suggest raising an issue or PR so it could be incorporated
into the driver itself.

Would you please provide a test case of what you are doing with PgArray?
What is exactly the sequence of JDBC calls, the actual and the
expected outcomes in your case?

The mentioned "useObjects" has been removed in
https://github.com/pgjdbc/pgjdbc/pull/1194 (Refactor decoding arrays)

Vladimir

---

From: Achilleas Mantzios - cloud <[email-masked]>
Date: Wed, 27 Sep 2023 11:59:57 +0300

Good Day Vladimir

> expected outcomes in your case?

E.g. for the case of floats, we have to change all:

java.sql.Array arr = rs.getArray(1);

Object objarr = arr.getArray();

float flarr[] = (float[]) objarr;

to

java.sql.Array arr = rs.getArray(1);

Object objarr = arr.getArray();

Float flarr[] = (Float[]) objarr;

---

From: Dave Cramer <[email-masked]>
Date: Wed, 27 Sep 2023 05:44:45 -0400

> Float flarr[] = (Float[]) objarr;

So the problem with using float instead of Float is that it is impossible
to have a null float and arrays can have nulls.

Dave

---

From: Vladimir Sitnikov <[email-masked]>
Date: Wed, 27 Sep 2023 12:55:35 +0300

There's a feature request for retrieving primitive arrays:
https://github.com/pgjdbc/pgjdbc/issues/2939

Vladimir

---

From: Achilleas Mantzios - cloud <[email-masked]>
Date: Wed, 27 Sep 2023 13:00:09 +0300

On 9/27/23 12:44, Dave Cramer wrote:
>
>
> On Wed, 27 Sept 2023 at 05:00, Achilleas Mantzios - cloud 
> <[email-masked]> wrote:
>
>     Good Day Vladimir
>
> So the problem with using float instead of Float is that it is 
> impossible to have a null float and arrays can have nulls.

Thanks Dave, we wrote much of the code before 2008, and we had to 
implement our own version of nulls in arrays of primitives. So this code 
was stuck everywhere. Plus we got like 120+ remote sites (slaves) ( 
which run ... even older code) so we must fully support them as well. 
Its only about 200 source files in the central system's repo, so we will 
have to do it sooner or later.

The hard part would be migrating from pgsql 10 to 16.

---

From: Achilleas Mantzios - cloud <[email-masked]>
Date: Wed, 27 Sep 2023 13:07:39 +0300

> https://github.com/pgjdbc/pgjdbc/issues/2939

back in pre-postgresql-42* dayswe used to use -888 and -888.888 to 
represent int NULL and float NULL respectively, quite ...primitive but 
it worked. We should retain this old mapping , but once all systems are 
up to date we will start storing and reading NULLs in arrays as NULLs.

---

From: Dave Cramer <[email-masked]>
Date: Wed, 27 Sep 2023 06:17:37 -0400

On Wed, 27 Sept 2023 at 06:07, Achilleas Mantzios - cloud <
[email-masked]> wrote:

> back in pre-postgresql-42* days we used to use -888 and -888.888 to
> represent int NULL and float NULL respectively, quite ...primitive but it
> worked. We should retain this old mapping , but once all systems are up to
> date we will start storing and reading NULLs in arrays as NULLs.
>
Using that mapping is not something that we would entertain.

If you are going to actually store NULL in the array, how would that work
with primitives ?

Dave

---

From: Achilleas Mantzios - cloud <[email-masked]>
Date: Wed, 27 Sep 2023 13:28:42 +0300

On 9/27/23 13:17, Dave Cramer wrote:
>
>
> On Wed, 27 Sept 2023 at 06:07, Achilleas Mantzios - cloud 
>     represent int NULL and float NULL respectively, quite ...primitive
>     but it worked. We should retain this old mapping , but once all
>     systems are up to date we will start storing and reading NULLs in
>     arrays as NULLs.
>
> If you are going to actually store NULL in the array, how would that 
> work with primitives ?

We support our central system (master) plus 120 slave systems 
communicating via satellites (running very old versions of everything , 
linux, postgersql , java, jdbc, etc) using custom replication code (a 
hack of DBmirror) that we wrote back in 2003 or so. Data come back and 
forth, so we have to support all those 120 archaic slaves while we 
upgrade and after we upgrade the central system.

When everything is up to date, (which will be some years from now), then 
we will start actually storing NULLs inside arrays instead of -888 and 
-888.888 . BUT even then, we are not willing to update the actual old 
data (-888) to the new NULL version, because that would trigger a 
massive replication traffic from our central system to the slaves. Also 
updating all past / historic values would cause bloating, huge 
autovacuum activity, among other things on those remote slaves that run 
unmanned (in simple english : I am the only DBA for all 1 + 120 
postgresql instances) . So we should support historic data as they are, 
until something bigger happens (like a vast update in our 
topology/logic/etc).

---

From: Dave Cramer <[email-masked]>
Date: Wed, 27 Sep 2023 06:39:27 -0400

On Wed, 27 Sept 2023 at 06:28, Achilleas Mantzios - cloud <
[email-masked]> wrote:

> linux, postgersql , java, jdbc, etc) using custom replication code (a hack
> of DBmirror) that we wrote back in 2003 or so. Data come back and forth, so
> we have to support all those 120 archaic slaves while we upgrade and after
> we upgrade the central system.
>
> -888.888 . BUT even then, we are not willing to update the actual old data
> (-888) to the new NULL version, because that would trigger a massive
> replication traffic from our central system to the slaves. Also updating
> all past / historic values would cause bloating, huge autovacuum activity,
> among other things on those remote slaves that run unmanned (in simple
> english : I am the only DBA for all 1 + 120 postgresql instances) . So we
> should support historic data as they are, until something bigger happens
> (like a vast update in our topology/logic/etc).
>

So it sounds like this would work for you if someone implemented primitive
arrays.

Dave

---

From: Achilleas Mantzios - cloud <[email-masked]>
Date: Wed, 27 Sep 2023 14:11:42 +0300

On 9/27/23 13:39, Dave Cramer wrote:
>
>
> On Wed, 27 Sept 2023 at 06:28, Achilleas Mantzios - cloud 
>>>         > So the problem with using float instead of Float is that
>>>         it is impossible to have a null float and arrays can have nulls.
>>>
>>         back in pre-postgresql-42* dayswe used to use -888 and
>>         -888.888 to represent int NULL and float NULL respectively,
>>         quite ...primitive but it worked. We should retain this old
>>         mapping , but once all systems are up to date we will start
>>         storing and reading NULLs in arrays as NULLs.
>>
>>     If you are going to actually store NULL in the array, how would
>>     that work with primitives ?
>
>     communicating via satellites (running very old versions of
>     everything , linux, postgersql , java, jdbc, etc) using custom
>     replication code (a hack of DBmirror) that we wrote back in 2003
>     or so. Data come back and forth, so we have to support all those
>     120 archaic slaves while we upgrade and after we upgrade the
>     central system.
>
>     When everything is up to date, (which will be some years from
>     now), then we will start actually storing NULLs inside arrays
>     instead of -888 and -888.888 . BUT even then, we are not willing
>     to update the actual old data (-888) to the new NULL version,
>     because that would trigger a massive replication traffic from our
>     central system to the slaves. Also updating all past / historic
>     values would cause bloating, huge autovacuum activity, among other
>     things on those remote slaves that run unmanned (in simple english
>     : I am the only DBA for all 1 + 120 postgresql instances) . So we
>     should support historic data as they are, until something bigger
>     happens (like a vast update in our topology/logic/etc).
>
>
> So it sounds like this would work for you if someone implemented 
> primitive arrays.

Yes or we could just upgrade JDBC driver in the central system, and 
touch those ~ 200 source files to just work (not throw exception), but 
still retaining the -888 representation.

But an implementation of primitive arrays for ints,longs,floats,doubles 
would be better at this stage.

---

From: DB Kho <[email-masked]>
Date: Wed, 4 Oct 2023 19:45:32 +0200 (CEST)

Unsubscribe

Op 27-09-2023 12:17 schreef Dave Cramer <[email-masked]>:

On Wed, 27 Sept 2023 at 06:07, Achilleas Mantzios - cloud <[email-masked]> wrote:

On 9/27/23 12:55, Vladimir Sitnikov wrote:

> So the problem with using float instead of Float is that it is impossible to have a null float and arrays can have nulls.

There's a feature request for retrieving primitive arrays: https://github.com/pgjdbc/pgjdbc/issues/2939

back in pre-postgresql-42* days we used to use -888 and -888.888 to represent int NULL and float NULL respectively, quite ...primitive but it worked. We should retain this old mapping , but once all systems are up to date we will start storing and reading NULLs in arrays as NULLs.

If you are going to actually store NULL in the array, how would that work with primitives ?

Dave

Vladimir