Subject: [pgjdbc/pgjdbc] issue #3567: Surprising behaviour with jagged arrays

From: "jerrinot (@jerrinot)" <[email-masked]>
Date: Mon, 17 Mar 2025 11:15:14 +0000

**Describe the issue**
It appears codecs for 2D primitive arrays implicitly assume rectangular arrays. This leads to surprising behaviour when jagged arrays are used. In some cases, there are hard-to-interpret errors. In other instances, arrays are silently transformed into a regular shape. 

**Driver Version?** 
42.7.5

**Java Version?**
JDK 23

**OS Version?**
Linux

**PostgreSQL Version?**
16.4

**To Reproduce**
TestContainers reproducer:
```java
package info.jerrinot.sanbox.pgjdbc;

import org.junit.Test;

import java.sql.*;

public class JaggedArraysTest {

    private static final String JDBC_URL = "jdbc:tc:postgresql:16.4:///postgres";

    @Test
    public void testJaggedArrayBadMessage() throws SQLException {
        // this creates a bad binding message, server can detect it and throw an error
        // still, the error is not very informative and hard to tell what's wrong:
        // ERROR: insufficient data left in message
        //  Where: unnamed portal parameter $1

        try (Connection conn = DriverManager.getConnection(JDBC_URL);
             PreparedStatement stmt = conn.prepareStatement("select ? ")) {
            stmt.setArray(1, conn.createArrayOf("int8", new long[][]{{1L, 2L}, {3L}, {4L}}));
            try (ResultSet rs = stmt.executeQuery()) {

            }
        }
    }

    @Test
    public void testJaggedArraysMetamorphosis() throws SQLException {
        // changes shape of the array, creates a rectangular array from a jagged array
        // there is no error, no warning. with binary encoding the server has no way
        // to tell anything is wrong.

        try (Connection conn = DriverManager.getConnection(JDBC_URL);
             PreparedStatement stmt = conn.prepareStatement("select ? ")) {
            stmt.setArray(1, conn.createArrayOf("int8", new long[][]{
                    {1L, 2L},
                    {3L},
                    {4L, 5L, 6L}
            }));
            try (ResultSet rs = stmt.executeQuery()) {
                assert rs.next();

                Array array = rs.getArray(1);
                Long[][] items = (Long[][]) array.getArray();
                printArray(items);
            }
        }
    }

    private static void printArray(Long[][] items) {
        for (Long[] row : items) {
            for (Long item : row) {
                System.out.print(item + " ");
            }
            System.out.println();
        }
    }
}
```

deps:
```xml
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>postgresql</artifactId>
            <version>1.20.6</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <version>42.7.5</version>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.5.11</version>
        </dependency>
```

logback.xml
```
<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="info">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>
```

**Expected behaviour**
I'd expect client to validate the array is not jagged.

---

From: "davecramer (@davecramer)" <[email-masked]>
Date: Mon, 17 Mar 2025 11:16:55 +0000

Cool!. Does PostgreSQL support jagged arrays ?

---

From: "jerrinot (@jerrinot)" <[email-masked]>
Date: Mon, 17 Mar 2025 12:05:50 +0000

@davecramer wow, you are fast! 🙇 

PostgreSQL does not support jagged arrays. I guess the codec could validate that the array shape is rectangular. `ArrayEncoding` already [iterates over all rows](https://github.com/pgjdbc/pgjdbc/blob/d86d7ce19030230f14e573b8095f04721d1d8e35/pgjdbc/src/main/java/org/postgresql/jdbc/ArrayEncoding.java#L1236-L1239) anyway so it could validate the size of each row is the same. I can send a PR draft if that helps.

---

From: "davecramer (@davecramer)" <[email-masked]>
Date: Mon, 17 Mar 2025 12:10:27 +0000

> [@davecramer](https://github.com/davecramer) wow, you are fast! 🙇
> 
Depends on the day :)

> PostgreSQL does not support jagged arrays. I guess the codec could validate that the array shape is rectangular. `ArrayEncoding` already [iterates over all rows](https://github.com/pgjdbc/pgjdbc/bl…
Please do

---

From: "awadhesh14 (@awadhesh14)" <[email-masked]>
Date: Sat, 17 May 2025 08:51:58 +0000

@davecramer @jerrinot 

https://github.com/pgjdbc/pgjdbc/pull/3633 fixes the silent corruption and bad formatted message.

The changes:

- Validate array rectangularity during encoding
- Throw clear errors with indices (e.g., Jagged array at [1][0])
- Preserve NULL/empty array behavior
- Include tests

Please review when convenient.