From: davecramer (@davecramer) <noreply+davecramer@github.com>
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
Subject: Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
Date: Thu, 05 Nov 2026 11:21:10 +0000
Message-ID: <gh-pgjdbc-pgjdbc-4331-r5784054162@github.com> (raw)
In-Reply-To: <gh-pgjdbc-pgjdbc-4331@github.com>
References: <gh-pgjdbc-pgjdbc-4331@github.com>
I read the change and ran it against PostgreSQL here. The approach is right, and it fixes #4328: `idleStreamSurvivesSocketReadTimeout` fails with "Tried to write to an inactive copy operation" when I revert the `QueryExecutorImpl` change, and passes with it. `LogicalReplicationTest`, `LogicalReplicationStatusTest`, `PhysicalReplicationTest`, `CopyBothResponseTest`, `ReplicationSlotTest`, `ReplicationConnectionTest` and `CopyTest` all pass on `246e29f`.
I found one regression, and I would like it fixed before this goes in.
### 1. A COPY on a closed connection still reports itself active
`abortAfterCopyFailure()` keeps `lockedFor` set, and `CopyOperationImpl.isActive()` is `hasLockOn(this)`. So once the COPY has closed the connection, `isActive()` returns true. In 42.7.11 to 42.7.13 it returned false, because the lock was released. Three callers decide from it:
- `CopyManager.copyIn(sql, InputStream)` calls `cancelCopy()` in its `finally` block. That writes CopyFail to the closed socket, and the exception it throws replaces the write failure the caller should see.
- `CopyManager.copyOut` calls `cancelCopy()`, which for a CopyOut opens a new connection to send a cancel request.
- `PGReplicationStream.isClosed()` is `closeFlag || !copyDual.isActive()`, so it returns false on a closed connection.
I checked it with a test that cuts the connection through `StrangeProxyServer` and then writes a 1 MB row:
| | `con.isClosed()` | `copyIn.isActive()` | `CopyManager.copyIn` throws |
| --- | --- | --- | --- |
| master | false | false | "Database connection failed when writing to copy" |
| this PR | true | true | "Database connection failed when canceling copy operation" |
| this PR with the change below | true | false | "Database connection failed when writing to copy" |
The fix I tried is one line in `hasLockOn` (GitHub will not take a suggestion there, because the line is outside this diff):
```java
boolean hasLockOn(@Nullable Object holder) {
try (ResourceLock ignore = lock.obtain()) {
return lockedFor == holder && !isClosed();
}
}
```
`cancelCopy` and the other internal paths use the private `hasLock`, so they are unaffected. With it, every test listed above still passes.
Here is the test. It checks where the exception was created and not its message, because the message is translated and CI runs with `fr_FR` and `tr_TR`. Both tests fail on this PR and pass with the fix. They could go into `CopyTest`.
<details><summary>CopyAfterAbortReproTest.java</summary>
```java
/*
* Copyright (c) 2026, PostgreSQL Global Development Group
* See the LICENSE file in the project root for more information.
*/
package org.postgresql.test.jdbc2;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import org.postgresql.PGConnection;
import org.postgresql.PGProperty;
import org.postgresql.copy.CopyIn;
import org.postgresql.copy.CopyManager;
import org.postgresql.test.TestUtil;
import org.postgresql.test.util.StrangeProxyServer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Timeout;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.Arrays;
import java.util.Properties;
import java.util.concurrent.TimeUnit;
class CopyAfterAbortReproTest {
private static Properties viaProxy(StrangeProxyServer proxy) {
Properties props = new Properties();
TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
String.valueOf(proxy.getServerPort()));
return props;
}
/** A row larger than the output buffer, so writeToCopy reaches the socket. */
private static byte[] bigRow() {
byte[] row = new byte[1024 * 1024];
Arrays.fill(row, (byte) 'x');
row[row.length - 1] = '\n';
return row;
}
/**
* A COPY whose connection has been closed can never end, so isActive()
* has to report false. Callers such as CopyManager.copyIn decide from it
* whether to cancel the COPY.
*/
@Test
@Timeout(value = 30, unit = TimeUnit.SECONDS)
void copyIsInactiveOnceItsConnectionIsClosed() throws Exception {
try (StrangeProxyServer proxy = new StrangeProxyServer(TestUtil.getServer(),
TestUtil.getPort());
Connection con = TestUtil.openDB(viaProxy(proxy))) {
TestUtil.createTempTable(con, "copyrepro_a", "data text");
CopyManager manager = con.unwrap(PGConnection.class).getCopyAPI();
CopyIn copyIn = manager.copyIn("COPY copyrepro_a (data) FROM STDIN");
proxy.closeAllClients();
byte[] row = bigRow();
assertThrows(SQLException.class, () -> copyIn.writeToCopy(row, 0, row.length));
assertTrue(con.isClosed(), "The failed write should close the connection");
assertFalse(copyIn.isActive(),
"A COPY on a closed connection should not report itself active");
}
}
/**
* When the connection drops during CopyManager.copyIn, the caller has to
* see the write that failed. A cancel attempted afterwards on the closed
* connection must not replace it.
*/
@Test
@Timeout(value = 30, unit = TimeUnit.SECONDS)
void copyInReportsTheWriteThatFailed() throws Exception {
try (StrangeProxyServer proxy = new StrangeProxyServer(TestUtil.getServer(),
TestUtil.getPort());
Connection con = TestUtil.openDB(viaProxy(proxy))) {
TestUtil.createTempTable(con, "copyrepro_b", "data text");
CopyManager manager = con.unwrap(PGConnection.class).getCopyAPI();
byte[] first = "a\n".getBytes(StandardCharsets.UTF_8);
byte[] big = bigRow();
// One small row, then the connection drops and a row too large for
// the output buffer follows
InputStream source = new InputStream() {
private int calls;
@Override
public int read() {
throw new UnsupportedOperationException();
}
@Override
public int read(byte[] b, int off, int len) {
calls++;
if (calls == 1) {
System.arraycopy(first, 0, b, off, first.length);
return first.length;
}
if (calls == 2) {
proxy.closeAllClients();
int n = Math.min(len, big.length);
System.arraycopy(big, 0, b, off, n);
return n;
}
return -1;
}
};
SQLException e = assertThrows(SQLException.class,
() -> manager.copyIn("COPY copyrepro_b (data) FROM STDIN", source,
1024 * 1024));
// The stack shows where the exception was created; the message text
// is translated, so it cannot be compared under every locale
assertTrue(createdIn(e, "writeToCopy"),
"copyIn should report the failed write, got: " + e);
assertFalse(createdIn(e, "cancelCopy"),
"copyIn should not report the cancel that followed, got: " + e);
}
}
private static boolean createdIn(Throwable t, String method) {
return Arrays.stream(t.getStackTrace())
.anyMatch(f -> f.getClassName().endsWith(".QueryExecutorImpl")
&& f.getMethodName().equals(method));
}
}
```
</details>
### 2. The rebase, and a smaller diff
`QueryExecutorImpl.java` conflicts with master, which has changed `processCopyResults` since this branch was cut (`receiveMessageType`, `receiveMessageLength`, `protocolViolation`). Most of the 393 changed lines in that file are indentation: `git diff -w` shows 82 added and 25 removed. The indentation comes from the new `try` around the whole `switch`.
The logic could be turned around: catch `SocketTimeoutException` only around the reads that happen at a message boundary (`hasMessagePending()`, `peekChar()` and `receiveMessageType()`), and treat every other timeout as one inside a message. The `switch` would stay as it is, the conflict would be small, and a `case` added later would be treated as inside a message without anyone having to remember it. Master's `messageEnd` check does not replace your check, because `messageEnd` is set only after the length is read.
### 3. The recoverable timeout looks like a failed connection
The timeout that keeps the COPY active is reported as "Database connection failed when reading from copy" with `08006`, the same as a real failure. Only `getCause()` tells them apart. That was already true before this PR, but the two now have different outcomes, so I think the recoverable one should have its own message.
### 4. The test timeout default
`8c95e8c31` makes `SEPARATE_THREAD` the default for the whole build. Each new test here already sets `threadMode = SEPARATE_THREAD`, so the fix does not need the global change. I would rather see it in its own PR.
There is one more inline comment, on the wording at `unlock()`.