pg.ddx.io GitHub: pgjdbc/pgjdbc (issues + PRs as email threads)
help / color / mirror / Atom feed[pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
14+ messages / 4 participants
[nested] [flat]
* [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-07-30 13:05 "vlsi (@vlsi)" <noreply+vlsi@github.com>
0 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-07-30 13:05 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
## Why
Since 42.7.11 a logical replication consumer that uses the blocking `PGReplicationStream.read()` API dies as soon as the slot is idle for one status interval, with `PSQLException: Tried to write to an inactive copy operation` ([#4328](https://github.com/pgjdbc/pgjdbc/issues/4328)).
`V3ReplicationProtocol.configureSocketTimeout` sets `SO_TIMEOUT` to `min(socketTimeout, statusInterval)` on purpose: the blocking read has to wake up periodically to send a standby status update. [b5080e76f](https://github.com/pgjdbc/pgjdbc/commit/b5080e76f) (the fix for [#3957](https://github.com/pgjdbc/pgjdbc/issues/3957), first released in 42.7.11) made `readFromCopy` release the COPY on every `IOException`, and `SocketTimeoutException` is one. So the routine idle timeout deactivated the COPY, and the status update that follows it in `V3PGReplicationStream.readInternal` failed. The reproducer needs a non-zero `socketTimeout`: without it `VisibleBufferedInputStream.readMore` swallows the timeout and retries, so the exception never surfaces.
The same regression hits a plain `COPY TO STDOUT` with `socketTimeout`: after a read timeout the operation is no longer active, so `cancelCopy()` throws `Tried to cancel an inactive copy operation` and the server is left in COPY mode while the driver believes the operation is over.
## What
**A read timeout keeps the COPY.** `readFromCopy` catches `SocketTimeoutException` separately and reports it without touching the lock. This is the only failure a caller can resume from, by reading again. The driver already treats a read timeout this way in `processNotifies`.
**The COPY lock is never released on an I/O failure.** Releasing it claimed the protocol stream was free while it was desynchronized, which is what took `cancelCopy()` away. Instead the driver gives the connection up with `abort()` — what every other I/O failure in `QueryExecutorImpl` already does — and `waitOnLock()` stops waiting once the connection is closed. That preserves the #3957 fix (operations hanging in `waitOnLock()` after a COPY died) without deciding the lock state from the exception type, which was the wrong question to ask: even a `SocketTimeoutException` is unrecoverable when it lands mid-message, and the set of "safe" exceptions is not closed.
**`unlock()` signals every waiter.** Threads wait in `waitOnLock()` without taking the lock, so `signal()` woke one of them and left the rest waiting for a copy that was already over.
**`startCopy` and `cancelCopy` give the connection up too.** Neither can be retried — a second `startCopy` would send the query twice — so a live connection whose response is still on the wire is worse than a closed one.
**A read timeout inside a message is not one to carry on from** (last commit, added after review). Keeping the COPY is sound at a message boundary and nowhere else: `VisibleBufferedInputStream.read()` hands out `buffer[index++]`, so past the type byte a caller that reads again takes the middle of the interrupted message for the start of the next one. `processCopyResults` now separates the two — a `SocketTimeoutException` raised while reading a body becomes a plain `IOException` and reaches `abortAfterCopyFailure`, while a timeout waiting for the next message to start is untouched.
`readTimeoutKeepsCopyActive` was asserting the unsound half without meaning to. `StrangeProxyServer` stops forwarding wherever it happens to be, and after the first `readFromCopy` the 8 KB buffer holds a partial message, so the timeout it observed landed inside one. It now starves the reader with `MessageStallProxyServer`, which forwards whole messages and can therefore pause on a boundary.
**Test timeouts default to `SEPARATE_THREAD`** (separate commit, droppable on its own). A test blocked in a socket read ignores `Thread.interrupt()`, so a same-thread timeout cannot stop it: JUnit reports the deadline only once the read returns, and a read that never returns hangs the build. Without this, a stalled replication test turns into a hung job rather than a failed one. It also covers `PhysicalReplicationTest` and `CopyBothResponseTest`, which have no `@Timeout` at all and relied on the global 5-minute default.
## How to verify
Six new tests, each checked to fail without its part of the change:
| test | what it pins |
| --- | --- |
| `CopyTest.readTimeoutKeepsCopyActive` | a reader starved **between messages** gets a timeout, the COPY stays active, `cancelCopy()` still works |
| `CopyTest.readTimeoutInsideMessageEndsTheCopy` | a reader starved **inside** a message is told the connection is gone, not handed something to resume from |
| `CopyTest.waitingThreadGivesUpWhenCopyDiesWithConnection` | a thread already parked on the COPY lock is woken and told the connection is gone |
| `CopyTest.allWaitingThreadsResumeWhenCopyEnds` | all three waiters resume, not just one |
| `CopyTest.startCopyFailureClosesConnection` | a COPY that cannot read its start response closes the connection |
| `CopyTest.cancelCopyFailureClosesConnection` | a cancel that never reached the server closes the connection |
| `LogicalReplicationTest.idleStreamSurvivesSocketReadTimeout` | a blocking read idles through several status intervals and keeps streaming |
`TestUtil.openReplicationConnection` gained a `Consumer<Properties>` overload, mirroring `openPrivilegedDB`, so the replication test can set `socketTimeout` itself instead of depending on the CI axis.
Verified against PostgreSQL 18.4 with SSL and SCRAM, both with the plain configuration and with the matrix flags of the job that first showed this (`socketTimeout=60`, `queryTimeout=15`, `connectTimeout=7`, `autosave=always`, `cleanupSavepoints=true`, `-XX:hashCode=2`, JaCoCo): full suite 9746 tests green, autostyle and checkstyle clean.
## Known issue
`endCopy` gives the connection up on a read timeout, where `readFromCopy` resumes from the same event. Reading again would be safe in itself — CopyDone is on the wire before the loop starts, so waiting for the answer a second time re-sends nothing — but resuming needs more than that catch, and the shipped callers are written for the current contract:
- `CopyManager.copyIn` ends with `finally { if (cp.isActive()) cp.cancelCopy(); }`. A copy left active by a timeout would take that branch and send CopyFail after CopyDone, which the backend ignores, so `cancelCopy` throws `Missing expected error response to copy cancel request` out of the `finally` and the real timeout is lost.
- `PGCopyOutputStream.close()` sets `this.op = null` after its try, so an `endCopy` that throws leaves the stream holding an operation whose lock nothing gives back, on a connection that is still open — the next borrower of a pooled connection then waits in `waitOnLock()` forever.
- `PGCopyOutputStream.endCopy()` does not reset its buffer offset, so a retry through the public stream would re-send up to 64 KB of CopyData.
Teaching those three the retryable contract is a change to the COPY API's behaviour rather than a fix for #4328, so it is left out of this PR. Recorded in the `abortAfterCopyFailure` javadoc as well, rather than only here.
Who this reaches: a caller that sets `socketTimeout` and ends a COPY the server is slow to acknowledge. **The stacked #4354 removes it for replication**, the case that would have hit it routinely: a replication stream shortens its wait to the status interval, so `V3PGReplicationStream.close()` used to run `endCopy` under a deadline of milliseconds. After #4354 the connection's own timeout is never shortened, and the stream puts its wake-up back before calling `endCopy`.
## Notes
- A cross-review of the stacked #4354 turned up that this PR's own test was asserting the unsound half of "a read timeout keeps the COPY", so the mid-message case is now handled here rather than left alone. See the last commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
inline patch 1: CHANGELOG.md, build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts, pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java and 4 more (+1023/-168 lines) download inline-1.patch
diff --git a/CHANGELOG.md b/CHANGELOG.md
index aa062160d6..468cd869ae 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
### Fixed
* fix: `LargeObjectManager` operations now work inside an active XA transaction. Since 42.7.13 an XA branch keeps the caller's `autoCommit=true`, and the guard rejected large objects as if the connection were idle. It now checks the server transaction state (`getTransactionState()`) instead of `autoCommit`, so a genuine auto-commit connection with no open transaction is still refused [Issue #4309](https://github.com/pgjdbc/pgjdbc/issues/4309) [PR #4310](https://github.com/pgjdbc/pgjdbc/pull/4310)
* fix: batch update counts are no longer secured as committed inside an XA transaction. `BatchResultHandler` treated per-statement progress as durable whenever `autoCommit=true`, but an XA branch keeps `autoCommit=true` while the transaction is open, so a batch that flushed mid-way and then failed could report rolled-back statements as succeeded. Progress is now secured only when no server transaction is open [Issue #4309](https://github.com/pgjdbc/pgjdbc/issues/4309) [PR #4311](https://github.com/pgjdbc/pgjdbc/pull/4311)
+* fix: a socket read timeout no longer ends a COPY operation. Logical replication sets the socket timeout to the status interval and uses the timeout as its wake-up to send a standby status update, so since 42.7.11 a consumer of the blocking `PGReplicationStream.read()` API failed with "Tried to write to an inactive copy operation" as soon as the slot went idle. A plain `COPY TO STDOUT` with `socketTimeout` lost `cancelCopy()` the same way, leaving the server in COPY mode. A COPY that dies with the connection now closes that connection rather than releasing its lock, so other operations fail instead of hanging. A timeout that lands inside a backend message is a different matter and now ends the connection: the bytes the driver already took are not in the stream any more, so carrying on from there would read the middle of that message as the start of the next one [Issue #4328](https://github.com/pgjdbc/pgjdbc/issues/4328) [PR #4331](https://github.com/pgjdbc/pgjdbc/pull/4331)
## [42.7.13] (2026-07-06)
diff --git a/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts b/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts
index 4007539786..0205830b59 100644
--- a/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts
+++ b/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts
@@ -42,4 +42,7 @@ tasks.configureEach<Test> {
// See https://github.com/junit-team/junit5/commit/347e3119d36a5c226cddd7981452f11335fad422
passProperty("junit.jupiter.execution.parallel.config.strategy", "DYNAMIC")
passProperty("junit.jupiter.execution.timeout.default", "5 m")
+ // A test blocked in a socket read ignores Thread.interrupt(), so a same-thread timeout cannot
+ // stop it and the build hangs instead of reporting the failure
+ passProperty("junit.jupiter.execution.timeout.thread.mode.default", "SEPARATE_THREAD")
}
diff --git a/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java b/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java
index 6bdc2a538a..cbee23c609 100644
--- a/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java
+++ b/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java
@@ -298,7 +298,9 @@ private void unlock(Object holder) throws PSQLException {
PSQLState.OBJECT_NOT_IN_STATE);
}
lockedFor = null;
- lockCondition.signal();
+ // Threads park in waitOnLock() without taking the lock, so waking one of them would leave
+ // the rest waiting for a copy that is already over
+ lockCondition.signalAll();
}
/**
@@ -307,6 +309,11 @@ private void unlock(Object holder) throws PSQLException {
*/
private void waitOnLock() throws PSQLException {
while (lockedFor != null) {
+ if (isClosed()) {
+ // The copy holding the lock died with the connection, so it will never release it
+ throw new PSQLException(GT.tr("This connection has been closed."),
+ PSQLState.CONNECTION_DOES_NOT_EXIST);
+ }
try {
lockCondition.await();
} catch (InterruptedException ie) {
@@ -318,6 +325,30 @@ private void waitOnLock() throws PSQLException {
}
}
+ /**
+ * Gives up on the connection after an I/O failure during COPY. A copy that holds the lock keeps
+ * it, since a desynchronized protocol stream is not free for anyone else to use; closing the
+ * connection is what lets threads waiting in {@link #waitOnLock()} fail instead of waiting for a
+ * copy that can no longer end.
+ *
+ * <p>The COPY methods call this for every I/O failure except one: a read timeout in
+ * {@link #readFromCopy}, the only failure a caller can resume from by reading again. A failed
+ * send has already put part of a message on the wire, and {@link #startCopy} cannot be retried,
+ * since that would send the query a second time.</p>
+ *
+ * <p>{@link #endCopy} gives the connection up on a read timeout too, though reading again would
+ * be safe there as well: CopyDone is on the wire before the loop starts, so waiting for the
+ * answer a second time re-sends nothing. Resuming would need a limit on how long that answer may
+ * take, since a server that never sends it would otherwise keep the loop going, and the deadline
+ * the caller asked for is the only limit there is today.</p>
+ */
+ private void abortAfterCopyFailure() {
+ abort();
+ try (ResourceLock ignore = lock.obtain()) {
+ lockCondition.signalAll();
+ }
+ }
+
/**
* @param holder object assumed to hold the lock
* @return whether given object actually holds the lock
@@ -1116,6 +1147,7 @@ public CopyOperation startCopy(String sql, boolean suppressBegin)
return castNonNull(processCopyResults(null, true));
// expect a CopyInResponse or CopyOutResponse to our query above
} catch (IOException ioe) {
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when starting copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1193,13 +1225,12 @@ public void cancelCopy(CopyOperationImpl op) throws SQLException {
}
} catch (IOException ioe) {
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when canceling copy operation"),
PSQLState.CONNECTION_FAILURE, ioe);
} finally {
- // Need to ensure the lock isn't held anymore, or else
- // future operations, rather than failing due to the
- // broken connection, will simply hang waiting for this
- // lock.
+ // Canceling ends the copy either way, so the lock has to go back. Otherwise later
+ // operations would wait for a copy that is already over.
try (ResourceLock ignore = lock.obtain()) {
if (hasLock(op)) {
unlock(op);
@@ -1243,15 +1274,19 @@ public long endCopy(CopyOperationImpl op) throws SQLException {
processCopyResults(op, true);
} while (hasLock(op));
return op.getHandledRowCount();
+ } catch (SQLException e) {
+ // A copy the server failed has already been unlocked by the ReadyForQuery that followed
+ // its error, and that connection is fine. Still holding the lock means the exchange
+ // stopped somewhere else, so the copy is neither over nor resumable and nothing may wait
+ // on a lock it will never get back
+ if (hasLock(op)) {
+ abortAfterCopyFailure();
+ }
+ throw e;
} catch (IOException ioe) {
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when ending copy"),
PSQLState.CONNECTION_FAILURE, ioe);
- } finally {
- // Release the lock if still held, otherwise future operations
- // will hang indefinitely in waitOnLock()
- if (hasLock(op)) {
- unlock(op);
- }
}
}
}
@@ -1281,9 +1316,7 @@ public void writeToCopy(CopyOperationImpl op, byte[] data, int off, int siz)
pgStream.sendInteger4(siz + 4);
pgStream.send(data, off, siz);
} catch (IOException ioe) {
- if (hasLock(op)) {
- unlock(op);
- }
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when writing to copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1314,9 +1347,7 @@ public void writeToCopy(CopyOperationImpl op, ByteStreamWriter from)
pgStream.sendInteger4(siz + 4);
pgStream.send(from);
} catch (IOException ioe) {
- if (hasLock(op)) {
- unlock(op);
- }
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when writing to copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1333,9 +1364,7 @@ public void flushCopy(CopyOperationImpl op) throws SQLException {
try {
pgStream.flush();
} catch (IOException ioe) {
- if (hasLock(op)) {
- unlock(op);
- }
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when writing to copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1359,12 +1388,14 @@ void readFromCopy(CopyOperationImpl op, boolean block) throws SQLException {
try {
processCopyResults(op, block); // expect a call to handleCopydata() to store the data
+ } catch (SocketTimeoutException ste) {
+ // A read timeout leaves the socket usable, so the copy stays active and the caller can
+ // read again. Replication depends on that: the stream sets the socket timeout to the
+ // status interval and treats the timeout as its wake-up to send a standby status update.
+ throw new PSQLException(GT.tr("Database connection failed when reading from copy"),
+ PSQLState.CONNECTION_FAILURE, ste);
} catch (IOException ioe) {
- // Release the lock if still held after a connection failure,
- // otherwise future operations will hang indefinitely in waitOnLock()
- if (hasLock(op)) {
- unlock(op);
- }
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when reading from copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1408,194 +1439,220 @@ void readFromCopy(CopyOperationImpl op, boolean block) throws SQLException {
SQLException errors = null;
int len;
- while (!endReceiving && (block || pgStream.hasMessagePending())) {
-
- // There is a bug in the server's implementation of the copy
- // protocol. It returns command complete immediately upon
- // receiving the EOF marker in the binary protocol,
- // potentially before we've issued CopyDone. When we are not
- // blocking, we don't think we are done, so we hold off on
- // processing command complete and any subsequent messages
- // until we actually are done with the copy.
- //
- if (!block) {
- int c = pgStream.peekChar();
- if (c == PgMessageType.COMMAND_COMPLETE_RESPONSE) {
- LOGGER.log(Level.FINEST, " <=BE CommandStatus, Ignored until CopyDone");
- break;
+ try {
+ while (!endReceiving && (block || pgStream.hasMessagePending())) {
+
+ // There is a bug in the server's implementation of the copy
+ // protocol. It returns command complete immediately upon
+ // receiving the EOF marker in the binary protocol,
+ // potentially before we've issued CopyDone. When we are not
+ // blocking, we don't think we are done, so we hold off on
+ // processing command complete and any subsequent messages
+ // until we actually are done with the copy.
+ //
+ if (!block) {
+ int c = pgStream.peekChar();
+ if (c == PgMessageType.COMMAND_COMPLETE_RESPONSE) {
+ LOGGER.log(Level.FINEST, " <=BE CommandStatus, Ignored until CopyDone");
+ break;
+ }
}
- }
- int c = pgStream.receiveChar();
- switch (c) {
+ int c = pgStream.receiveChar();
+ try {
+ switch (c) {
- case PgMessageType.ASYNCHRONOUS_NOTICE:
- LOGGER.log(Level.FINEST, " <=BE Asynchronous Notification while copying");
+ case PgMessageType.ASYNCHRONOUS_NOTICE:
+ LOGGER.log(Level.FINEST, " <=BE Asynchronous Notification while copying");
- receiveAsyncNotify();
- break;
+ receiveAsyncNotify();
+ break;
- case PgMessageType.NOTICE_RESPONSE:
+ case PgMessageType.NOTICE_RESPONSE:
- LOGGER.log(Level.FINEST, " <=BE Notification while copying");
+ LOGGER.log(Level.FINEST, " <=BE Notification while copying");
- addWarning(receiveNoticeResponse());
- break;
+ addWarning(receiveNoticeResponse());
+ break;
- case PgMessageType.COMMAND_COMPLETE_RESPONSE: // Command Complete
+ case PgMessageType.COMMAND_COMPLETE_RESPONSE: // Command Complete
- String status = receiveCommandStatus();
+ String status = receiveCommandStatus();
- try {
- if (op == null) {
- throw new PSQLException(GT
- .tr("Received CommandComplete ''{0}'' without an active copy operation", status),
- PSQLState.OBJECT_NOT_IN_STATE);
- }
- op.handleCommandStatus(status);
- } catch (SQLException se) {
- error = se;
- }
+ try {
+ if (op == null) {
+ throw new PSQLException(GT
+ .tr("Received CommandComplete ''{0}'' without an active copy operation", status),
+ PSQLState.OBJECT_NOT_IN_STATE);
+ }
+ op.handleCommandStatus(status);
+ } catch (SQLException se) {
+ error = se;
+ }
- block = true;
- break;
+ block = true;
+ break;
- case PgMessageType.ERROR_RESPONSE: // ErrorMessage (expected response to CopyFail)
+ case PgMessageType.ERROR_RESPONSE: // ErrorMessage (expected response to CopyFail)
- error = receiveErrorResponse();
- // We've received the error and we now expect to receive
- // Ready for query, but we must block because it might still be
- // on the wire and not here yet.
- block = true;
- break;
+ error = receiveErrorResponse();
+ // We've received the error and we now expect to receive
+ // Ready for query, but we must block because it might still be
+ // on the wire and not here yet.
+ block = true;
+ break;
- case PgMessageType.COPY_IN_RESPONSE: // CopyInResponse
+ case PgMessageType.COPY_IN_RESPONSE: // CopyInResponse
- LOGGER.log(Level.FINEST, " <=BE CopyInResponse");
+ LOGGER.log(Level.FINEST, " <=BE CopyInResponse");
- if (op != null) {
- error = new PSQLException(GT.tr("Got CopyInResponse from server during an active {0}",
- op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);
- }
+ if (op != null) {
+ error = new PSQLException(GT.tr("Got CopyInResponse from server during an active {0}",
+ op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);
+ }
- op = new CopyInImpl();
- initCopy(op);
- endReceiving = true;
- break;
+ op = new CopyInImpl();
+ initCopy(op);
+ endReceiving = true;
+ break;
- case PgMessageType.COPY_OUT_RESPONSE: // CopyOutResponse
+ case PgMessageType.COPY_OUT_RESPONSE: // CopyOutResponse
- LOGGER.log(Level.FINEST, " <=BE CopyOutResponse");
+ LOGGER.log(Level.FINEST, " <=BE CopyOutResponse");
- if (op != null) {
- error = new PSQLException(GT.tr("Got CopyOutResponse from server during an active {0}",
- op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);
- }
+ if (op != null) {
+ error = new PSQLException(GT.tr("Got CopyOutResponse from server during an active {0}",
+ op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);
+ }
- op = new CopyOutImpl();
- initCopy(op);
- endReceiving = true;
- break;
+ op = new CopyOutImpl();
+ initCopy(op);
+ endReceiving = true;
+ break;
- case PgMessageType.COPY_BOTH_RESPONSE: // CopyBothResponse
+ case PgMessageType.COPY_BOTH_RESPONSE: // CopyBothResponse
- LOGGER.log(Level.FINEST, " <=BE CopyBothResponse");
+ LOGGER.log(Level.FINEST, " <=BE CopyBothResponse");
- if (op != null) {
- error = new PSQLException(GT.tr("Got CopyBothResponse from server during an active {0}",
- op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);
- }
+ if (op != null) {
+ error = new PSQLException(GT.tr("Got CopyBothResponse from server during an active {0}",
+ op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);
+ }
- op = new CopyDualImpl();
- initCopy(op);
- endReceiving = true;
- break;
+ op = new CopyDualImpl();
+ initCopy(op);
+ endReceiving = true;
+ break;
- case PgMessageType.COPY_DATA: // CopyData
+ case PgMessageType.COPY_DATA: // CopyData
- LOGGER.log(Level.FINEST, " <=BE CopyData");
+ LOGGER.log(Level.FINEST, " <=BE CopyData");
- len = pgStream.receiveInteger4() - 4;
+ len = pgStream.receiveInteger4() - 4;
- assert len > 0 : "Copy Data length must be greater than 4";
+ assert len > 0 : "Copy Data length must be greater than 4";
- byte[] buf = pgStream.receive(len);
- if (op == null) {
- error = new PSQLException(GT.tr("Got CopyData without an active copy operation"),
- PSQLState.OBJECT_NOT_IN_STATE);
- } else if (!(op instanceof CopyOut)) {
- error = new PSQLException(
- GT.tr("Unexpected copydata from server for {0}", op.getClass().getName()),
- PSQLState.COMMUNICATION_ERROR);
- } else {
- op.handleCopydata(buf);
- }
- endReceiving = true;
- break;
+ byte[] buf = pgStream.receive(len);
+ if (op == null) {
+ error = new PSQLException(GT.tr("Got CopyData without an active copy operation"),
+ PSQLState.OBJECT_NOT_IN_STATE);
+ } else if (!(op instanceof CopyOut)) {
+ error = new PSQLException(
+ GT.tr("Unexpected copydata from server for {0}", op.getClass().getName()),
+ PSQLState.COMMUNICATION_ERROR);
+ } else {
+ op.handleCopydata(buf);
+ }
+ endReceiving = true;
+ break;
- case PgMessageType.COPY_DONE: // CopyDone (expected after all copydata received)
+ case PgMessageType.COPY_DONE: // CopyDone (expected after all copydata received)
- LOGGER.log(Level.FINEST, " <=BE CopyDone");
+ LOGGER.log(Level.FINEST, " <=BE CopyDone");
- len = pgStream.receiveInteger4() - 4;
- if (len > 0) {
- pgStream.receive(len); // not in specification; should never appear
- }
+ len = pgStream.receiveInteger4() - 4;
+ if (len > 0) {
+ pgStream.receive(len); // not in specification; should never appear
+ }
- if (!(op instanceof CopyOut)) {
- error = new PSQLException("Got CopyDone while not copying from server",
- PSQLState.OBJECT_NOT_IN_STATE);
- }
+ if (!(op instanceof CopyOut)) {
+ error = new PSQLException("Got CopyDone while not copying from server",
+ PSQLState.OBJECT_NOT_IN_STATE);
+ }
- // keep receiving since we expect a CommandComplete
- block = true;
- break;
- case PgMessageType.PARAMETER_STATUS_RESPONSE: // Parameter Status
- try {
- receiveParameterStatus();
- } catch (SQLException e) {
- error = e;
- endReceiving = true;
- }
- break;
+ // keep receiving since we expect a CommandComplete
+ block = true;
+ break;
+ case PgMessageType.PARAMETER_STATUS_RESPONSE: // Parameter Status
+ try {
+ receiveParameterStatus();
+ } catch (SQLException e) {
+ error = e;
+ endReceiving = true;
+ }
+ break;
- case PgMessageType.READY_FOR_QUERY_RESPONSE: // ReadyForQuery: After FE:CopyDone => BE:CommandComplete
+ case PgMessageType.READY_FOR_QUERY_RESPONSE: // ReadyForQuery: After FE:CopyDone => BE:CommandComplete
- receiveRFQ();
- if (op != null && hasLock(op)) {
- unlock(op);
- }
- op = null;
- endReceiving = true;
- break;
+ receiveRFQ();
+ if (op != null && hasLock(op)) {
+ unlock(op);
+ }
+ op = null;
+ endReceiving = true;
+ break;
- // If the user sends a non-copy query, we've got to handle some additional things.
- //
- case PgMessageType.ROW_DESCRIPTION_RESPONSE: // Row Description (response to Describe)
- LOGGER.log(Level.FINEST, " <=BE RowDescription (during copy ignored)");
+ // If the user sends a non-copy query, we've got to handle some additional things.
+ //
+ case PgMessageType.ROW_DESCRIPTION_RESPONSE: // Row Description (response to Describe)
+ LOGGER.log(Level.FINEST, " <=BE RowDescription (during copy ignored)");
- skipMessage();
- break;
+ skipMessage();
+ break;
- case PgMessageType.DATA_ROW_RESPONSE: // DataRow
- LOGGER.log(Level.FINEST, " <=BE DataRow (during copy ignored)");
+ case PgMessageType.DATA_ROW_RESPONSE: // DataRow
+ LOGGER.log(Level.FINEST, " <=BE DataRow (during copy ignored)");
- skipMessage();
- break;
+ skipMessage();
+ break;
- default:
- throw new IOException(
- GT.tr("Unexpected packet type during copy: {0}", Integer.toString(c)));
- }
+ default:
+ throw new IOException(
+ GT.tr("Unexpected packet type during copy: {0}", Integer.toString(c)));
+ }
+ } catch (SocketTimeoutException ste) {
+ // The type byte is already out of the stream, so there is no way back to a
+ // message boundary. Report it as the connection failure it is rather than as
+ // something the caller can resume from, which would read the middle of this
+ // message as the start of the next one
+ IOException desynchronized = new IOException(GT.tr(
+ "Read timed out inside a backend message, so the copy stream is out of step"), ste);
+ if (errors != null) {
+ // The backend had already reported this copy failed. That is why the copy is over;
+ // losing the message boundary is only why the connection goes with it
+ desynchronized.addSuppressed(errors);
+ }
+ throw desynchronized;
+ }
- // Collect errors into a neat chain for completeness
- if (error != null) {
- if (errors != null) {
- error.setNextException(errors);
+ // Collect errors into a neat chain for completeness
+ if (error != null) {
+ if (errors != null) {
+ error.setNextException(errors);
+ }
+ errors = error;
+ error = null;
}
- errors = error;
- error = null;
}
+
+ } catch (SocketTimeoutException ste) {
+ if (errors != null) {
+ // The backend has already reported this copy failed. Handing the caller a timeout
+ // instead would drop that and read as "nothing arrived yet", so a caller that reads
+ // again could see the copy end successfully
+ throw errors;
+ }
+ throw ste;
}
if (errors != null) {
diff --git a/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.java b/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.java
index 890329fe2f..e02859adf6 100644
--- a/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.java
+++ b/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.java
@@ -11,6 +11,7 @@
import static org.junit.jupiter.api.Assumptions.assumeTrue;
import org.postgresql.PGConnection;
+import org.postgresql.PGProperty;
import org.postgresql.core.BaseConnection;
import org.postgresql.core.ServerVersion;
import org.postgresql.test.TestUtil;
@@ -45,6 +46,12 @@
@EnabledForServerVersionRange(gte = "9.4")
class LogicalReplicationTest {
private static final String SLOT_NAME = "pgjdbc_logical_replication_slot";
+ private static final int STATUS_INTERVAL_MS = 200;
+ /**
+ * Slot for {@link #idleStreamSurvivesSocketReadTimeout()}, which streams over a second connection
+ * of its own. Sharing {@link #SLOT_NAME} would put two consumers on one slot.
+ */
+ private static final String IDLE_SLOT_NAME = SLOT_NAME + "_idle";
private Connection replConnection;
private Connection sqlConnection;
@@ -632,6 +639,70 @@ public Object call() throws Exception {
);
}
+ /**
+ * A replication stream sets the socket timeout to the status interval and uses the resulting read
+ * timeout as its wake-up to send a standby status update. The blocking read has to survive that
+ * timeout, however many times it fires while the source is idle.
+ *
+ * <p>The connection sets {@code socketTimeout} on purpose: without it
+ * {@link org.postgresql.core.VisibleBufferedInputStream#setTimeoutRequested(boolean)} leaves the
+ * driver retrying the read instead of reporting the timeout, so the wake-up path is never
+ * exercised.</p>
+ *
+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/4328">Issue 4328</a>
+ */
+ @Test
+ @Timeout(30)
+ void idleStreamSurvivesSocketReadTimeout() throws Exception {
+ // The blocking read times out after min(socketTimeout, statusInterval), so it wakes up every
+ // status interval and socketTimeout stays the connection-level guard
+ TestUtil.recreateLogicalReplicationSlot(sqlConnection, IDLE_SLOT_NAME, "test_decoding");
+ try (Connection connection = TestUtil.openReplicationConnection(
+ props -> PGProperty.SOCKET_TIMEOUT.set(props, 10))) {
+ PGReplicationStream stream =
+ ((PGConnection) connection)
+ .getReplicationAPI()
+ .replicationStream()
+ .logical()
+ .withSlotName(IDLE_SLOT_NAME)
+ .withStartPosition(getCurrentLSN())
+ .withSlotOption("include-xids", false)
+ .withSlotOption("skip-empty-xacts", true)
+ .withStatusInterval(STATUS_INTERVAL_MS, TimeUnit.MILLISECONDS)
+ .start();
+
+ ExecutorService executor = Executors.newSingleThreadExecutor();
+ try {
+ // Nothing happens on the source for several status intervals, so the read spends that
+ // time waking up on socket timeouts and sending status updates
+ Future<?> insert = executor.submit(() -> {
+ TimeUnit.MILLISECONDS.sleep(STATUS_INTERVAL_MS * 10);
+ try (Statement st = sqlConnection.createStatement()) {
+ st.execute("insert into test_logic_table(name) values('after idle')");
+ }
+ return null;
+ });
+
+ String result = group(receiveMessage(stream, 3));
+ insert.get(10, TimeUnit.SECONDS);
+
+ String wait = group(Arrays.asList(
+ "BEGIN",
+ "table public.test_logic_table: INSERT: pk[integer]:1 name[character varying]:'after idle'",
+ "COMMIT"
+ ));
+
+ assertThat("The stream should keep streaming after idling through several socket timeouts",
+ result, equalTo(wait)
+ );
+ } finally {
+ executor.shutdownNow();
+ }
+ } finally {
+ TestUtil.dropReplicationSlot(sqlConnection, IDLE_SLOT_NAME);
+ }
+ }
+
@Test
void restartReplicationFromRestartSlotLSNWhenFeedbackAbsent() throws Exception {
PGConnection pgConnection = (PGConnection) replConnection;
diff --git a/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.java b/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.java
index 3ff95fcb55..828704a336 100644
--- a/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.java
+++ b/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.java
@@ -7,6 +7,7 @@
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTimeout;
@@ -23,6 +24,7 @@
import org.postgresql.core.ServerVersion;
import org.postgresql.jdbc.PgConnection;
import org.postgresql.test.TestUtil;
+import org.postgresql.test.util.MessageStallProxyServer;
import org.postgresql.test.util.StrangeProxyServer;
import org.postgresql.util.ByteBufferByteStreamWriter;
import org.postgresql.util.PSQLState;
@@ -39,6 +41,7 @@
import java.io.OutputStream;
import java.io.PrintStream;
import java.io.StringReader;
+import java.net.SocketTimeoutException;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
@@ -46,9 +49,17 @@
import java.sql.SQLException;
import java.sql.Statement;
import java.time.Duration;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
import java.util.Locale;
import java.util.Properties;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
/**
* @author kato@iki.fi
@@ -701,4 +712,462 @@ void isValidShouldNotHangAfterCopyInputStreamNetworkFailure() throws Exception {
}
}
}
+
+ /**
+ * A socket read timeout is not a connection failure, so it must leave the COPY operation active.
+ * Deactivating the COPY takes away {@link CopyOut#cancelCopy()}, the only way out of a stalled
+ * one, and it takes away the wake-up logical replication sends its status updates on.
+ *
+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/4328">Issue 4328</a>
+ */
+ @Test
+ @Timeout(value = 60, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void readTimeoutKeepsCopyActive() throws Exception {
+ TestUtil.createTable(con, "copytest_timeout", "data text");
+ try {
+ try (Statement stmt = con.createStatement()) {
+ stmt.execute(
+ "INSERT INTO copytest_timeout SELECT 'row' || g FROM generate_series(1, 20000) g");
+ }
+
+ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+ // Without socketTimeout the driver retries the read instead of reporting the timeout
+ PGProperty.SOCKET_TIMEOUT.set(props, 1);
+ // The cancel request goes through the same dead proxy, so do not wait 10s for its EOF
+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);
+ // The proxy reads the v3 framing to find a message boundary, so it needs plain traffic
+ PGProperty.SSL_MODE.set(props, "disable");
+ PGProperty.GSS_ENC_MODE.set(props, "disable");
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+ CopyOut copyOut = manager.copyOut("COPY copytest_timeout TO STDOUT");
+ assertNotNull(copyOut.readFromCopy(), "Expected data row from COPY TO STDOUT");
+
+ // Starve the reader between messages, which is the only place a timeout leaves the
+ // stream somewhere the caller can carry on from
+ proxyServer.stallBeforeNextMessage(TimeUnit.SECONDS.toMillis(30));
+
+ SQLException timeout = readUntilTimeout(copyOut);
+ assertInstanceOf(SocketTimeoutException.class, timeout.getCause(),
+ () -> "readFromCopy should report the socket read timeout, got " + timeout);
+ assertTrue(copyOut.isActive(),
+ "A socket read timeout must not deactivate the COPY operation");
+
+ // Canceling is the way out of a stalled COPY, and it needs the COPY lock
+ copyOut.cancelCopy();
+ assertFalse(copyOut.isActive(), "cancelCopy() should end the COPY operation");
+ }
+ }
+ } finally {
+ TestUtil.dropTable(con, "copytest_timeout");
+ }
+ }
+
+ /**
+ * Fails when a read timeout inside a backend message is reported as something the caller can
+ * carry on from. The bytes the driver already took are not in the stream any more, so the next
+ * read would take the middle of that message for the start of the next one.
+ */
+ @Test
+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void readTimeoutInsideMessageEndsTheCopy() throws Exception {
+ TestUtil.createTable(con, "copytest_stall", "data text");
+ try {
+ try (Statement stmt = con.createStatement()) {
+ stmt.execute(
+ "INSERT INTO copytest_stall SELECT 'row' || g FROM generate_series(1, 20000) g");
+ }
+
+ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+ PGProperty.SOCKET_TIMEOUT.set(props, 1);
+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);
+ PGProperty.SSL_MODE.set(props, "disable");
+ PGProperty.GSS_ENC_MODE.set(props, "disable");
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+ CopyOut copyOut = manager.copyOut("COPY copytest_stall TO STDOUT");
+ assertNotNull(copyOut.readFromCopy(), "Expected data row from COPY TO STDOUT");
+
+ proxyServer.stallInsideNextMessage(TimeUnit.SECONDS.toMillis(30));
+
+ SQLException timeout = readUntilTimeout(copyOut);
+ assertInstanceOf(IOException.class, timeout.getCause(),
+ () -> "a timeout inside a message is a connection failure, got " + timeout);
+ assertFalse(timeout.getCause() instanceof SocketTimeoutException,
+ "a timeout inside a message must not be reported as one the caller can resume from");
+ assertTrue(proxyCon.isClosed(),
+ "a stream that lost its message boundary has to take the connection with it");
+ assertThrows(SQLException.class, copyOut::readFromCopy,
+ "the copy must not look like something to carry on reading from");
+ }
+ }
+ } finally {
+ TestUtil.dropTable(con, "copytest_stall");
+ }
+ }
+
+
+ /**
+ * A COPY that dies with the connection keeps the COPY lock, so a thread already waiting for that
+ * lock has to be told the connection is gone rather than wait for a copy that can no longer end.
+ *
+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/3957">Issue 3957</a>
+ */
+ @Test
+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void waitingThreadGivesUpWhenCopyDiesWithConnection() throws Exception {
+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ TestUtil.createTempTable(proxyCon, "copytest_wait", "data text");
+
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+ CopyIn copyIn = manager.copyIn("COPY copytest_wait (data) FROM STDIN WITH (FORMAT CSV)");
+ byte[] row = "somedata\n".getBytes(StandardCharsets.UTF_8);
+ copyIn.writeToCopy(row, 0, row.length);
+ copyIn.flushCopy();
+
+ ExecutorService executor = Executors.newSingleThreadExecutor();
+ try {
+ AtomicReference<Thread> waiter = new AtomicReference<>();
+ Future<Boolean> valid = executor.submit(() -> {
+ waiter.set(Thread.currentThread());
+ return proxyCon.isValid(20);
+ });
+ awaitWaitingForCopyLock(waiter);
+
+ // Break the connection under the COPY while the other thread waits for the lock
+ proxyServer.closeAllClients();
+ assertThrows(SQLException.class, () -> {
+ for (int i = 0; i < 100_000; i++) {
+ copyIn.writeToCopy(row, 0, row.length);
+ copyIn.flushCopy();
+ }
+ }, "Expected SQLException from broken connection during writeToCopy");
+
+ assertFalse(valid.get(10, TimeUnit.SECONDS),
+ "The waiting thread should report the connection as invalid instead of waiting for "
+ + "a COPY that can no longer end");
+ } finally {
+ executor.shutdownNow();
+ }
+ }
+ }
+ }
+
+ /**
+ * The query that starts a COPY is already on the wire when the connection breaks, so its response
+ * can neither be read nor asked for again. The driver has to give the connection up rather than
+ * hand back a live one whose protocol stream is out of step.
+ */
+ @Test
+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void startCopyFailureClosesConnection() throws Exception {
+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ TestUtil.createTempTable(proxyCon, "copytest_start", "data text");
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+
+ proxyServer.closeAllClients();
+
+ assertThrows(SQLException.class,
+ () -> manager.copyIn("COPY copytest_start (data) FROM STDIN WITH (FORMAT CSV)"),
+ "Expected SQLException from broken connection while starting COPY");
+ assertTrue(proxyCon.isClosed(),
+ "A COPY that cannot read its own start response leaves the protocol stream out of "
+ + "step, so the connection should be given up");
+ }
+ }
+ }
+
+ /**
+ * Canceling a COPY over a broken connection cannot reach the server, so the driver has to give
+ * the connection up. The COPY itself is over either way.
+ */
+ @Test
+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void cancelCopyFailureClosesConnection() throws Exception {
+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ TestUtil.createTempTable(proxyCon, "copytest_cancel", "data text");
+
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+ CopyIn copyIn = manager.copyIn("COPY copytest_cancel (data) FROM STDIN WITH (FORMAT CSV)");
+ byte[] row = "somedata\n".getBytes(StandardCharsets.UTF_8);
+ copyIn.writeToCopy(row, 0, row.length);
+ copyIn.flushCopy();
+
+ proxyServer.closeAllClients();
+
+ assertThrows(SQLException.class, copyIn::cancelCopy,
+ "Expected SQLException from broken connection while canceling COPY");
+ assertFalse(copyIn.isActive(), "cancelCopy() should end the COPY operation");
+ assertTrue(proxyCon.isClosed(),
+ "A cancel that never reached the server should give the connection up");
+ }
+ }
+ }
+
+ /**
+ * The {@link CopyIn#writeToCopy(org.postgresql.util.ByteStreamWriter)} overload has to give the
+ * connection up on an I/O failure just like the byte-array one.
+ *
+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/3957">Issue 3957</a>
+ */
+ @Test
+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void writeToCopyFromByteStreamWriterFailureClosesConnection() throws Exception {
+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ TestUtil.createTempTable(proxyCon, "copytest_writer", "data text");
+
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+ CopyIn copyIn = manager.copyIn("COPY copytest_writer (data) FROM STDIN WITH (FORMAT CSV)");
+ byte[] row = "somedata\n".getBytes(StandardCharsets.UTF_8);
+ copyIn.writeToCopy(new ByteBufferByteStreamWriter(ByteBuffer.wrap(row)));
+ copyIn.flushCopy();
+
+ proxyServer.closeAllClients();
+
+ // A payload this large outgrows the output buffer, so it has to reach the socket within
+ // writeToCopy itself rather than waiting for the flush
+ byte[] bigRow = new byte[1024 * 1024];
+ Arrays.fill(bigRow, (byte) 'x');
+ bigRow[bigRow.length - 1] = '\n';
+ assertThrows(SQLException.class,
+ () -> copyIn.writeToCopy(new ByteBufferByteStreamWriter(ByteBuffer.wrap(bigRow))),
+ "Expected SQLException from broken connection during writeToCopy");
+
+ assertTrue(proxyCon.isClosed(),
+ "A COPY that failed to send should give the connection up");
+ }
+ }
+ }
+
+ /**
+ * Interrupting a thread that waits for a COPY to finish fails its query and leaves the interrupt
+ * flag set, rather than swallowing the interrupt.
+ */
+ @Test
+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void interruptingAThreadWaitingForTheCopyKeepsTheInterruptFlag() throws Exception {
+ CopyIn copyIn = copyAPI.copyIn("COPY copytest FROM STDIN");
+ byte[] row = origData[0].getBytes(StandardCharsets.UTF_8);
+ copyIn.writeToCopy(row, 0, row.length);
+ copyIn.flushCopy();
+
+ ExecutorService executor = Executors.newSingleThreadExecutor();
+ try {
+ AtomicReference<Thread> waiter = new AtomicReference<>();
+ AtomicBoolean stillInterrupted = new AtomicBoolean();
+ Future<SQLException> failure = executor.submit(() -> {
+ waiter.set(Thread.currentThread());
+ try (Statement stmt = con.createStatement()) {
+ stmt.execute("SELECT 1");
+ return null;
+ } catch (SQLException e) {
+ stillInterrupted.set(Thread.currentThread().isInterrupted());
+ return e;
+ }
+ });
+ awaitWaitingForCopyLock(waiter).interrupt();
+
+ SQLException e = failure.get(10, TimeUnit.SECONDS);
+ assertNotNull(e, "The interrupted query should fail instead of waiting for the COPY");
+ assertEquals(PSQLState.OBJECT_NOT_IN_STATE.getState(), e.getSQLState(),
+ () -> "Unexpected failure: " + e);
+ assertTrue(stillInterrupted.get(), "The interrupt flag should survive the failure");
+ } finally {
+ executor.shutdownNow();
+ copyIn.endCopy();
+ }
+ }
+
+ /**
+ * Every thread waiting for a COPY to finish has to be woken when it does, not just one of them.
+ */
+ @Test
+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void allWaitingThreadsResumeWhenCopyEnds() throws Exception {
+ CopyIn copyIn = copyAPI.copyIn("COPY copytest FROM STDIN");
+ byte[] row = origData[0].getBytes(StandardCharsets.UTF_8);
+ copyIn.writeToCopy(row, 0, row.length);
+ copyIn.flushCopy();
+
+ ExecutorService executor = Executors.newFixedThreadPool(3);
+ try {
+ List<Future<Boolean>> waiters = new ArrayList<>();
+ List<AtomicReference<Thread>> threads = new ArrayList<>();
+ for (int i = 0; i < 3; i++) {
+ AtomicReference<Thread> thread = new AtomicReference<>();
+ threads.add(thread);
+ waiters.add(executor.submit(() -> {
+ thread.set(Thread.currentThread());
+ return con.isValid(20);
+ }));
+ }
+ for (AtomicReference<Thread> thread : threads) {
+ awaitWaitingForCopyLock(thread);
+ }
+
+ copyIn.endCopy();
+
+ for (Future<Boolean> waiter : waiters) {
+ assertTrue(waiter.get(10, TimeUnit.SECONDS),
+ "Every thread waiting for the COPY should resume once it ends");
+ }
+ } finally {
+ executor.shutdownNow();
+ }
+ }
+
+ /**
+ * Waits until the helper thread parks on the COPY lock. Fails the test if it never does.
+ */
+ private static Thread awaitWaitingForCopyLock(AtomicReference<Thread> waiter) {
+ long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(10);
+ while (System.nanoTime() < deadline) {
+ Thread thread = waiter.get();
+ if (thread != null && thread.getState() == Thread.State.WAITING) {
+ return thread;
+ }
+ Thread.yield();
+ }
+ return fail("The helper thread never started waiting for the COPY lock");
+ }
+
+ /**
+ * Fails when a read timeout throws away an error the backend already sent. The copy loop keeps
+ * the error until the ReadyForQuery that follows it, so a timeout waiting for that message must
+ * not be reported as "nothing arrived yet" — a caller that reads again would then see the copy
+ * finish successfully.
+ */
+ @Test
+ @Timeout(value = 60, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void readTimeoutAfterAnErrorResponseStillReportsTheError() throws Exception {
+ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+ PGProperty.SOCKET_TIMEOUT.set(props, 1);
+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);
+ PGProperty.SSL_MODE.set(props, "disable");
+ PGProperty.GSS_ENC_MODE.set(props, "disable");
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+ // The last row divides by zero, so the server streams data and then reports an error
+ CopyOut copyOut = manager.copyOut(
+ "COPY (SELECT 1 / (20000 - g) FROM generate_series(1, 20000) g) TO STDOUT");
+
+ // Hold back the ReadyForQuery that closes the exchange, so the read that waits for it
+ // times out with the error already in hand
+ proxyServer.stallBeforeMessageType('Z', TimeUnit.SECONDS.toMillis(20));
+
+ SQLException thrown = readUntilTimeout(copyOut);
+ assertEquals(PSQLState.DIVISION_BY_ZERO.getState(), thrown.getSQLState(),
+ () -> "the backend error has to survive the timeout that followed it, got " + thrown);
+ }
+ }
+ }
+
+ /**
+ * Fails when endCopy keeps the COPY lock after the exchange stopped somewhere other than
+ * ReadyForQuery. A retained lock with a live connection is a lock nothing can take back:
+ * waitOnLock() gives up only once the connection is closed, so every later operation on it
+ * waits forever.
+ */
+ @Test
+ @Timeout(value = 60, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)
+ void endCopyGivesTheConnectionUpWhenItCannotReleaseTheLock() throws Exception {
+ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),
+ TestUtil.getPort())) {
+ Properties props = new Properties();
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
+ String.valueOf(proxyServer.getServerPort()));
+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);
+ PGProperty.SSL_MODE.set(props, "disable");
+ PGProperty.GSS_ENC_MODE.set(props, "disable");
+
+ try (Connection proxyCon = TestUtil.openDB(props)) {
+ TestUtil.createTempTable(proxyCon, "copytest_endcopy", "data text");
+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();
+ CopyIn copyIn = manager.copyIn("COPY copytest_endcopy FROM STDIN");
+ byte[] row = "one\n".getBytes(StandardCharsets.UTF_8);
+ copyIn.writeToCopy(row, 0, row.length);
+
+ // CopyData is the server's answer to a COPY TO, never to a COPY FROM. The copy loop stops
+ // on it without reaching the ReadyForQuery that would hand the lock back
+ proxyServer.injectBeforeNextMessage(copyDataMessage("stray"));
+
+ assertThrows(SQLException.class, copyIn::endCopy,
+ "the driver should report the message it cannot make sense of");
+ assertTrue(proxyCon.isClosed(),
+ "a copy that cannot give its lock back has to take the connection with it");
+ }
+ }
+ }
+
+ /** Builds a CopyData message: the type byte, a self-inclusive length, then the payload. */
+ private static byte[] copyDataMessage(String payload) {
+ byte[] body = payload.getBytes(StandardCharsets.UTF_8);
+ ByteBuffer message = ByteBuffer.allocate(1 + 4 + body.length);
+ message.put((byte) 'd').putInt(4 + body.length).put(body);
+ return message.array();
+ }
+
+ /**
+ * Reads until {@code readFromCopy} fails, and returns that failure.
+ */
+ private static SQLException readUntilTimeout(CopyOut copyOut) throws SQLException {
+ long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(30);
+ while (System.nanoTime() < deadline) {
+ try {
+ if (copyOut.readFromCopy() == null) {
+ return fail("COPY completed before the reader ran out of buffered data");
+ }
+ } catch (SQLException e) {
+ return e;
+ }
+ }
+ return fail("readFromCopy kept returning data even though the proxy stopped forwarding");
+ }
}
diff --git a/pgjdbc/src/test/java/org/postgresql/test/util/MessageStallProxyServer.java b/pgjdbc/src/test/java/org/postgresql/test/util/MessageStallProxyServer.java
new file mode 100644
index 0000000000..8463276e31
--- /dev/null
+++ b/pgjdbc/src/test/java/org/postgresql/test/util/MessageStallProxyServer.java
@@ -0,0 +1,248 @@
+/*
+ * Copyright (c) 2026, PostgreSQL Global Development Group
+ * See the LICENSE file in the project root for more information.
+ */
+
+package org.postgresql.test.util;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.ServerSocket;
+import java.net.Socket;
+import java.net.SocketTimeoutException;
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+
+/**
+ * Proxy server that can hold a backend message open, so that the driver sits partway through one
+ * for as long as the test wants.
+ *
+ * <p>It forwards both directions untouched until one of the two stalls arms it.
+ * {@link #stallInsideNextMessage(long)} delivers the next message in two parts, the type byte on
+ * its own and then, after a pause, the length and the body: the driver has consumed the type byte
+ * and is waiting for the rest, which is where a read deadline of its own does damage, since the
+ * bytes it already took are not in the stream any more.
+ * {@link #stallBeforeNextMessage(long)} pauses ahead of the type byte instead, leaving the driver
+ * waiting with nothing consumed — the same wait, at a point it can still come back from.
+ *
+ * <p>The proxy reads the v3 framing to find that point, so the connection through it has to be
+ * plain: pass {@code sslmode=disable} and {@code gssEncMode=disable}, or the traffic is encrypted
+ * and the framing is not there to be found.
+ */
+public class MessageStallProxyServer implements Closeable {
+ private final ServerSocket serverSock;
+ private final List<Socket> sockets = new CopyOnWriteArrayList<>();
+ private final AtomicLong stallMillis = new AtomicLong();
+ private final AtomicLong boundaryStallMillis = new AtomicLong();
+ private final AtomicLong typeStallMillis = new AtomicLong();
+ private final AtomicInteger stallBeforeType = new AtomicInteger();
+ private final AtomicReference<byte @Nullable []> injection = new AtomicReference<>();
+ private volatile boolean keepRunning = true;
+
+ public MessageStallProxyServer(String destHost, int destPort) throws IOException {
+ this.serverSock = new ServerSocket(0);
+ this.serverSock.setSoTimeout(100);
+ doAsync(() -> {
+ while (keepRunning) {
+ try {
+ Socket client = serverSock.accept();
+ sockets.add(client);
+ Socket server = new Socket(destHost, destPort);
+ sockets.add(server);
+ doAsync(() -> forward(client, server));
+ doAsync(() -> forwardBackendMessages(server, client));
+ } catch (SocketTimeoutException ignore) {
+ // the accept timeout is what lets this loop notice close()
+ } catch (IOException e) {
+ return;
+ }
+ }
+ });
+ }
+
+ public int getServerPort() {
+ return serverSock.getLocalPort();
+ }
+
+ /**
+ * Splits the next backend message, delivering its type byte and then pausing before the rest.
+ * Applies to one message; call it again to arm the proxy for another.
+ *
+ * @param millis how long to hold the message open
+ */
+ public void stallInsideNextMessage(long millis) {
+ stallMillis.set(millis);
+ }
+
+ /**
+ * Pauses before the next backend message instead of inside it, so the driver waits with nothing
+ * consumed. Forwarding whole messages is what makes this a boundary: the driver's buffer can only
+ * end where a message does.
+ *
+ * @param millis how long to wait before starting the next message
+ */
+ public void stallBeforeNextMessage(long millis) {
+ boundaryStallMillis.set(millis);
+ }
+
+ /**
+ * Pauses before the next backend message of the given type, leaving the driver waiting for that
+ * one with everything ahead of it already delivered. Use it to hold back the message that ends
+ * an exchange, such as the ReadyForQuery that follows an ErrorResponse.
+ *
+ * @param messageType the v3 type byte to hold back, for example {@code 'Z'}
+ * @param millis how long to wait before sending it
+ */
+ public void stallBeforeMessageType(int messageType, long millis) {
+ stallBeforeType.set(messageType);
+ typeStallMillis.set(millis);
+ }
+
+ /**
+ * Sends the bytes to the driver ahead of the next real backend message, so a test can put a
+ * message on the wire that the server would not send. The driver reads it as if the backend had.
+ *
+ * @param raw a complete v3 message: the type byte, a self-inclusive four-byte length, the body
+ */
+ public void injectBeforeNextMessage(byte[] raw) {
+ injection.set(raw.clone());
+ }
+
+ @Override
+ public void close() {
+ keepRunning = false;
+ for (Socket socket : sockets) {
+ try {
+ socket.close();
+ } catch (IOException ignore) {
+ // closing the rest matters more than this one
+ }
+ }
+ try {
+ serverSock.close();
+ } catch (IOException ignore) {
+ // nothing left to do about it
+ }
+ }
+
+ /** Copies one direction byte for byte, with no interest in what the bytes mean. */
+ private void forward(Socket source, Socket dest) {
+ try {
+ InputStream in = source.getInputStream();
+ OutputStream out = dest.getOutputStream();
+ byte[] buffer = new byte[8192];
+ int read;
+ while (keepRunning && (read = in.read(buffer)) >= 0) {
+ out.write(buffer, 0, read);
+ out.flush();
+ }
+ } catch (IOException ignore) {
+ // the test closes the sockets under us
+ }
+ }
+
+ /**
+ * Copies the backend's direction one message at a time, so that an armed stall lands after the
+ * type byte and before the length.
+ */
+ private void forwardBackendMessages(Socket source, Socket dest) {
+ try {
+ InputStream in = source.getInputStream();
+ OutputStream out = dest.getOutputStream();
+ byte[] header = new byte[4];
+ while (keepRunning) {
+ byte[] inject = injection.getAndSet(null);
+ if (inject != null) {
+ out.write(inject);
+ out.flush();
+ }
+ long boundaryStall = boundaryStallMillis.getAndSet(0);
+ if (boundaryStall > 0) {
+ Thread.sleep(boundaryStall);
+ }
+ int type = in.read();
+ if (type < 0) {
+ return;
+ }
+ if (type == stallBeforeType.get()) {
+ long typeStall = typeStallMillis.getAndSet(0);
+ if (typeStall > 0) {
+ Thread.sleep(typeStall);
+ }
+ }
+ out.write(type);
+ out.flush();
+
+ long stall = stallMillis.getAndSet(0);
+ if (stall > 0) {
+ Thread.sleep(stall);
+ }
+
+ if (!readFully(in, header, header.length)) {
+ return;
+ }
+ out.write(header);
+ // The length counts itself, so the body is what is left after the four bytes
+ int length = (header[0] & 0xFF) << 24 | (header[1] & 0xFF) << 16
+ | (header[2] & 0xFF) << 8 | header[3] & 0xFF;
+ if (!copy(in, out, length - 4)) {
+ return;
+ }
+ out.flush();
+ }
+ } catch (IOException ignore) {
+ // the test closes the sockets under us
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ }
+ }
+
+ private static boolean readFully(InputStream in, byte[] target, int length) throws IOException {
+ int done = 0;
+ while (done < length) {
+ int read = in.read(target, done, length - done);
+ if (read < 0) {
+ return false;
+ }
+ done += read;
+ }
+ return true;
+ }
+
+ private static boolean copy(InputStream in, OutputStream out, int length) throws IOException {
+ byte[] buffer = new byte[Math.min(8192, Math.max(length, 1))];
+ int left = length;
+ while (left > 0) {
+ int read = in.read(buffer, 0, Math.min(buffer.length, left));
+ if (read < 0) {
+ return false;
+ }
+ out.write(buffer, 0, read);
+ left -= read;
+ }
+ return true;
+ }
+
+ private static void doAsync(RunnableWithException task) {
+ Thread thread = new Thread(() -> {
+ try {
+ task.run();
+ } catch (Exception ignore) {
+ // a proxy thread that dies takes its connection down, which the test sees
+ }
+ });
+ thread.setDaemon(true);
+ thread.start();
+ }
+
+ private interface RunnableWithException {
+ void run() throws Exception;
+ }
+}
diff --git a/testkit/src/main/java/org/postgresql/test/TestUtil.java b/testkit/src/main/java/org/postgresql/test/TestUtil.java
index 4e35b95187..4351c4ebe9 100644
--- a/testkit/src/main/java/org/postgresql/test/TestUtil.java
+++ b/testkit/src/main/java/org/postgresql/test/TestUtil.java
@@ -328,6 +328,11 @@ public static Connection openPrivilegedDB(Consumer<? super Properties> props) th
}
public static Connection openReplicationConnection() throws Exception {
+ return openReplicationConnection(x -> { });
+ }
+
+ public static Connection openReplicationConnection(Consumer<? super Properties> props)
+ throws Exception {
Properties properties = new Properties();
PGProperty.ASSUME_MIN_SERVER_VERSION.set(properties, "9.4");
PGProperty.PROTOCOL_VERSION.set(properties, "3");
@@ -337,6 +342,7 @@ public static Connection openReplicationConnection() throws Exception {
PGProperty.USER.set(properties, TestUtil.getPrivilegedUser());
PGProperty.PASSWORD.set(properties, TestUtil.getPrivilegedPassword());
PGProperty.OPTIONS.set(properties, "-c synchronous_commit=on");
+ props.accept(properties);
return TestUtil.openDB(properties);
}
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-07-30 13:15 "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: codecov[bot] (bot) (@codecov[bot]) @ 2026-07-30 13:15 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
## [Codecov](https://app.codecov.io/gh/pgjdbc/pgjdbc/pull/4331?dropdown=coverage&src=pr&el=h1&utm_med...) Report
:x: Patch coverage is `71.09375%` with `37 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 72.05%. Comparing base ([`3ba23fc`](https://app.codecov.io/gh/pgjdbc/pgjdbc/commit/3ba23fcff8c110fcad7723ce7a20589676debd48?dropdown=cov...)) to head ([`246e29f`](https://app.codecov.io/gh/pgjdbc/pgjdbc/commit/246e29f6a7ea6512abfdc232a24d8738ef797d31?dropdown=cov...)).
:warning: Report is 87 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #4331 +/- ##
============================================
+ Coverage 71.85% 72.05% +0.19%
- Complexity 5400 5411 +11
============================================
Files 244 244
Lines 22040 22062 +22
Branches 3714 3713 -1
============================================
+ Hits 15837 15896 +59
+ Misses 4515 4487 -28
+ Partials 1688 1679 -9
```
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-07-30 13:21 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-07-30 13:21 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
@vlsi applied label "bug".
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-08-20 18:49 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-08-20 18:49 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
This has been open for reviews for three weeks already. The bug is real, the fix is there, the test are passing.
I'm about to merge this.
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-08-21 13:09 "sehrope (@sehrope)" <noreply+sehrope@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: sehrope (@sehrope) @ 2026-08-21 13:09 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
A few of us have been away over the summer and this isn't the only PR that's gone unlooked at. Nothing personal in the delay.
That said, this is not a trivial change and three weeks of no comments isn't the same as approval. This touches the COPY lock and the connection-failure path in QueryExecutorImpl, which is exactly the area where the last fix caused this regression, so it needs someone to actually read it before it goes in. I'd rather it wait a bit longer than merge unreviewed.
I'll try to get to it this week. If someone else can take a look sooner, please do.
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-08-21 18:38 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-08-21 18:38 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
#4354 is stacked on this branch and has to merge second. Its last commit makes the replication read timeout surface on connections opened without `socketTimeout`, and before this PR a surfaced timeout ended the COPY — the status update that follows it would then have failed with `Tried to write to an inactive copy operation` (#4328) for every default-configured consumer.
`TestUtil.openReplicationConnection(Consumer<Properties>)` now lives only here. #4354 had added it identically and dropped its copy when it was rebased onto this branch.
Both branches are rebased onto master as of 3ba23fcff.
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-08-24 12:36 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-08-24 12:36 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
@vlsi applied label "merge-after-review-window".
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-08-24 12:37 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-08-24 12:37 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
I've reviewed this and I'm happy with the current implementation.
Since this touches fairly subtle COPY/error-handling semantics, I'd like to leave it open for other maintainers to review. Unless there are concerns, I plan to merge it after September 7.
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-09-10 04:55 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-09-10 04:55 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
@vlsi removed label "merge-after-review-window".
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-09-10 04:56 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-09-10 04:56 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
@vlsi applied label "waiting-for-changes".
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-09-15 16:17 "vlsi (@vlsi)" <noreply+vlsi@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: vlsi (@vlsi) @ 2026-09-15 16:17 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
@vlsi removed label "waiting-for-changes".
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-11-05 11:21 "davecramer (@davecramer)" <noreply+davecramer@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: davecramer (@davecramer) @ 2026-11-05 11:21 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
I read the change and ran it against PostgreSQL here. The approach is right, and it fixes #4328: `idleStreamSurvivesSocketReadTimeout` fails with "Tried to write to an inactive copy operation" when I revert the `QueryExecutorImpl` change, and passes with it. `LogicalReplicationTest`, `LogicalReplicationStatusTest`, `PhysicalReplicationTest`, `CopyBothResponseTest`, `ReplicationSlotTest`, `ReplicationConnectionTest` and `CopyTest` all pass on `246e29f`.
I found one regression, and I would like it fixed before this goes in.
### 1. A COPY on a closed connection still reports itself active
`abortAfterCopyFailure()` keeps `lockedFor` set, and `CopyOperationImpl.isActive()` is `hasLockOn(this)`. So once the COPY has closed the connection, `isActive()` returns true. In 42.7.11 to 42.7.13 it returned false, because the lock was released. Three callers decide from it:
- `CopyManager.copyIn(sql, InputStream)` calls `cancelCopy()` in its `finally` block. That writes CopyFail to the closed socket, and the exception it throws replaces the write failure the caller should see.
- `CopyManager.copyOut` calls `cancelCopy()`, which for a CopyOut opens a new connection to send a cancel request.
- `PGReplicationStream.isClosed()` is `closeFlag || !copyDual.isActive()`, so it returns false on a closed connection.
I checked it with a test that cuts the connection through `StrangeProxyServer` and then writes a 1 MB row:
| | `con.isClosed()` | `copyIn.isActive()` | `CopyManager.copyIn` throws |
| --- | --- | --- | --- |
| master | false | false | "Database connection failed when writing to copy" |
| this PR | true | true | "Database connection failed when canceling copy operation" |
| this PR with the change below | true | false | "Database connection failed when writing to copy" |
The fix I tried is one line in `hasLockOn` (GitHub will not take a suggestion there, because the line is outside this diff):
```java
boolean hasLockOn(@Nullable Object holder) {
try (ResourceLock ignore = lock.obtain()) {
return lockedFor == holder && !isClosed();
}
}
```
`cancelCopy` and the other internal paths use the private `hasLock`, so they are unaffected. With it, every test listed above still passes.
Here is the test. It checks where the exception was created and not its message, because the message is translated and CI runs with `fr_FR` and `tr_TR`. Both tests fail on this PR and pass with the fix. They could go into `CopyTest`.
<details><summary>CopyAfterAbortReproTest.java</summary>
```java
/*
* Copyright (c) 2026, PostgreSQL Global Development Group
* See the LICENSE file in the project root for more information.
*/
package org.postgresql.test.jdbc2;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import org.postgresql.PGConnection;
import org.postgresql.PGProperty;
import org.postgresql.copy.CopyIn;
import org.postgresql.copy.CopyManager;
import org.postgresql.test.TestUtil;
import org.postgresql.test.util.StrangeProxyServer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Timeout;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.Arrays;
import java.util.Properties;
import java.util.concurrent.TimeUnit;
class CopyAfterAbortReproTest {
private static Properties viaProxy(StrangeProxyServer proxy) {
Properties props = new Properties();
TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");
TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,
String.valueOf(proxy.getServerPort()));
return props;
}
/** A row larger than the output buffer, so writeToCopy reaches the socket. */
private static byte[] bigRow() {
byte[] row = new byte[1024 * 1024];
Arrays.fill(row, (byte) 'x');
row[row.length - 1] = '\n';
return row;
}
/**
* A COPY whose connection has been closed can never end, so isActive()
* has to report false. Callers such as CopyManager.copyIn decide from it
* whether to cancel the COPY.
*/
@Test
@Timeout(value = 30, unit = TimeUnit.SECONDS)
void copyIsInactiveOnceItsConnectionIsClosed() throws Exception {
try (StrangeProxyServer proxy = new StrangeProxyServer(TestUtil.getServer(),
TestUtil.getPort());
Connection con = TestUtil.openDB(viaProxy(proxy))) {
TestUtil.createTempTable(con, "copyrepro_a", "data text");
CopyManager manager = con.unwrap(PGConnection.class).getCopyAPI();
CopyIn copyIn = manager.copyIn("COPY copyrepro_a (data) FROM STDIN");
proxy.closeAllClients();
byte[] row = bigRow();
assertThrows(SQLException.class, () -> copyIn.writeToCopy(row, 0, row.length));
assertTrue(con.isClosed(), "The failed write should close the connection");
assertFalse(copyIn.isActive(),
"A COPY on a closed connection should not report itself active");
}
}
/**
* When the connection drops during CopyManager.copyIn, the caller has to
* see the write that failed. A cancel attempted afterwards on the closed
* connection must not replace it.
*/
@Test
@Timeout(value = 30, unit = TimeUnit.SECONDS)
void copyInReportsTheWriteThatFailed() throws Exception {
try (StrangeProxyServer proxy = new StrangeProxyServer(TestUtil.getServer(),
TestUtil.getPort());
Connection con = TestUtil.openDB(viaProxy(proxy))) {
TestUtil.createTempTable(con, "copyrepro_b", "data text");
CopyManager manager = con.unwrap(PGConnection.class).getCopyAPI();
byte[] first = "a\n".getBytes(StandardCharsets.UTF_8);
byte[] big = bigRow();
// One small row, then the connection drops and a row too large for
// the output buffer follows
InputStream source = new InputStream() {
private int calls;
@Override
public int read() {
throw new UnsupportedOperationException();
}
@Override
public int read(byte[] b, int off, int len) {
calls++;
if (calls == 1) {
System.arraycopy(first, 0, b, off, first.length);
return first.length;
}
if (calls == 2) {
proxy.closeAllClients();
int n = Math.min(len, big.length);
System.arraycopy(big, 0, b, off, n);
return n;
}
return -1;
}
};
SQLException e = assertThrows(SQLException.class,
() -> manager.copyIn("COPY copyrepro_b (data) FROM STDIN", source,
1024 * 1024));
// The stack shows where the exception was created; the message text
// is translated, so it cannot be compared under every locale
assertTrue(createdIn(e, "writeToCopy"),
"copyIn should report the failed write, got: " + e);
assertFalse(createdIn(e, "cancelCopy"),
"copyIn should not report the cancel that followed, got: " + e);
}
}
private static boolean createdIn(Throwable t, String method) {
return Arrays.stream(t.getStackTrace())
.anyMatch(f -> f.getClassName().endsWith(".QueryExecutorImpl")
&& f.getMethodName().equals(method));
}
}
```
</details>
### 2. The rebase, and a smaller diff
`QueryExecutorImpl.java` conflicts with master, which has changed `processCopyResults` since this branch was cut (`receiveMessageType`, `receiveMessageLength`, `protocolViolation`). Most of the 393 changed lines in that file are indentation: `git diff -w` shows 82 added and 25 removed. The indentation comes from the new `try` around the whole `switch`.
The logic could be turned around: catch `SocketTimeoutException` only around the reads that happen at a message boundary (`hasMessagePending()`, `peekChar()` and `receiveMessageType()`), and treat every other timeout as one inside a message. The `switch` would stay as it is, the conflict would be small, and a `case` added later would be treated as inside a message without anyone having to remember it. Master's `messageEnd` check does not replace your check, because `messageEnd` is set only after the length is read.
### 3. The recoverable timeout looks like a failed connection
The timeout that keeps the COPY active is reported as "Database connection failed when reading from copy" with `08006`, the same as a real failure. Only `getCause()` tells them apart. That was already true before this PR, but the two now have different outcomes, so I think the recoverable one should have its own message.
### 4. The test timeout default
`8c95e8c31` makes `SEPARATE_THREAD` the default for the whole build. Each new test here already sets `threadMode = SEPARATE_THREAD`, so the fix does not need the global change. I would rather see it in its own PR.
There is one more inline comment, on the wording at `unlock()`.
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-11-05 11:21 "davecramer (@davecramer)" <noreply+davecramer@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: davecramer (@davecramer) @ 2026-11-05 11:21 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
(on pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java:301)
There are two locks here: threads in `waitOnLock()` do hold `lock`, because `Condition.await` requires it. What they do not take is the COPY lock, `lockedFor`. Naming it would avoid the second reading, for example "Threads that return from waitOnLock() do not take the COPY lock, so ...".
^ permalink raw [nested|flat] 14+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
@ 2026-11-05 11:25 "davecramer (@davecramer)" <noreply+davecramer@github.com>
12 siblings, 0 replies; 14+ messages in thread
From: davecramer (@davecramer) @ 2026-11-05 11:25 UTC (permalink / raw)
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
I found the description hard to follow, so here is a rewrite you could use. It is about half the length and lists all eleven new tests. The current text says "six new tests" and its table has seven rows, but the PR adds ten tests to `CopyTest` and one to `LogicalReplicationTest`.
I left out "each checked to fail without its part of the change", because I could only confirm that for `idleStreamSurvivesSocketReadTimeout`. I also left out the paragraph about your own test run. If you take the `hasLockOn` change from my review, the "What I changed" section needs a line saying a COPY on a closed connection reports itself inactive, and the test list needs those two tests.
````markdown
## Why
Since 42.7.11, logical replication with the blocking `PGReplicationStream.read()` fails as soon as the slot has been idle for one status interval. The error is `Tried to write to an inactive copy operation` (#4328).
Here is what happens. The replication stream sets the socket timeout to the status interval on purpose. When the read times out, the stream sends a standby status update and reads again. In 42.7.11, b5080e76f (the fix for #3957) made `readFromCopy` release the COPY lock on any `IOException`. A read timeout is an `IOException`, so the timeout ended the COPY, and the status update after it failed.
You only see this with `socketTimeout` set. Without it, the driver catches the timeout itself and reads again.
The same change breaks `COPY TO STDOUT` with `socketTimeout`. After a timeout the COPY is no longer active, so `cancelCopy()` throws `Tried to cancel an inactive copy operation`, and the server is left in COPY mode.
## What I changed
I keep the COPY active when a read times out between two messages. That is the one failure where reading again is safe, and replication depends on it. `processNotifies` already handles a read timeout the same way.
I close the connection when a read times out inside a message. By then the type byte has been read, so reading again would treat the rest of that message as a new one.
For every other I/O failure during a COPY, I close the connection and keep the lock. Releasing the lock said the connection was free to use when it was out of sync. Closing it is what the rest of `QueryExecutorImpl` does after an I/O failure. `waitOnLock()` now gives up once the connection is closed, so the hang from #3957 stays fixed.
`startCopy` and `cancelCopy` also close the connection on an I/O failure, because neither can be retried. Retrying `startCopy` would send the query twice.
`unlock()` now wakes every waiting thread. With `signal()`, one thread woke and the others kept waiting for a COPY that had already ended.
In a separate commit, I made `SEPARATE_THREAD` the default test timeout mode. A test stuck in a socket read ignores interrupts, so without this a stuck test hangs the build. That commit can be dropped on its own.
## Tests
I added eleven tests:
- **A read timeout between messages:** `CopyTest.readTimeoutKeepsCopyActive` checks that the COPY stays active and `cancelCopy()` still works. `LogicalReplicationTest.idleStreamSurvivesSocketReadTimeout` checks that a replication stream keeps working after being idle for several status intervals.
- **A read timeout inside a message:** `readTimeoutInsideMessageEndsTheCopy` checks that this closes the connection. `readTimeoutAfterAnErrorResponseStillReportsTheError` checks that an error the server sent before the timeout still reaches the caller.
- **Failures that close the connection:** `startCopyFailureClosesConnection`, `cancelCopyFailureClosesConnection`, `writeToCopyFromByteStreamWriterFailureClosesConnection` and `endCopyGivesTheConnectionUpWhenItCannotReleaseTheLock`.
- **Threads waiting for the COPY lock:** `waitingThreadGivesUpWhenCopyDiesWithConnection`, `allWaitingThreadsResumeWhenCopyEnds` and `interruptingAThreadWaitingForTheCopyKeepsTheInterruptFlag`.
`TestUtil.openReplicationConnection` has a new overload that takes a `Consumer<Properties>`, so the replication test can set `socketTimeout` itself.
## What I left out
`endCopy` still closes the connection on a read timeout. Reading again there would be safe on the wire, but three callers expect the current behavior:
- `CopyManager.copyIn` would send a cancel that fails and hides the timeout.
- `PGCopyOutputStream.close()` would leave a pooled connection holding the lock.
- `PGCopyOutputStream.endCopy()` would send up to 64 KB of data twice.
Fixing those changes how the COPY API behaves, so it belongs in its own PR.
````
^ permalink raw [nested|flat] 14+ messages in thread
end of thread, other threads:[~2026-11-05 11:25 UTC | newest]
Thread overview: 14+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-07-30 13:05 [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-07-30 13:15 ` "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
2026-07-30 13:21 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-08-20 18:49 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-08-21 13:09 ` "sehrope (@sehrope)" <noreply+sehrope@github.com>
2026-08-21 18:38 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-08-24 12:36 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-08-24 12:37 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-09-10 04:55 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-09-10 04:56 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-09-15 16:17 ` "vlsi (@vlsi)" <noreply+vlsi@github.com>
2026-11-05 11:21 ` "davecramer (@davecramer)" <noreply+davecramer@github.com>
2026-11-05 11:21 ` "davecramer (@davecramer)" <noreply+davecramer@github.com>
2026-11-05 11:25 ` "davecramer (@davecramer)" <noreply+davecramer@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox