From: vlsi (@vlsi) <noreply+vlsi@github.com>
To: pgjdbc/pgjdbc <noreply+pgjdbc-pgjdbc@github.com>
Subject: [pgjdbc/pgjdbc] PR #4331: fix(copy): keep the COPY active when a read times out
Date: Thu, 30 Jul 2026 13:05:44 +0000
Message-ID: <gh-pgjdbc-pgjdbc-4331@github.com> (raw)
## Why
Since 42.7.11 a logical replication consumer that uses the blocking `PGReplicationStream.read()` API dies as soon as the slot is idle for one status interval, with `PSQLException: Tried to write to an inactive copy operation` ([#4328](https://github.com/pgjdbc/pgjdbc/issues/4328)).
`V3ReplicationProtocol.configureSocketTimeout` sets `SO_TIMEOUT` to `min(socketTimeout, statusInterval)` on purpose: the blocking read has to wake up periodically to send a standby status update. [b5080e76f](https://github.com/pgjdbc/pgjdbc/commit/b5080e76f) (the fix for [#3957](https://github.com/pgjdbc/pgjdbc/issues/3957), first released in 42.7.11) made `readFromCopy` release the COPY on every `IOException`, and `SocketTimeoutException` is one. So the routine idle timeout deactivated the COPY, and the status update that follows it in `V3PGReplicationStream.readInternal` failed. The reproducer needs a non-zero `socketTimeout`: without it `VisibleBufferedInputStream.readMore` swallows the timeout and retries, so the exception never surfaces.
The same regression hits a plain `COPY TO STDOUT` with `socketTimeout`: after a read timeout the operation is no longer active, so `cancelCopy()` throws `Tried to cancel an inactive copy operation` and the server is left in COPY mode while the driver believes the operation is over.
## What
**A read timeout keeps the COPY.** `readFromCopy` catches `SocketTimeoutException` separately and reports it without touching the lock. This is the only failure a caller can resume from, by reading again. The driver already treats a read timeout this way in `processNotifies`.
**The COPY lock is never released on an I/O failure.** Releasing it claimed the protocol stream was free while it was desynchronized, which is what took `cancelCopy()` away. Instead the driver gives the connection up with `abort()` — what every other I/O failure in `QueryExecutorImpl` already does — and `waitOnLock()` stops waiting once the connection is closed. That preserves the #3957 fix (operations hanging in `waitOnLock()` after a COPY died) without deciding the lock state from the exception type, which was the wrong question to ask: even a `SocketTimeoutException` is unrecoverable when it lands mid-message, and the set of "safe" exceptions is not closed.
**`unlock()` signals every waiter.** Threads wait in `waitOnLock()` without taking the lock, so `signal()` woke one of them and left the rest waiting for a copy that was already over.
**`startCopy` and `cancelCopy` give the connection up too.** Neither can be retried — a second `startCopy` would send the query twice — so a live connection whose response is still on the wire is worse than a closed one.
**A read timeout inside a message is not one to carry on from** (last commit, added after review). Keeping the COPY is sound at a message boundary and nowhere else: `VisibleBufferedInputStream.read()` hands out `buffer[index++]`, so past the type byte a caller that reads again takes the middle of the interrupted message for the start of the next one. `processCopyResults` now separates the two — a `SocketTimeoutException` raised while reading a body becomes a plain `IOException` and reaches `abortAfterCopyFailure`, while a timeout waiting for the next message to start is untouched.
`readTimeoutKeepsCopyActive` was asserting the unsound half without meaning to. `StrangeProxyServer` stops forwarding wherever it happens to be, and after the first `readFromCopy` the 8 KB buffer holds a partial message, so the timeout it observed landed inside one. It now starves the reader with `MessageStallProxyServer`, which forwards whole messages and can therefore pause on a boundary.
**Test timeouts default to `SEPARATE_THREAD`** (separate commit, droppable on its own). A test blocked in a socket read ignores `Thread.interrupt()`, so a same-thread timeout cannot stop it: JUnit reports the deadline only once the read returns, and a read that never returns hangs the build. Without this, a stalled replication test turns into a hung job rather than a failed one. It also covers `PhysicalReplicationTest` and `CopyBothResponseTest`, which have no `@Timeout` at all and relied on the global 5-minute default.
## How to verify
Six new tests, each checked to fail without its part of the change:
| test | what it pins |
| --- | --- |
| `CopyTest.readTimeoutKeepsCopyActive` | a reader starved **between messages** gets a timeout, the COPY stays active, `cancelCopy()` still works |
| `CopyTest.readTimeoutInsideMessageEndsTheCopy` | a reader starved **inside** a message is told the connection is gone, not handed something to resume from |
| `CopyTest.waitingThreadGivesUpWhenCopyDiesWithConnection` | a thread already parked on the COPY lock is woken and told the connection is gone |
| `CopyTest.allWaitingThreadsResumeWhenCopyEnds` | all three waiters resume, not just one |
| `CopyTest.startCopyFailureClosesConnection` | a COPY that cannot read its start response closes the connection |
| `CopyTest.cancelCopyFailureClosesConnection` | a cancel that never reached the server closes the connection |
| `LogicalReplicationTest.idleStreamSurvivesSocketReadTimeout` | a blocking read idles through several status intervals and keeps streaming |
`TestUtil.openReplicationConnection` gained a `Consumer<Properties>` overload, mirroring `openPrivilegedDB`, so the replication test can set `socketTimeout` itself instead of depending on the CI axis.
Verified against PostgreSQL 18.4 with SSL and SCRAM, both with the plain configuration and with the matrix flags of the job that first showed this (`socketTimeout=60`, `queryTimeout=15`, `connectTimeout=7`, `autosave=always`, `cleanupSavepoints=true`, `-XX:hashCode=2`, JaCoCo): full suite 9746 tests green, autostyle and checkstyle clean.
## Known issue
`endCopy` gives the connection up on a read timeout, where `readFromCopy` resumes from the same event. Reading again would be safe in itself — CopyDone is on the wire before the loop starts, so waiting for the answer a second time re-sends nothing — but resuming needs more than that catch, and the shipped callers are written for the current contract:
- `CopyManager.copyIn` ends with `finally { if (cp.isActive()) cp.cancelCopy(); }`. A copy left active by a timeout would take that branch and send CopyFail after CopyDone, which the backend ignores, so `cancelCopy` throws `Missing expected error response to copy cancel request` out of the `finally` and the real timeout is lost.
- `PGCopyOutputStream.close()` sets `this.op = null` after its try, so an `endCopy` that throws leaves the stream holding an operation whose lock nothing gives back, on a connection that is still open — the next borrower of a pooled connection then waits in `waitOnLock()` forever.
- `PGCopyOutputStream.endCopy()` does not reset its buffer offset, so a retry through the public stream would re-send up to 64 KB of CopyData.
Teaching those three the retryable contract is a change to the COPY API's behaviour rather than a fix for #4328, so it is left out of this PR. Recorded in the `abortAfterCopyFailure` javadoc as well, rather than only here.
Who this reaches: a caller that sets `socketTimeout` and ends a COPY the server is slow to acknowledge. **The stacked #4354 removes it for replication**, the case that would have hit it routinely: a replication stream shortens its wait to the status interval, so `V3PGReplicationStream.close()` used to run `endCopy` under a deadline of milliseconds. After #4354 the connection's own timeout is never shortened, and the stream puts its wake-up back before calling `endCopy`.
## Notes
- A cross-review of the stacked #4354 turned up that this PR's own test was asserting the unsound half of "a read timeout keeps the COPY", so the mid-message case is now handled here rather than left alone. See the last commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
inline patch 1: CHANGELOG.md, build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts, pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java and 4 more (+1023/-168 lines) download inline-1.patchdiff --git a/CHANGELOG.md b/CHANGELOG.mdindex aa062160d6..468cd869ae 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
### Fixed
* fix: `LargeObjectManager` operations now work inside an active XA transaction. Since 42.7.13 an XA branch keeps the caller's `autoCommit=true`, and the guard rejected large objects as if the connection were idle. It now checks the server transaction state (`getTransactionState()`) instead of `autoCommit`, so a genuine auto-commit connection with no open transaction is still refused [Issue #4309](https://github.com/pgjdbc/pgjdbc/issues/4309) [PR #4310](https://github.com/pgjdbc/pgjdbc/pull/4310)
* fix: batch update counts are no longer secured as committed inside an XA transaction. `BatchResultHandler` treated per-statement progress as durable whenever `autoCommit=true`, but an XA branch keeps `autoCommit=true` while the transaction is open, so a batch that flushed mid-way and then failed could report rolled-back statements as succeeded. Progress is now secured only when no server transaction is open [Issue #4309](https://github.com/pgjdbc/pgjdbc/issues/4309) [PR #4311](https://github.com/pgjdbc/pgjdbc/pull/4311)
+* fix: a socket read timeout no longer ends a COPY operation. Logical replication sets the socket timeout to the status interval and uses the timeout as its wake-up to send a standby status update, so since 42.7.11 a consumer of the blocking `PGReplicationStream.read()` API failed with "Tried to write to an inactive copy operation" as soon as the slot went idle. A plain `COPY TO STDOUT` with `socketTimeout` lost `cancelCopy()` the same way, leaving the server in COPY mode. A COPY that dies with the connection now closes that connection rather than releasing its lock, so other operations fail instead of hanging. A timeout that lands inside a backend message is a different matter and now ends the connection: the bytes the driver already took are not in the stream any more, so carrying on from there would read the middle of that message as the start of the next one [Issue #4328](https://github.com/pgjdbc/pgjdbc/issues/4328) [PR #4331](https://github.com/pgjdbc/pgjdbc/pull/4331)
## [42.7.13] (2026-07-06)
diff --git a/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts b/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.ktsindex 4007539786..0205830b59 100644--- a/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts+++ b/build-logic/jvm/src/main/kotlin/build-logic.test-junit5.gradle.kts@@ -42,4 +42,7 @@ tasks.configureEach<Test> {
// See https://github.com/junit-team/junit5/commit/347e3119d36a5c226cddd7981452f11335fad422
passProperty("junit.jupiter.execution.parallel.config.strategy", "DYNAMIC")
passProperty("junit.jupiter.execution.timeout.default", "5 m")
+ // A test blocked in a socket read ignores Thread.interrupt(), so a same-thread timeout cannot+ // stop it and the build hangs instead of reporting the failure+ passProperty("junit.jupiter.execution.timeout.thread.mode.default", "SEPARATE_THREAD")
}
diff --git a/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java b/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.javaindex 6bdc2a538a..cbee23c609 100644--- a/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java+++ b/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java@@ -298,7 +298,9 @@ private void unlock(Object holder) throws PSQLException {
PSQLState.OBJECT_NOT_IN_STATE);
}
lockedFor = null;
- lockCondition.signal();+ // Threads park in waitOnLock() without taking the lock, so waking one of them would leave+ // the rest waiting for a copy that is already over+ lockCondition.signalAll();
}
/**
@@ -307,6 +309,11 @@ private void unlock(Object holder) throws PSQLException {
*/
private void waitOnLock() throws PSQLException {
while (lockedFor != null) {
+ if (isClosed()) {+ // The copy holding the lock died with the connection, so it will never release it+ throw new PSQLException(GT.tr("This connection has been closed."),+ PSQLState.CONNECTION_DOES_NOT_EXIST);+ }
try {
lockCondition.await();
} catch (InterruptedException ie) {
@@ -318,6 +325,30 @@ private void waitOnLock() throws PSQLException {
}
}
+ /**+ * Gives up on the connection after an I/O failure during COPY. A copy that holds the lock keeps+ * it, since a desynchronized protocol stream is not free for anyone else to use; closing the+ * connection is what lets threads waiting in {@link #waitOnLock()} fail instead of waiting for a+ * copy that can no longer end.+ *+ * <p>The COPY methods call this for every I/O failure except one: a read timeout in+ * {@link #readFromCopy}, the only failure a caller can resume from by reading again. A failed+ * send has already put part of a message on the wire, and {@link #startCopy} cannot be retried,+ * since that would send the query a second time.</p>+ *+ * <p>{@link #endCopy} gives the connection up on a read timeout too, though reading again would+ * be safe there as well: CopyDone is on the wire before the loop starts, so waiting for the+ * answer a second time re-sends nothing. Resuming would need a limit on how long that answer may+ * take, since a server that never sends it would otherwise keep the loop going, and the deadline+ * the caller asked for is the only limit there is today.</p>+ */+ private void abortAfterCopyFailure() {+ abort();+ try (ResourceLock ignore = lock.obtain()) {+ lockCondition.signalAll();+ }+ }+
/**
* @param holder object assumed to hold the lock
* @return whether given object actually holds the lock
@@ -1116,6 +1147,7 @@ public CopyOperation startCopy(String sql, boolean suppressBegin)
return castNonNull(processCopyResults(null, true));
// expect a CopyInResponse or CopyOutResponse to our query above
} catch (IOException ioe) {
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when starting copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1193,13 +1225,12 @@ public void cancelCopy(CopyOperationImpl op) throws SQLException {
}
} catch (IOException ioe) {
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when canceling copy operation"),
PSQLState.CONNECTION_FAILURE, ioe);
} finally {
- // Need to ensure the lock isn't held anymore, or else- // future operations, rather than failing due to the- // broken connection, will simply hang waiting for this- // lock.+ // Canceling ends the copy either way, so the lock has to go back. Otherwise later+ // operations would wait for a copy that is already over.
try (ResourceLock ignore = lock.obtain()) {
if (hasLock(op)) {
unlock(op);
@@ -1243,15 +1274,19 @@ public long endCopy(CopyOperationImpl op) throws SQLException {
processCopyResults(op, true);
} while (hasLock(op));
return op.getHandledRowCount();
+ } catch (SQLException e) {+ // A copy the server failed has already been unlocked by the ReadyForQuery that followed+ // its error, and that connection is fine. Still holding the lock means the exchange+ // stopped somewhere else, so the copy is neither over nor resumable and nothing may wait+ // on a lock it will never get back+ if (hasLock(op)) {+ abortAfterCopyFailure();+ }+ throw e;
} catch (IOException ioe) {
+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when ending copy"),
PSQLState.CONNECTION_FAILURE, ioe);
- } finally {- // Release the lock if still held, otherwise future operations- // will hang indefinitely in waitOnLock()- if (hasLock(op)) {- unlock(op);- }
}
}
}
@@ -1281,9 +1316,7 @@ public void writeToCopy(CopyOperationImpl op, byte[] data, int off, int siz)
pgStream.sendInteger4(siz + 4);
pgStream.send(data, off, siz);
} catch (IOException ioe) {
- if (hasLock(op)) {- unlock(op);- }+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when writing to copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1314,9 +1347,7 @@ public void writeToCopy(CopyOperationImpl op, ByteStreamWriter from)
pgStream.sendInteger4(siz + 4);
pgStream.send(from);
} catch (IOException ioe) {
- if (hasLock(op)) {- unlock(op);- }+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when writing to copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1333,9 +1364,7 @@ public void flushCopy(CopyOperationImpl op) throws SQLException {
try {
pgStream.flush();
} catch (IOException ioe) {
- if (hasLock(op)) {- unlock(op);- }+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when writing to copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1359,12 +1388,14 @@ void readFromCopy(CopyOperationImpl op, boolean block) throws SQLException {
try {
processCopyResults(op, block); // expect a call to handleCopydata() to store the data
+ } catch (SocketTimeoutException ste) {+ // A read timeout leaves the socket usable, so the copy stays active and the caller can+ // read again. Replication depends on that: the stream sets the socket timeout to the+ // status interval and treats the timeout as its wake-up to send a standby status update.+ throw new PSQLException(GT.tr("Database connection failed when reading from copy"),+ PSQLState.CONNECTION_FAILURE, ste);
} catch (IOException ioe) {
- // Release the lock if still held after a connection failure,- // otherwise future operations will hang indefinitely in waitOnLock()- if (hasLock(op)) {- unlock(op);- }+ abortAfterCopyFailure();
throw new PSQLException(GT.tr("Database connection failed when reading from copy"),
PSQLState.CONNECTION_FAILURE, ioe);
}
@@ -1408,194 +1439,220 @@ void readFromCopy(CopyOperationImpl op, boolean block) throws SQLException {
SQLException errors = null;
int len;
- while (!endReceiving && (block || pgStream.hasMessagePending())) {-- // There is a bug in the server's implementation of the copy- // protocol. It returns command complete immediately upon- // receiving the EOF marker in the binary protocol,- // potentially before we've issued CopyDone. When we are not- // blocking, we don't think we are done, so we hold off on- // processing command complete and any subsequent messages- // until we actually are done with the copy.- //- if (!block) {- int c = pgStream.peekChar();- if (c == PgMessageType.COMMAND_COMPLETE_RESPONSE) {- LOGGER.log(Level.FINEST, " <=BE CommandStatus, Ignored until CopyDone");- break;+ try {+ while (!endReceiving && (block || pgStream.hasMessagePending())) {++ // There is a bug in the server's implementation of the copy+ // protocol. It returns command complete immediately upon+ // receiving the EOF marker in the binary protocol,+ // potentially before we've issued CopyDone. When we are not+ // blocking, we don't think we are done, so we hold off on+ // processing command complete and any subsequent messages+ // until we actually are done with the copy.+ //+ if (!block) {+ int c = pgStream.peekChar();+ if (c == PgMessageType.COMMAND_COMPLETE_RESPONSE) {+ LOGGER.log(Level.FINEST, " <=BE CommandStatus, Ignored until CopyDone");+ break;+ }
}
- }- int c = pgStream.receiveChar();- switch (c) {+ int c = pgStream.receiveChar();+ try {+ switch (c) {- case PgMessageType.ASYNCHRONOUS_NOTICE:- LOGGER.log(Level.FINEST, " <=BE Asynchronous Notification while copying");+ case PgMessageType.ASYNCHRONOUS_NOTICE:+ LOGGER.log(Level.FINEST, " <=BE Asynchronous Notification while copying");- receiveAsyncNotify();- break;+ receiveAsyncNotify();+ break;- case PgMessageType.NOTICE_RESPONSE:+ case PgMessageType.NOTICE_RESPONSE:- LOGGER.log(Level.FINEST, " <=BE Notification while copying");+ LOGGER.log(Level.FINEST, " <=BE Notification while copying");- addWarning(receiveNoticeResponse());- break;+ addWarning(receiveNoticeResponse());+ break;- case PgMessageType.COMMAND_COMPLETE_RESPONSE: // Command Complete+ case PgMessageType.COMMAND_COMPLETE_RESPONSE: // Command Complete- String status = receiveCommandStatus();+ String status = receiveCommandStatus();- try {- if (op == null) {- throw new PSQLException(GT- .tr("Received CommandComplete ''{0}'' without an active copy operation", status),- PSQLState.OBJECT_NOT_IN_STATE);- }- op.handleCommandStatus(status);- } catch (SQLException se) {- error = se;- }+ try {+ if (op == null) {+ throw new PSQLException(GT+ .tr("Received CommandComplete ''{0}'' without an active copy operation", status),+ PSQLState.OBJECT_NOT_IN_STATE);+ }+ op.handleCommandStatus(status);+ } catch (SQLException se) {+ error = se;+ }- block = true;- break;+ block = true;+ break;- case PgMessageType.ERROR_RESPONSE: // ErrorMessage (expected response to CopyFail)+ case PgMessageType.ERROR_RESPONSE: // ErrorMessage (expected response to CopyFail)- error = receiveErrorResponse();- // We've received the error and we now expect to receive- // Ready for query, but we must block because it might still be- // on the wire and not here yet.- block = true;- break;+ error = receiveErrorResponse();+ // We've received the error and we now expect to receive+ // Ready for query, but we must block because it might still be+ // on the wire and not here yet.+ block = true;+ break;- case PgMessageType.COPY_IN_RESPONSE: // CopyInResponse+ case PgMessageType.COPY_IN_RESPONSE: // CopyInResponse- LOGGER.log(Level.FINEST, " <=BE CopyInResponse");+ LOGGER.log(Level.FINEST, " <=BE CopyInResponse");- if (op != null) {- error = new PSQLException(GT.tr("Got CopyInResponse from server during an active {0}",- op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);- }+ if (op != null) {+ error = new PSQLException(GT.tr("Got CopyInResponse from server during an active {0}",+ op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);+ }- op = new CopyInImpl();- initCopy(op);- endReceiving = true;- break;+ op = new CopyInImpl();+ initCopy(op);+ endReceiving = true;+ break;- case PgMessageType.COPY_OUT_RESPONSE: // CopyOutResponse+ case PgMessageType.COPY_OUT_RESPONSE: // CopyOutResponse- LOGGER.log(Level.FINEST, " <=BE CopyOutResponse");+ LOGGER.log(Level.FINEST, " <=BE CopyOutResponse");- if (op != null) {- error = new PSQLException(GT.tr("Got CopyOutResponse from server during an active {0}",- op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);- }+ if (op != null) {+ error = new PSQLException(GT.tr("Got CopyOutResponse from server during an active {0}",+ op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);+ }- op = new CopyOutImpl();- initCopy(op);- endReceiving = true;- break;+ op = new CopyOutImpl();+ initCopy(op);+ endReceiving = true;+ break;- case PgMessageType.COPY_BOTH_RESPONSE: // CopyBothResponse+ case PgMessageType.COPY_BOTH_RESPONSE: // CopyBothResponse- LOGGER.log(Level.FINEST, " <=BE CopyBothResponse");+ LOGGER.log(Level.FINEST, " <=BE CopyBothResponse");- if (op != null) {- error = new PSQLException(GT.tr("Got CopyBothResponse from server during an active {0}",- op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);- }+ if (op != null) {+ error = new PSQLException(GT.tr("Got CopyBothResponse from server during an active {0}",+ op.getClass().getName()), PSQLState.OBJECT_NOT_IN_STATE);+ }- op = new CopyDualImpl();- initCopy(op);- endReceiving = true;- break;+ op = new CopyDualImpl();+ initCopy(op);+ endReceiving = true;+ break;- case PgMessageType.COPY_DATA: // CopyData+ case PgMessageType.COPY_DATA: // CopyData- LOGGER.log(Level.FINEST, " <=BE CopyData");+ LOGGER.log(Level.FINEST, " <=BE CopyData");- len = pgStream.receiveInteger4() - 4;+ len = pgStream.receiveInteger4() - 4;- assert len > 0 : "Copy Data length must be greater than 4";+ assert len > 0 : "Copy Data length must be greater than 4";- byte[] buf = pgStream.receive(len);- if (op == null) {- error = new PSQLException(GT.tr("Got CopyData without an active copy operation"),- PSQLState.OBJECT_NOT_IN_STATE);- } else if (!(op instanceof CopyOut)) {- error = new PSQLException(- GT.tr("Unexpected copydata from server for {0}", op.getClass().getName()),- PSQLState.COMMUNICATION_ERROR);- } else {- op.handleCopydata(buf);- }- endReceiving = true;- break;+ byte[] buf = pgStream.receive(len);+ if (op == null) {+ error = new PSQLException(GT.tr("Got CopyData without an active copy operation"),+ PSQLState.OBJECT_NOT_IN_STATE);+ } else if (!(op instanceof CopyOut)) {+ error = new PSQLException(+ GT.tr("Unexpected copydata from server for {0}", op.getClass().getName()),+ PSQLState.COMMUNICATION_ERROR);+ } else {+ op.handleCopydata(buf);+ }+ endReceiving = true;+ break;- case PgMessageType.COPY_DONE: // CopyDone (expected after all copydata received)+ case PgMessageType.COPY_DONE: // CopyDone (expected after all copydata received)- LOGGER.log(Level.FINEST, " <=BE CopyDone");+ LOGGER.log(Level.FINEST, " <=BE CopyDone");- len = pgStream.receiveInteger4() - 4;- if (len > 0) {- pgStream.receive(len); // not in specification; should never appear- }+ len = pgStream.receiveInteger4() - 4;+ if (len > 0) {+ pgStream.receive(len); // not in specification; should never appear+ }- if (!(op instanceof CopyOut)) {- error = new PSQLException("Got CopyDone while not copying from server",- PSQLState.OBJECT_NOT_IN_STATE);- }+ if (!(op instanceof CopyOut)) {+ error = new PSQLException("Got CopyDone while not copying from server",+ PSQLState.OBJECT_NOT_IN_STATE);+ }- // keep receiving since we expect a CommandComplete- block = true;- break;- case PgMessageType.PARAMETER_STATUS_RESPONSE: // Parameter Status- try {- receiveParameterStatus();- } catch (SQLException e) {- error = e;- endReceiving = true;- }- break;+ // keep receiving since we expect a CommandComplete+ block = true;+ break;+ case PgMessageType.PARAMETER_STATUS_RESPONSE: // Parameter Status+ try {+ receiveParameterStatus();+ } catch (SQLException e) {+ error = e;+ endReceiving = true;+ }+ break;- case PgMessageType.READY_FOR_QUERY_RESPONSE: // ReadyForQuery: After FE:CopyDone => BE:CommandComplete+ case PgMessageType.READY_FOR_QUERY_RESPONSE: // ReadyForQuery: After FE:CopyDone => BE:CommandComplete- receiveRFQ();- if (op != null && hasLock(op)) {- unlock(op);- }- op = null;- endReceiving = true;- break;+ receiveRFQ();+ if (op != null && hasLock(op)) {+ unlock(op);+ }+ op = null;+ endReceiving = true;+ break;- // If the user sends a non-copy query, we've got to handle some additional things.- //- case PgMessageType.ROW_DESCRIPTION_RESPONSE: // Row Description (response to Describe)- LOGGER.log(Level.FINEST, " <=BE RowDescription (during copy ignored)");+ // If the user sends a non-copy query, we've got to handle some additional things.+ //+ case PgMessageType.ROW_DESCRIPTION_RESPONSE: // Row Description (response to Describe)+ LOGGER.log(Level.FINEST, " <=BE RowDescription (during copy ignored)");- skipMessage();- break;+ skipMessage();+ break;- case PgMessageType.DATA_ROW_RESPONSE: // DataRow- LOGGER.log(Level.FINEST, " <=BE DataRow (during copy ignored)");+ case PgMessageType.DATA_ROW_RESPONSE: // DataRow+ LOGGER.log(Level.FINEST, " <=BE DataRow (during copy ignored)");- skipMessage();- break;+ skipMessage();+ break;- default:- throw new IOException(- GT.tr("Unexpected packet type during copy: {0}", Integer.toString(c)));- }+ default:+ throw new IOException(+ GT.tr("Unexpected packet type during copy: {0}", Integer.toString(c)));+ }+ } catch (SocketTimeoutException ste) {+ // The type byte is already out of the stream, so there is no way back to a+ // message boundary. Report it as the connection failure it is rather than as+ // something the caller can resume from, which would read the middle of this+ // message as the start of the next one+ IOException desynchronized = new IOException(GT.tr(+ "Read timed out inside a backend message, so the copy stream is out of step"), ste);+ if (errors != null) {+ // The backend had already reported this copy failed. That is why the copy is over;+ // losing the message boundary is only why the connection goes with it+ desynchronized.addSuppressed(errors);+ }+ throw desynchronized;+ }- // Collect errors into a neat chain for completeness- if (error != null) {- if (errors != null) {- error.setNextException(errors);+ // Collect errors into a neat chain for completeness+ if (error != null) {+ if (errors != null) {+ error.setNextException(errors);+ }+ errors = error;+ error = null;
}
- errors = error;- error = null;
}
++ } catch (SocketTimeoutException ste) {+ if (errors != null) {+ // The backend has already reported this copy failed. Handing the caller a timeout+ // instead would drop that and read as "nothing arrived yet", so a caller that reads+ // again could see the copy end successfully+ throw errors;+ }+ throw ste;
}
if (errors != null) {
diff --git a/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.java b/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.javaindex 890329fe2f..e02859adf6 100644--- a/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.java+++ b/pgjdbc/src/test/java/org/postgresql/replication/LogicalReplicationTest.java@@ -11,6 +11,7 @@
import static org.junit.jupiter.api.Assumptions.assumeTrue;
import org.postgresql.PGConnection;
+import org.postgresql.PGProperty;
import org.postgresql.core.BaseConnection;
import org.postgresql.core.ServerVersion;
import org.postgresql.test.TestUtil;
@@ -45,6 +46,12 @@
@EnabledForServerVersionRange(gte = "9.4")
class LogicalReplicationTest {
private static final String SLOT_NAME = "pgjdbc_logical_replication_slot";
+ private static final int STATUS_INTERVAL_MS = 200;+ /**+ * Slot for {@link #idleStreamSurvivesSocketReadTimeout()}, which streams over a second connection+ * of its own. Sharing {@link #SLOT_NAME} would put two consumers on one slot.+ */+ private static final String IDLE_SLOT_NAME = SLOT_NAME + "_idle";
private Connection replConnection;
private Connection sqlConnection;
@@ -632,6 +639,70 @@ public Object call() throws Exception {
);
}
+ /**+ * A replication stream sets the socket timeout to the status interval and uses the resulting read+ * timeout as its wake-up to send a standby status update. The blocking read has to survive that+ * timeout, however many times it fires while the source is idle.+ *+ * <p>The connection sets {@code socketTimeout} on purpose: without it+ * {@link org.postgresql.core.VisibleBufferedInputStream#setTimeoutRequested(boolean)} leaves the+ * driver retrying the read instead of reporting the timeout, so the wake-up path is never+ * exercised.</p>+ *+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/4328">Issue 4328</a>+ */+ @Test+ @Timeout(30)+ void idleStreamSurvivesSocketReadTimeout() throws Exception {+ // The blocking read times out after min(socketTimeout, statusInterval), so it wakes up every+ // status interval and socketTimeout stays the connection-level guard+ TestUtil.recreateLogicalReplicationSlot(sqlConnection, IDLE_SLOT_NAME, "test_decoding");+ try (Connection connection = TestUtil.openReplicationConnection(+ props -> PGProperty.SOCKET_TIMEOUT.set(props, 10))) {+ PGReplicationStream stream =+ ((PGConnection) connection)+ .getReplicationAPI()+ .replicationStream()+ .logical()+ .withSlotName(IDLE_SLOT_NAME)+ .withStartPosition(getCurrentLSN())+ .withSlotOption("include-xids", false)+ .withSlotOption("skip-empty-xacts", true)+ .withStatusInterval(STATUS_INTERVAL_MS, TimeUnit.MILLISECONDS)+ .start();++ ExecutorService executor = Executors.newSingleThreadExecutor();+ try {+ // Nothing happens on the source for several status intervals, so the read spends that+ // time waking up on socket timeouts and sending status updates+ Future<?> insert = executor.submit(() -> {+ TimeUnit.MILLISECONDS.sleep(STATUS_INTERVAL_MS * 10);+ try (Statement st = sqlConnection.createStatement()) {+ st.execute("insert into test_logic_table(name) values('after idle')");+ }+ return null;+ });++ String result = group(receiveMessage(stream, 3));+ insert.get(10, TimeUnit.SECONDS);++ String wait = group(Arrays.asList(+ "BEGIN",+ "table public.test_logic_table: INSERT: pk[integer]:1 name[character varying]:'after idle'",+ "COMMIT"+ ));++ assertThat("The stream should keep streaming after idling through several socket timeouts",+ result, equalTo(wait)+ );+ } finally {+ executor.shutdownNow();+ }+ } finally {+ TestUtil.dropReplicationSlot(sqlConnection, IDLE_SLOT_NAME);+ }+ }+
@Test
void restartReplicationFromRestartSlotLSNWhenFeedbackAbsent() throws Exception {
PGConnection pgConnection = (PGConnection) replConnection;
diff --git a/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.java b/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.javaindex 3ff95fcb55..828704a336 100644--- a/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.java+++ b/pgjdbc/src/test/java/org/postgresql/test/jdbc2/CopyTest.java@@ -7,6 +7,7 @@
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTimeout;
@@ -23,6 +24,7 @@
import org.postgresql.core.ServerVersion;
import org.postgresql.jdbc.PgConnection;
import org.postgresql.test.TestUtil;
+import org.postgresql.test.util.MessageStallProxyServer;
import org.postgresql.test.util.StrangeProxyServer;
import org.postgresql.util.ByteBufferByteStreamWriter;
import org.postgresql.util.PSQLState;
@@ -39,6 +41,7 @@
import java.io.OutputStream;
import java.io.PrintStream;
import java.io.StringReader;
+import java.net.SocketTimeoutException;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
@@ -46,9 +49,17 @@
import java.sql.SQLException;
import java.sql.Statement;
import java.time.Duration;
+import java.util.ArrayList;+import java.util.Arrays;+import java.util.List;
import java.util.Locale;
import java.util.Properties;
+import java.util.concurrent.ExecutorService;+import java.util.concurrent.Executors;+import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;+import java.util.concurrent.atomic.AtomicReference;
/**
* @author kato@iki.fi
@@ -701,4 +712,462 @@ void isValidShouldNotHangAfterCopyInputStreamNetworkFailure() throws Exception {
}
}
}
++ /**+ * A socket read timeout is not a connection failure, so it must leave the COPY operation active.+ * Deactivating the COPY takes away {@link CopyOut#cancelCopy()}, the only way out of a stalled+ * one, and it takes away the wake-up logical replication sends its status updates on.+ *+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/4328">Issue 4328</a>+ */+ @Test+ @Timeout(value = 60, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void readTimeoutKeepsCopyActive() throws Exception {+ TestUtil.createTable(con, "copytest_timeout", "data text");+ try {+ try (Statement stmt = con.createStatement()) {+ stmt.execute(+ "INSERT INTO copytest_timeout SELECT 'row' || g FROM generate_series(1, 20000) g");+ }++ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));+ // Without socketTimeout the driver retries the read instead of reporting the timeout+ PGProperty.SOCKET_TIMEOUT.set(props, 1);+ // The cancel request goes through the same dead proxy, so do not wait 10s for its EOF+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);+ // The proxy reads the v3 framing to find a message boundary, so it needs plain traffic+ PGProperty.SSL_MODE.set(props, "disable");+ PGProperty.GSS_ENC_MODE.set(props, "disable");++ try (Connection proxyCon = TestUtil.openDB(props)) {+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();+ CopyOut copyOut = manager.copyOut("COPY copytest_timeout TO STDOUT");+ assertNotNull(copyOut.readFromCopy(), "Expected data row from COPY TO STDOUT");++ // Starve the reader between messages, which is the only place a timeout leaves the+ // stream somewhere the caller can carry on from+ proxyServer.stallBeforeNextMessage(TimeUnit.SECONDS.toMillis(30));++ SQLException timeout = readUntilTimeout(copyOut);+ assertInstanceOf(SocketTimeoutException.class, timeout.getCause(),+ () -> "readFromCopy should report the socket read timeout, got " + timeout);+ assertTrue(copyOut.isActive(),+ "A socket read timeout must not deactivate the COPY operation");++ // Canceling is the way out of a stalled COPY, and it needs the COPY lock+ copyOut.cancelCopy();+ assertFalse(copyOut.isActive(), "cancelCopy() should end the COPY operation");+ }+ }+ } finally {+ TestUtil.dropTable(con, "copytest_timeout");+ }+ }++ /**+ * Fails when a read timeout inside a backend message is reported as something the caller can+ * carry on from. The bytes the driver already took are not in the stream any more, so the next+ * read would take the middle of that message for the start of the next one.+ */+ @Test+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void readTimeoutInsideMessageEndsTheCopy() throws Exception {+ TestUtil.createTable(con, "copytest_stall", "data text");+ try {+ try (Statement stmt = con.createStatement()) {+ stmt.execute(+ "INSERT INTO copytest_stall SELECT 'row' || g FROM generate_series(1, 20000) g");+ }++ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));+ PGProperty.SOCKET_TIMEOUT.set(props, 1);+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);+ PGProperty.SSL_MODE.set(props, "disable");+ PGProperty.GSS_ENC_MODE.set(props, "disable");++ try (Connection proxyCon = TestUtil.openDB(props)) {+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();+ CopyOut copyOut = manager.copyOut("COPY copytest_stall TO STDOUT");+ assertNotNull(copyOut.readFromCopy(), "Expected data row from COPY TO STDOUT");++ proxyServer.stallInsideNextMessage(TimeUnit.SECONDS.toMillis(30));++ SQLException timeout = readUntilTimeout(copyOut);+ assertInstanceOf(IOException.class, timeout.getCause(),+ () -> "a timeout inside a message is a connection failure, got " + timeout);+ assertFalse(timeout.getCause() instanceof SocketTimeoutException,+ "a timeout inside a message must not be reported as one the caller can resume from");+ assertTrue(proxyCon.isClosed(),+ "a stream that lost its message boundary has to take the connection with it");+ assertThrows(SQLException.class, copyOut::readFromCopy,+ "the copy must not look like something to carry on reading from");+ }+ }+ } finally {+ TestUtil.dropTable(con, "copytest_stall");+ }+ }+++ /**+ * A COPY that dies with the connection keeps the COPY lock, so a thread already waiting for that+ * lock has to be told the connection is gone rather than wait for a copy that can no longer end.+ *+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/3957">Issue 3957</a>+ */+ @Test+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void waitingThreadGivesUpWhenCopyDiesWithConnection() throws Exception {+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));++ try (Connection proxyCon = TestUtil.openDB(props)) {+ TestUtil.createTempTable(proxyCon, "copytest_wait", "data text");++ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();+ CopyIn copyIn = manager.copyIn("COPY copytest_wait (data) FROM STDIN WITH (FORMAT CSV)");+ byte[] row = "somedata\n".getBytes(StandardCharsets.UTF_8);+ copyIn.writeToCopy(row, 0, row.length);+ copyIn.flushCopy();++ ExecutorService executor = Executors.newSingleThreadExecutor();+ try {+ AtomicReference<Thread> waiter = new AtomicReference<>();+ Future<Boolean> valid = executor.submit(() -> {+ waiter.set(Thread.currentThread());+ return proxyCon.isValid(20);+ });+ awaitWaitingForCopyLock(waiter);++ // Break the connection under the COPY while the other thread waits for the lock+ proxyServer.closeAllClients();+ assertThrows(SQLException.class, () -> {+ for (int i = 0; i < 100_000; i++) {+ copyIn.writeToCopy(row, 0, row.length);+ copyIn.flushCopy();+ }+ }, "Expected SQLException from broken connection during writeToCopy");++ assertFalse(valid.get(10, TimeUnit.SECONDS),+ "The waiting thread should report the connection as invalid instead of waiting for "+ + "a COPY that can no longer end");+ } finally {+ executor.shutdownNow();+ }+ }+ }+ }++ /**+ * The query that starts a COPY is already on the wire when the connection breaks, so its response+ * can neither be read nor asked for again. The driver has to give the connection up rather than+ * hand back a live one whose protocol stream is out of step.+ */+ @Test+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void startCopyFailureClosesConnection() throws Exception {+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));++ try (Connection proxyCon = TestUtil.openDB(props)) {+ TestUtil.createTempTable(proxyCon, "copytest_start", "data text");+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();++ proxyServer.closeAllClients();++ assertThrows(SQLException.class,+ () -> manager.copyIn("COPY copytest_start (data) FROM STDIN WITH (FORMAT CSV)"),+ "Expected SQLException from broken connection while starting COPY");+ assertTrue(proxyCon.isClosed(),+ "A COPY that cannot read its own start response leaves the protocol stream out of "+ + "step, so the connection should be given up");+ }+ }+ }++ /**+ * Canceling a COPY over a broken connection cannot reach the server, so the driver has to give+ * the connection up. The COPY itself is over either way.+ */+ @Test+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void cancelCopyFailureClosesConnection() throws Exception {+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));++ try (Connection proxyCon = TestUtil.openDB(props)) {+ TestUtil.createTempTable(proxyCon, "copytest_cancel", "data text");++ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();+ CopyIn copyIn = manager.copyIn("COPY copytest_cancel (data) FROM STDIN WITH (FORMAT CSV)");+ byte[] row = "somedata\n".getBytes(StandardCharsets.UTF_8);+ copyIn.writeToCopy(row, 0, row.length);+ copyIn.flushCopy();++ proxyServer.closeAllClients();++ assertThrows(SQLException.class, copyIn::cancelCopy,+ "Expected SQLException from broken connection while canceling COPY");+ assertFalse(copyIn.isActive(), "cancelCopy() should end the COPY operation");+ assertTrue(proxyCon.isClosed(),+ "A cancel that never reached the server should give the connection up");+ }+ }+ }++ /**+ * The {@link CopyIn#writeToCopy(org.postgresql.util.ByteStreamWriter)} overload has to give the+ * connection up on an I/O failure just like the byte-array one.+ *+ * @see <a href="https://github.com/pgjdbc/pgjdbc/issues/3957">Issue 3957</a>+ */+ @Test+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void writeToCopyFromByteStreamWriterFailureClosesConnection() throws Exception {+ try (StrangeProxyServer proxyServer = new StrangeProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));++ try (Connection proxyCon = TestUtil.openDB(props)) {+ TestUtil.createTempTable(proxyCon, "copytest_writer", "data text");++ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();+ CopyIn copyIn = manager.copyIn("COPY copytest_writer (data) FROM STDIN WITH (FORMAT CSV)");+ byte[] row = "somedata\n".getBytes(StandardCharsets.UTF_8);+ copyIn.writeToCopy(new ByteBufferByteStreamWriter(ByteBuffer.wrap(row)));+ copyIn.flushCopy();++ proxyServer.closeAllClients();++ // A payload this large outgrows the output buffer, so it has to reach the socket within+ // writeToCopy itself rather than waiting for the flush+ byte[] bigRow = new byte[1024 * 1024];+ Arrays.fill(bigRow, (byte) 'x');+ bigRow[bigRow.length - 1] = '\n';+ assertThrows(SQLException.class,+ () -> copyIn.writeToCopy(new ByteBufferByteStreamWriter(ByteBuffer.wrap(bigRow))),+ "Expected SQLException from broken connection during writeToCopy");++ assertTrue(proxyCon.isClosed(),+ "A COPY that failed to send should give the connection up");+ }+ }+ }++ /**+ * Interrupting a thread that waits for a COPY to finish fails its query and leaves the interrupt+ * flag set, rather than swallowing the interrupt.+ */+ @Test+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void interruptingAThreadWaitingForTheCopyKeepsTheInterruptFlag() throws Exception {+ CopyIn copyIn = copyAPI.copyIn("COPY copytest FROM STDIN");+ byte[] row = origData[0].getBytes(StandardCharsets.UTF_8);+ copyIn.writeToCopy(row, 0, row.length);+ copyIn.flushCopy();++ ExecutorService executor = Executors.newSingleThreadExecutor();+ try {+ AtomicReference<Thread> waiter = new AtomicReference<>();+ AtomicBoolean stillInterrupted = new AtomicBoolean();+ Future<SQLException> failure = executor.submit(() -> {+ waiter.set(Thread.currentThread());+ try (Statement stmt = con.createStatement()) {+ stmt.execute("SELECT 1");+ return null;+ } catch (SQLException e) {+ stillInterrupted.set(Thread.currentThread().isInterrupted());+ return e;+ }+ });+ awaitWaitingForCopyLock(waiter).interrupt();++ SQLException e = failure.get(10, TimeUnit.SECONDS);+ assertNotNull(e, "The interrupted query should fail instead of waiting for the COPY");+ assertEquals(PSQLState.OBJECT_NOT_IN_STATE.getState(), e.getSQLState(),+ () -> "Unexpected failure: " + e);+ assertTrue(stillInterrupted.get(), "The interrupt flag should survive the failure");+ } finally {+ executor.shutdownNow();+ copyIn.endCopy();+ }+ }++ /**+ * Every thread waiting for a COPY to finish has to be woken when it does, not just one of them.+ */+ @Test+ @Timeout(value = 30, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void allWaitingThreadsResumeWhenCopyEnds() throws Exception {+ CopyIn copyIn = copyAPI.copyIn("COPY copytest FROM STDIN");+ byte[] row = origData[0].getBytes(StandardCharsets.UTF_8);+ copyIn.writeToCopy(row, 0, row.length);+ copyIn.flushCopy();++ ExecutorService executor = Executors.newFixedThreadPool(3);+ try {+ List<Future<Boolean>> waiters = new ArrayList<>();+ List<AtomicReference<Thread>> threads = new ArrayList<>();+ for (int i = 0; i < 3; i++) {+ AtomicReference<Thread> thread = new AtomicReference<>();+ threads.add(thread);+ waiters.add(executor.submit(() -> {+ thread.set(Thread.currentThread());+ return con.isValid(20);+ }));+ }+ for (AtomicReference<Thread> thread : threads) {+ awaitWaitingForCopyLock(thread);+ }++ copyIn.endCopy();++ for (Future<Boolean> waiter : waiters) {+ assertTrue(waiter.get(10, TimeUnit.SECONDS),+ "Every thread waiting for the COPY should resume once it ends");+ }+ } finally {+ executor.shutdownNow();+ }+ }++ /**+ * Waits until the helper thread parks on the COPY lock. Fails the test if it never does.+ */+ private static Thread awaitWaitingForCopyLock(AtomicReference<Thread> waiter) {+ long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(10);+ while (System.nanoTime() < deadline) {+ Thread thread = waiter.get();+ if (thread != null && thread.getState() == Thread.State.WAITING) {+ return thread;+ }+ Thread.yield();+ }+ return fail("The helper thread never started waiting for the COPY lock");+ }++ /**+ * Fails when a read timeout throws away an error the backend already sent. The copy loop keeps+ * the error until the ReadyForQuery that follows it, so a timeout waiting for that message must+ * not be reported as "nothing arrived yet" — a caller that reads again would then see the copy+ * finish successfully.+ */+ @Test+ @Timeout(value = 60, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void readTimeoutAfterAnErrorResponseStillReportsTheError() throws Exception {+ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));+ PGProperty.SOCKET_TIMEOUT.set(props, 1);+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);+ PGProperty.SSL_MODE.set(props, "disable");+ PGProperty.GSS_ENC_MODE.set(props, "disable");++ try (Connection proxyCon = TestUtil.openDB(props)) {+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();+ // The last row divides by zero, so the server streams data and then reports an error+ CopyOut copyOut = manager.copyOut(+ "COPY (SELECT 1 / (20000 - g) FROM generate_series(1, 20000) g) TO STDOUT");++ // Hold back the ReadyForQuery that closes the exchange, so the read that waits for it+ // times out with the error already in hand+ proxyServer.stallBeforeMessageType('Z', TimeUnit.SECONDS.toMillis(20));++ SQLException thrown = readUntilTimeout(copyOut);+ assertEquals(PSQLState.DIVISION_BY_ZERO.getState(), thrown.getSQLState(),+ () -> "the backend error has to survive the timeout that followed it, got " + thrown);+ }+ }+ }++ /**+ * Fails when endCopy keeps the COPY lock after the exchange stopped somewhere other than+ * ReadyForQuery. A retained lock with a live connection is a lock nothing can take back:+ * waitOnLock() gives up only once the connection is closed, so every later operation on it+ * waits forever.+ */+ @Test+ @Timeout(value = 60, unit = TimeUnit.SECONDS, threadMode = Timeout.ThreadMode.SEPARATE_THREAD)+ void endCopyGivesTheConnectionUpWhenItCannotReleaseTheLock() throws Exception {+ try (MessageStallProxyServer proxyServer = new MessageStallProxyServer(TestUtil.getServer(),+ TestUtil.getPort())) {+ Properties props = new Properties();+ TestUtil.setTestUrlProperty(props, PGProperty.PG_HOST, "localhost");+ TestUtil.setTestUrlProperty(props, PGProperty.PG_PORT,+ String.valueOf(proxyServer.getServerPort()));+ PGProperty.CANCEL_SIGNAL_TIMEOUT.set(props, 1);+ PGProperty.SSL_MODE.set(props, "disable");+ PGProperty.GSS_ENC_MODE.set(props, "disable");++ try (Connection proxyCon = TestUtil.openDB(props)) {+ TestUtil.createTempTable(proxyCon, "copytest_endcopy", "data text");+ CopyManager manager = proxyCon.unwrap(PGConnection.class).getCopyAPI();+ CopyIn copyIn = manager.copyIn("COPY copytest_endcopy FROM STDIN");+ byte[] row = "one\n".getBytes(StandardCharsets.UTF_8);+ copyIn.writeToCopy(row, 0, row.length);++ // CopyData is the server's answer to a COPY TO, never to a COPY FROM. The copy loop stops+ // on it without reaching the ReadyForQuery that would hand the lock back+ proxyServer.injectBeforeNextMessage(copyDataMessage("stray"));++ assertThrows(SQLException.class, copyIn::endCopy,+ "the driver should report the message it cannot make sense of");+ assertTrue(proxyCon.isClosed(),+ "a copy that cannot give its lock back has to take the connection with it");+ }+ }+ }++ /** Builds a CopyData message: the type byte, a self-inclusive length, then the payload. */+ private static byte[] copyDataMessage(String payload) {+ byte[] body = payload.getBytes(StandardCharsets.UTF_8);+ ByteBuffer message = ByteBuffer.allocate(1 + 4 + body.length);+ message.put((byte) 'd').putInt(4 + body.length).put(body);+ return message.array();+ }++ /**+ * Reads until {@code readFromCopy} fails, and returns that failure.+ */+ private static SQLException readUntilTimeout(CopyOut copyOut) throws SQLException {+ long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(30);+ while (System.nanoTime() < deadline) {+ try {+ if (copyOut.readFromCopy() == null) {+ return fail("COPY completed before the reader ran out of buffered data");+ }+ } catch (SQLException e) {+ return e;+ }+ }+ return fail("readFromCopy kept returning data even though the proxy stopped forwarding");+ }
}
diff --git a/pgjdbc/src/test/java/org/postgresql/test/util/MessageStallProxyServer.java b/pgjdbc/src/test/java/org/postgresql/test/util/MessageStallProxyServer.javanew file mode 100644index 0000000000..8463276e31--- /dev/null+++ b/pgjdbc/src/test/java/org/postgresql/test/util/MessageStallProxyServer.java@@ -0,0 +1,248 @@+/*+ * Copyright (c) 2026, PostgreSQL Global Development Group+ * See the LICENSE file in the project root for more information.+ */++package org.postgresql.test.util;++import org.checkerframework.checker.nullness.qual.Nullable;++import java.io.Closeable;+import java.io.IOException;+import java.io.InputStream;+import java.io.OutputStream;+import java.net.ServerSocket;+import java.net.Socket;+import java.net.SocketTimeoutException;+import java.util.List;+import java.util.concurrent.CopyOnWriteArrayList;+import java.util.concurrent.atomic.AtomicInteger;+import java.util.concurrent.atomic.AtomicLong;+import java.util.concurrent.atomic.AtomicReference;++/**+ * Proxy server that can hold a backend message open, so that the driver sits partway through one+ * for as long as the test wants.+ *+ * <p>It forwards both directions untouched until one of the two stalls arms it.+ * {@link #stallInsideNextMessage(long)} delivers the next message in two parts, the type byte on+ * its own and then, after a pause, the length and the body: the driver has consumed the type byte+ * and is waiting for the rest, which is where a read deadline of its own does damage, since the+ * bytes it already took are not in the stream any more.+ * {@link #stallBeforeNextMessage(long)} pauses ahead of the type byte instead, leaving the driver+ * waiting with nothing consumed — the same wait, at a point it can still come back from.+ *+ * <p>The proxy reads the v3 framing to find that point, so the connection through it has to be+ * plain: pass {@code sslmode=disable} and {@code gssEncMode=disable}, or the traffic is encrypted+ * and the framing is not there to be found.+ */+public class MessageStallProxyServer implements Closeable {+ private final ServerSocket serverSock;+ private final List<Socket> sockets = new CopyOnWriteArrayList<>();+ private final AtomicLong stallMillis = new AtomicLong();+ private final AtomicLong boundaryStallMillis = new AtomicLong();+ private final AtomicLong typeStallMillis = new AtomicLong();+ private final AtomicInteger stallBeforeType = new AtomicInteger();+ private final AtomicReference<byte @Nullable []> injection = new AtomicReference<>();+ private volatile boolean keepRunning = true;++ public MessageStallProxyServer(String destHost, int destPort) throws IOException {+ this.serverSock = new ServerSocket(0);+ this.serverSock.setSoTimeout(100);+ doAsync(() -> {+ while (keepRunning) {+ try {+ Socket client = serverSock.accept();+ sockets.add(client);+ Socket server = new Socket(destHost, destPort);+ sockets.add(server);+ doAsync(() -> forward(client, server));+ doAsync(() -> forwardBackendMessages(server, client));+ } catch (SocketTimeoutException ignore) {+ // the accept timeout is what lets this loop notice close()+ } catch (IOException e) {+ return;+ }+ }+ });+ }++ public int getServerPort() {+ return serverSock.getLocalPort();+ }++ /**+ * Splits the next backend message, delivering its type byte and then pausing before the rest.+ * Applies to one message; call it again to arm the proxy for another.+ *+ * @param millis how long to hold the message open+ */+ public void stallInsideNextMessage(long millis) {+ stallMillis.set(millis);+ }++ /**+ * Pauses before the next backend message instead of inside it, so the driver waits with nothing+ * consumed. Forwarding whole messages is what makes this a boundary: the driver's buffer can only+ * end where a message does.+ *+ * @param millis how long to wait before starting the next message+ */+ public void stallBeforeNextMessage(long millis) {+ boundaryStallMillis.set(millis);+ }++ /**+ * Pauses before the next backend message of the given type, leaving the driver waiting for that+ * one with everything ahead of it already delivered. Use it to hold back the message that ends+ * an exchange, such as the ReadyForQuery that follows an ErrorResponse.+ *+ * @param messageType the v3 type byte to hold back, for example {@code 'Z'}+ * @param millis how long to wait before sending it+ */+ public void stallBeforeMessageType(int messageType, long millis) {+ stallBeforeType.set(messageType);+ typeStallMillis.set(millis);+ }++ /**+ * Sends the bytes to the driver ahead of the next real backend message, so a test can put a+ * message on the wire that the server would not send. The driver reads it as if the backend had.+ *+ * @param raw a complete v3 message: the type byte, a self-inclusive four-byte length, the body+ */+ public void injectBeforeNextMessage(byte[] raw) {+ injection.set(raw.clone());+ }++ @Override+ public void close() {+ keepRunning = false;+ for (Socket socket : sockets) {+ try {+ socket.close();+ } catch (IOException ignore) {+ // closing the rest matters more than this one+ }+ }+ try {+ serverSock.close();+ } catch (IOException ignore) {+ // nothing left to do about it+ }+ }++ /** Copies one direction byte for byte, with no interest in what the bytes mean. */+ private void forward(Socket source, Socket dest) {+ try {+ InputStream in = source.getInputStream();+ OutputStream out = dest.getOutputStream();+ byte[] buffer = new byte[8192];+ int read;+ while (keepRunning && (read = in.read(buffer)) >= 0) {+ out.write(buffer, 0, read);+ out.flush();+ }+ } catch (IOException ignore) {+ // the test closes the sockets under us+ }+ }++ /**+ * Copies the backend's direction one message at a time, so that an armed stall lands after the+ * type byte and before the length.+ */+ private void forwardBackendMessages(Socket source, Socket dest) {+ try {+ InputStream in = source.getInputStream();+ OutputStream out = dest.getOutputStream();+ byte[] header = new byte[4];+ while (keepRunning) {+ byte[] inject = injection.getAndSet(null);+ if (inject != null) {+ out.write(inject);+ out.flush();+ }+ long boundaryStall = boundaryStallMillis.getAndSet(0);+ if (boundaryStall > 0) {+ Thread.sleep(boundaryStall);+ }+ int type = in.read();+ if (type < 0) {+ return;+ }+ if (type == stallBeforeType.get()) {+ long typeStall = typeStallMillis.getAndSet(0);+ if (typeStall > 0) {+ Thread.sleep(typeStall);+ }+ }+ out.write(type);+ out.flush();++ long stall = stallMillis.getAndSet(0);+ if (stall > 0) {+ Thread.sleep(stall);+ }++ if (!readFully(in, header, header.length)) {+ return;+ }+ out.write(header);+ // The length counts itself, so the body is what is left after the four bytes+ int length = (header[0] & 0xFF) << 24 | (header[1] & 0xFF) << 16+ | (header[2] & 0xFF) << 8 | header[3] & 0xFF;+ if (!copy(in, out, length - 4)) {+ return;+ }+ out.flush();+ }+ } catch (IOException ignore) {+ // the test closes the sockets under us+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ }+ }++ private static boolean readFully(InputStream in, byte[] target, int length) throws IOException {+ int done = 0;+ while (done < length) {+ int read = in.read(target, done, length - done);+ if (read < 0) {+ return false;+ }+ done += read;+ }+ return true;+ }++ private static boolean copy(InputStream in, OutputStream out, int length) throws IOException {+ byte[] buffer = new byte[Math.min(8192, Math.max(length, 1))];+ int left = length;+ while (left > 0) {+ int read = in.read(buffer, 0, Math.min(buffer.length, left));+ if (read < 0) {+ return false;+ }+ out.write(buffer, 0, read);+ left -= read;+ }+ return true;+ }++ private static void doAsync(RunnableWithException task) {+ Thread thread = new Thread(() -> {+ try {+ task.run();+ } catch (Exception ignore) {+ // a proxy thread that dies takes its connection down, which the test sees+ }+ });+ thread.setDaemon(true);+ thread.start();+ }++ private interface RunnableWithException {+ void run() throws Exception;+ }+}diff --git a/testkit/src/main/java/org/postgresql/test/TestUtil.java b/testkit/src/main/java/org/postgresql/test/TestUtil.javaindex 4e35b95187..4351c4ebe9 100644--- a/testkit/src/main/java/org/postgresql/test/TestUtil.java+++ b/testkit/src/main/java/org/postgresql/test/TestUtil.java@@ -328,6 +328,11 @@ public static Connection openPrivilegedDB(Consumer<? super Properties> props) th
}
public static Connection openReplicationConnection() throws Exception {
+ return openReplicationConnection(x -> { });+ }++ public static Connection openReplicationConnection(Consumer<? super Properties> props)+ throws Exception {
Properties properties = new Properties();
PGProperty.ASSUME_MIN_SERVER_VERSION.set(properties, "9.4");
PGProperty.PROTOCOL_VERSION.set(properties, "3");
@@ -337,6 +342,7 @@ public static Connection openReplicationConnection() throws Exception {
PGProperty.USER.set(properties, TestUtil.getPrivilegedUser());
PGProperty.PASSWORD.set(properties, TestUtil.getPrivilegedPassword());
PGProperty.OPTIONS.set(properties, "-c synchronous_commit=on");
+ props.accept(properties);
return TestUtil.openDB(properties);
}