pg.ddx.io aws/aws-advanced-jdbc-wrapper GitHub issues and pull requestshelp / color / mirror / Atom feed
[aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover 8+ messages / 3 participants [nested] [flat]
* [aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-16 18:54 "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com> 0 siblings, 0 replies; 8+ messages in thread From: sergiyvamz (@sergiyvamz) @ 2026-11-16 18:54 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> ### Summary Fix the Blue/Green plugin getting stuck in POST after a completed switchover. ### Description After a completed switchover, `resetContextWhenCompleted()` starts fresh monitors, which can still read the finished deployment's POST or COMPLETED status for a few seconds. The fresh context adopted that phase, but it had never collected the start-of-switchover IPs and topology, so the DNS updates that end POST could never be observed. The switchover timer was only started in PREPARATION, so nothing ended it: the provider reported POST forever, logged no warning, and ignored both the deletion of the deployment and the creation of a new one. Seen on Aurora MySQL with 4.6.0 on about half of the application instances. - A context that never saw the deployment in CREATED now ignores a POST or COMPLETED status. It records the status hash but doesn't adopt the phase or keep the finished deployment's hosts and IPs, so it stays in idle monitoring until a new deployment is created. A context that saw CREATED still adopts POST, including when a fast switchover hides PREPARATION and IN_PROGRESS from it. - Start the switchover timer in every phase from PREPARATION onwards, so a context that first sees IN_PROGRESS is also bounded. It is a no-op once running. - When the timer expires, also handle COMPLETED, and reset the context on the unchanged-status path the same way the next status change would. This also stops `bgd.switchoverTimeout` repeating on every unchanged tick. - Unit tests for the ignored finished status, a context that saw CREATED, a switchover stalled in IN_PROGRESS, and the COMPLETED timeout branch. - A parameterized test that replays the whole switchover with the finished deployment's status staying readable for 0 ms, 3 s, or longer than the switchover timeout after the reset. On main the 3 s and longer cases stay in POST, which reproduces the split seen in the 4.6.0 case. ### Additional Reviewers ### By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. ^ permalink raw [nested|flat] 8+ messages in thread
* Re: [aws/aws-advanced-jdbc-wrapper] PR #2242 (approved): fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-16 22:23 "sophia-bq (@sophia-bq)" <noreply+sophia-bq@github.com> 6 siblings, 0 replies; 8+ messages in thread From: sophia-bq (@sophia-bq) @ 2026-11-16 22:23 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> ^ permalink raw [nested|flat] 8+ messages in thread
* Re: [aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-16 22:58 "aaron-congo (@aaron-congo)" <noreply+aaron-congo@github.com> 6 siblings, 0 replies; 8+ messages in thread From: aaron-congo (@aaron-congo) @ 2026-11-16 22:58 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> (on wrapper/src/main/java/software/amazon/jdbc/plugin/bluegreen/BlueGreenStatusProvider.java:637) When a fresh context first reads a lingering POST or COMPLETED, it now reports POST for the full `bgSwitchoverTimeoutMs` before this block resets it. That's for a switchover that already finished before this context existed: - Its monitors record the post-switchover IPs as their start IPs, so `allStartTopologyIpChanged` can't become true and nothing ends POST early. - When the timer expires, `logSwitchoverFinalSummary()` logs a "switchover COMPLETED" summary at INFO, together with the `bgd.switchoverTimeout` WARNING, for a switchover this context never actually saw. How long does the deployment keep reporting POST or COMPLETED after a switchover? If it can stay readable for longer than the timeout, the reset context reads it again and the cycle repeats: another 3 minutes of POST, a new pair of monitors, and another WARNING and summary, for as long as the status stays readable. Do you know from the 4.6.0 case how long it lasted? Either way, should a context that never saw the start of the switchover (no PREPARATION or IN_PROGRESS in `phaseTimeNano`) treat a first-seen POST or COMPLETED as already finished, and go back to idle monitoring right away without arming the timer? ^ permalink raw [nested|flat] 8+ messages in thread
* Re: [aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-16 23:02 "aaron-congo (@aaron-congo)" <noreply+aaron-congo@github.com> 6 siblings, 0 replies; 8+ messages in thread From: aaron-congo (@aaron-congo) @ 2026-11-16 23:02 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> (on wrapper/src/test/java/software/amazon/jdbc/plugin/bluegreen/BlueGreenStatusProviderTest.java:694) nit: should we consider adding the following tests? - a test that covers the new COMPLETED branch in `checkSwitchoverTimerExpiry()` - a test for a switchover observed from PREPARATION that stalls in IN_PROGRESS with an unchanged status. That's the case the timer originally existed for, and it now ends in `resetContext()` instead of keeping the forced COMPLETED status. ^ permalink raw [nested|flat] 8+ messages in thread
* Re: [aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-17 18:25 "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com> 6 siblings, 0 replies; 8+ messages in thread From: sergiyvamz (@sergiyvamz) @ 2026-11-17 18:25 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> (on wrapper/src/main/java/software/amazon/jdbc/plugin/bluegreen/BlueGreenStatusProvider.java:637) That makes sense. I think the following condition would be more stable: a context that never saw CREATED (latestStatusPhase == NOT_CREATED) ignores a POST or COMPLETED status. Thank you for bringing this. ^ permalink raw [nested|flat] 8+ messages in thread
* Re: [aws/aws-advanced-jdbc-wrapper] PR #2242 (approved): fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-17 20:28 "aaron-congo (@aaron-congo)" <noreply+aaron-congo@github.com> 6 siblings, 0 replies; 8+ messages in thread From: aaron-congo (@aaron-congo) @ 2026-11-17 20:28 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> ^ permalink raw [nested|flat] 8+ messages in thread
* Re: [aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-17 21:56 "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com> 6 siblings, 0 replies; 8+ messages in thread From: sergiyvamz (@sergiyvamz) @ 2026-11-17 21:56 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> @sergiyvamz merged this (commit b5dc35cb51f9). ^ permalink raw [nested|flat] 8+ messages in thread
* Re: [aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover @ 2026-11-17 21:56 "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com> 6 siblings, 0 replies; 8+ messages in thread From: sergiyvamz (@sergiyvamz) @ 2026-11-17 21:56 UTC (permalink / raw) To: aws/aws-advanced-jdbc-wrapper <noreply+aws-aws-advanced-jdbc-wrapper@github.com> @sergiyvamz closed this. ^ permalink raw [nested|flat] 8+ messages in thread
end of thread, other threads:[~2026-11-17 21:56 UTC | newest] Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-11-16 18:54 [aws/aws-advanced-jdbc-wrapper] PR #2242: fix: B/G plugin stuck in POST after a completed switchover "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com> 2026-11-16 22:23 ` Re: [aws/aws-advanced-jdbc-wrapper] PR #2242 (approved): fix: B/G plugin stuck in POST after a completed switchover "sophia-bq (@sophia-bq)" <noreply+sophia-bq@github.com> 2026-11-16 22:58 ` "aaron-congo (@aaron-congo)" <noreply+aaron-congo@github.com> 2026-11-16 23:02 ` "aaron-congo (@aaron-congo)" <noreply+aaron-congo@github.com> 2026-11-17 18:25 ` "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com> 2026-11-17 20:28 ` Re: [aws/aws-advanced-jdbc-wrapper] PR #2242 (approved): fix: B/G plugin stuck in POST after a completed switchover "aaron-congo (@aaron-congo)" <noreply+aaron-congo@github.com> 2026-11-17 21:56 ` "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com> 2026-11-17 21:56 ` "sergiyvamz (@sergiyvamz)" <noreply+sergiyvamz@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox