feat: try every address a contact point resolves to (DRIVER-201) by nikagra · Pull Request #1074 · scylladb/java-driver

nikagra · 2026-09-09T22:04:49Z

A contact point given as a hostname was one connection attempt, to whatever address Netty's resolver returned first: a dead first record failed CqlSession.build() (DRIVER-201), and the reconnection fallback from #1065 re-tried one address per round.

When the control connection reaches an unresolved contact point, at startup or through the fallback, the name is resolved to all of its addresses (ChannelFactory.resolveAll, feat: add resolveAll through Netty's configured resolver (DRIVER-201) #1073) and deduplicated. The address the resolver returns first is tried first — that is where a connect would have gone on its own, so expanding a name can never do worse than not expanding it — and up to advanced.connection.max-candidate-addresses - 1 of the others follow in random order (5 addresses in all by default). The cap bounds what one contact point costs a round; shuffling the rest makes a capped set that missed the live nodes a different set next round.
Each address is tried as its own temporary node named cluster.example.com/10.0.0.1:9042, ahead of the rest of the plan. AllNodesFailedException carries one entry per address; TLS and authentication keep seeing the configured name.
Nothing else expands: IP literals, resolved contact points, custom EndPoints and identified nodes are tried as before. A resolver failure or an unusable answer falls back to today's single connect.
The node reached through a contact point is registered under the labelled address (resolve() resolved, toString()/node tag gain /ip, prefix, TLS and auth name unchanged; its pool stops re-resolving the name). That is the labelled form of 4.x: the node reached through a contact-point hostname is registered under that hostname, which can later resolve to another node #1072, documented in the upgrade guide.

Verified: ControlConnectionTest +15 (9 red against the code they guard, 6 pin unchanged paths), full core suite (4102) green on JDK 11, docs build; MockResolverIT against ScyllaDB 2026.1.9 including the dead-first-record case. Not covered locally: the Cassandra CCM lanes.

Sits on scylla-4.x directly, one commit, now that #1073 has merged.

Fixes #356
Fixes #989
Fixes #1072
Refs: #890

🤖 Generated with Claude Code

coderabbitai · 2026-09-09T22:04:55Z

Warning

Review limit reached

Next included review available in 7 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: QUIET

Plan: Advanced

Run ID: 5a85b8c0-0294-4ba9-9c02-af916fdf7f40

📥 Commits

Reviewing files that changed from the base of the PR and between d954342 and 81e216d.

📒 Files selected for processing (13)

core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
core/src/main/java/com/datastax/oss/driver/internal/core/channel/ChannelFactory.java
core/src/main/java/com/datastax/oss/driver/internal/core/context/NettyOptions.java
core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java
core/src/main/resources/reference.conf
core/src/test/java/com/datastax/oss/driver/internal/core/channel/ChannelFactoryConnectTest.java
core/src/test/java/com/datastax/oss/driver/internal/core/control/ControlConnectionTest.java
core/src/test/java/com/datastax/oss/driver/internal/core/control/ControlConnectionTestBase.java
integration-tests/src/test/java/com/datastax/oss/driver/core/resolver/MockResolverIT.java
manual/core/address_resolution/README.md
upgrade_guide/README.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

A contact point given as a hostname was one connect: whatever address Netty's resolver returned first. A dead first record failed build(), and the fallback re-tried one address per round. When the control connection reaches such a contact point it now resolves the name to all its addresses (ChannelFactory.resolveAll) and tries each as its own temporary node labelled hostname/ip:port, so AllNodesFailedException names every address. That first address still goes first, so expanding can never do worse; the rest follow shuffled, capped at advanced.connection.max-candidate-addresses. The node reached is registered under the address it answered on. Fixes scylladb#989 Fixes scylladb#1072 Refs: scylladb#890 Co-Authored-By: Claude Fable 5.1 <[email protected]> Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

qodo-scylladb · 2026-09-16T15:35:44Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

🔴 High

1. ~~Startup hangs after hook failure~~ ✓ Resolved 🐞 Bug ☼ Reliability

Description

expandContactPoint catches exceptions escaping tryCandidates and only logs them, without
invoking either completion callback to settle the connection round. When candidate dialing throws
synchronously, including from NettyOptions.afterBootstrapInitialized, both initial session
construction and later reconnection can remain pending indefinitely, with reconnection stuck in its
in-progress state instead of scheduling another attempt.

Code

core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java[R662-667]

+                } catch (Exception e) {
+                  // Logged, never retried: this stage is dropped, so a throw would go unseen, and
+                  // dialling again here would repeat an attempt the round has already made.
+                  Loggers.warnWithException(
+                      LOG, "[{}] Unexpected error while expanding {}", logPrefix, contactPoint, e);
+                }

Relevance

●●● Strong
Recent accepted precedent requires every resolver callback path to complete its stage; this catch
leaves startup and reconnection pending.
PR-#1073

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The expansion callback invokes tryCandidates but catches and logs any escaping exception without
settling the round. tryCandidates synchronously enters connect, and ChannelFactory.connect
invokes the customizable afterBootstrapInitialized hook outside an exception-to-future guard;
because initialization and reconnection rely on the supplied callbacks to settle their futures, such
an exception leaves startup pending or keeps the reconnection scheduler in its in-progress state.
core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java[651-669]
core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java[677-715]
core/src/main/java/com/datastax/oss/driver/internal/core/channel/ChannelFactory.java[436-446]
core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java[361-407]
core/src/main/java/com/datastax/oss/driver/internal/core/util/concurrent/Reconnection.java[210-235]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`expandContactPoint` logs exceptions escaping `tryCandidates` but abandons the connection round without calling `onSuccess` or `onFailure`. A synchronous failure during candidate `ChannelFactory.connect`, including a throwing `NettyOptions.afterBootstrapInitialized` hook, can therefore leave startup or reconnection pending forever.

## Fix Focus Areas
- core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java[651-669]
- core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java[677-715]
- core/src/main/java/com/datastax/oss/driver/internal/core/channel/ChannelFactory.java[436-446]

## Recommended Fix
Ensure every exception escaping the asynchronous expansion callback settles the current connection round instead of only being logged. Prefer converting synchronous exceptions from each candidate connection into an exceptionally completed stage in `ChannelFactory`, allowing the normal failed-node path to record the attempt and continue through the remaining candidates; otherwise route the caught exception to `onFailure` so the round always terminates. Add startup and reconnection tests using a `NettyOptions.afterBootstrapInitialized` hook that throws synchronously.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Context sources

✅ Cross-repo context — repo relationships

Explored: repo: scylladb/scylla-java-driver-matrix (sha: 7d16ad3c)

Explored: repo: scylladb/kafka-connect-scylladb (sha: 3d543e17)

Explored: repo: scylladb/spark-scylladb-connector (sha: 064dcf69)

Explored: repo: scylladb/scylla-ccm (sha: 14b03bc1)

Explored: repo: scylladb/scylladb (sha: e5b03ded)

Explored: repo: scylladb/cassandra-stress (sha: 96c0c5d9)

✅ REVIEW.md

Review mode: 🧠 Deep: This broad, behavior-changing control-connection feature spans substantial asynchronous networking logic, configuration, fallback paths, and endpoint semantics, creating multiple independent defect opportunities that benefit from redundant review.

Tip of the day

💡 Did you know, you can group findings by type and pick your Finding display, from Minimal to Full

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Powered by Qodo

A bootstrap hook that throws left an expansion round unsettled: the callback logged the escape and returned, so init() waited for ever and a Reconnection stayed stuck on an attempt that never completed. ChannelFactory.connect now fails its future rather than throwing at its caller, and the expansion callback ends the round if anything still escapes. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

The candidate order rested on getByName being getAllByName[0], which reads as an API coincidence and invited the question of what happens when a resolver disagrees. Name what actually ranks the list: getaddrinfo sorts destinations per RFC 6724 and the JVM regroups them once, so element 0 is the platform's choice for this client. A resolver that randomises instead randomises resolveAll too, so pinning its first answer costs nothing. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

scylladb#1130 was written against scylla-4.x. At 4.19.0 a hostname contact point is tried at its first address only (scylladb#1074 is newer), and there is no subnet translator, so reword both and drop the links to sections this branch lacks. Backport the fixed proxy hostname section the new pages point to. Use the eval_rst toctree this branch is built with, and name the client_routes/clientroutes spellings for search. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

scylladb#1130 was written against scylla-4.x, where a hostname contact point expands to several addresses (scylladb#1074). 4.19.2 tries only the first address a lookup returns, so say that and drop the links. Carry the sibling ports' fixes: Cloud TLS on 9142 with the cluster CA, rpc_address in the local query, qualified client routes pointers, and DNS lookups on the admin threads. Use the eval_rst toctree, and name the client_routes/clientroutes spellings for search. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

scylladb#1130 was written against scylla-4.x. At 4.19.0 a hostname contact point is tried at its first address only (scylladb#1074 is newer), and there is no subnet translator, so reword both and drop the links to sections this branch lacks. Backport the fixed proxy hostname section, and carry the sibling ports' fixes: Cloud TLS on 9142 with the cluster CA, rpc_address in the local query, qualified client routes pointers, and DNS lookups on the admin threads. Use the eval_rst toctree, and name the client_routes/clientroutes spellings for search. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

scylladb#1130 was written against scylla-4.x, where a hostname contact point expands to several addresses (scylladb#1074). 4.19.2 tries only the first address a lookup returns, so say that and drop the links. Carry the sibling ports' fixes: Cloud TLS on 9142 with the cluster CA, rpc_address in the local query, qualified client routes pointers, and DNS lookups on the admin threads. Use the eval_rst toctree, and name the client_routes/clientroutes spellings for search. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

scylladb#1130 was written against scylla-4.x. At 4.19.0 a hostname contact point is tried at its first address only (scylladb#1074 is newer), and there is no subnet translator, so reword both and drop the links to sections this branch lacks. Backport the fixed proxy hostname section, and carry the sibling ports' fixes: Cloud TLS on 9142 with the cluster CA, rpc_address in the local query, qualified client routes pointers, and DNS lookups on the admin threads. Use the eval_rst toctree, and name the client_routes/clientroutes spellings for search. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: add a connectivity section to the core manual (#1130) How an application reaches a cluster had no home in the manual. Client routes were an H3 inside Address resolution, so the site had no URL and no search result of its own for them, and nothing covered VPC peering, Transit Gateway or direct connections at all. Add manual/core/connectivity/ with an index routing each kind of network to what the driver needs, a page on the cases that need no translation, and the client routes content moved out of Address resolution. The old heading stays as a pointer so the 3.x deep link keeps resolving. Co-authored-by: Claude Opus 5 (1M context) <[email protected]> (cherry picked from commit 9167ca1) * docs: fit the connectivity section to 4.19.0 #1130 was written against scylla-4.x. At 4.19.0 a hostname contact point is tried at its first address only (#1074 is newer), and there is no subnet translator, so reword both and drop the links to sections this branch lacks. Backport the fixed proxy hostname section, and carry the sibling ports' fixes: Cloud TLS on 9142 with the cluster CA, rpc_address in the local query, qualified client routes pointers, and DNS lookups on the admin threads. Use the eval_rst toctree, and name the client_routes/clientroutes spellings for search. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs: enable TLS in the ScyllaDB Cloud connection example The example connected to 9142, the TLS port, without enabling TLS, and the truststore setup came only afterwards as a HOCON block, so the code copied as shown fails to connect. Configure the engine factory in the builder and keep the HOCON block as the application.conf alternative. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]>

nikagra force-pushed the s2/07-expand-contact-points branch from 650e41c to aee0480 Compare September 9, 2026 22:12

nikagra mentioned this pull request Sep 9, 2026

feat: fall back to contact points on control reconnect (DRIVER-201) #1065

Merged

nikagra force-pushed the s2/07-expand-contact-points branch 3 times, most recently from 5d325b7 to 9dd3b25 Compare September 10, 2026 23:03

nikagra mentioned this pull request Sep 11, 2026

feat: add resolveAll through Netty's configured resolver (DRIVER-201) #1073

Merged

nikagra force-pushed the s2/07-expand-contact-points branch 3 times, most recently from dc69c16 to bdb3844 Compare September 16, 2026 13:54

nikagra force-pushed the s2/07-expand-contact-points branch from bdb3844 to 2589b9f Compare September 16, 2026 13:58

nikagra marked this pull request as ready for review September 16, 2026 15:30

nikagra requested a review from dkropachev September 16, 2026 15:30

qodo-scylladb Bot reviewed Sep 16, 2026

View reviewed changes

Comment thread core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java Outdated

dkropachev reviewed Sep 18, 2026

View reviewed changes

Comment thread core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java

nikagra requested a review from dkropachev September 18, 2026 20:19

dkropachev approved these changes Sep 19, 2026

View reviewed changes

dkropachev merged commit 87eb224 into scylladb:scylla-4.x Sep 19, 2026
28 checks passed

nikagra deleted the s2/07-expand-contact-points branch September 21, 2026 10:40

This was referenced Sep 24, 2026

docs: add the connectivity section to the 4.19.0 manual #1143

Merged

docs: add the connectivity section to the 4.19.2 manual #1149

Closed

Navigation Menu

Repository navigation

feat: try every address a contact point resolves to (DRIVER-201) - #1074

feat: try every address a contact point resolves to (DRIVER-201)#1074
dkropachev merged 3 commits into
scylladb:scylla-4.xfrom
nikagra:s2/07-expand-contact-points

nikagra commented Sep 9, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading

Review limit reached

Uh oh!

qodo-scylladb Bot commented Sep 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nikagra commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

qodo-scylladb Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nikagra commented Sep 9, 2026 •

edited

Loading

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading

qodo-scylladb Bot commented Sep 16, 2026 •

edited

Loading