DataStax driver reconnection policies
If a driver can’t connect to a node or it loses an existing connection, then the driver can schedule a periodic reconnection job based on the driver’s reconnection policy configuration.
Reconnection jobs stop when the target node is decommissioned, the reconnection attempts are exhausted, or the connection is reestablished.
Reconnection policy concepts
A driver’s reconnection policy can determine the following:
- Condition
-
The scenario that triggers the reconnection policy:
-
Existing connection: Attempt to reconnect after losing an established connection to a node. All drivers have a default reconnection policy for this condition.
-
Initial connection: When creating a driver session, allow the driver to reattempt the connection if the first attempt fails. The typical default behavior for this condition is to fail and abandon session creation. Some drivers offer reconnection settings specifically for this condition.
-
- Interval
-
How long the driver should wait between reconnection jobs.
Supported intervals vary by driver. For example:
-
Constant: The driver waits the same amount of time between each reconnection attempt.
-
Exponential: The driver waits exponentially longer between each reconnection attempt.
-
- Maximum attempts
-
The total number of times that the driver should attempt reconnection before abandoning the connection. This can be a cumulative duration or a fixed number of attempts (jobs).
For some drivers, this threshold can be much greater when repairing the control connection (administration) node compared to connection pool nodes.
Configure reconnection policies
All DataStax drivers offer built-in reconnection policies. Each driver has a default reconnection policy, and some drivers offer custom reconnection policies.
Generally, DataStax recommends the default reconnection policy. However, you must determine the ideal reconnection policy for your use case.
For specific policies and configuration options, see your driver’s documentation:
C/C++ driver reconnection policies
C# driver reconnection policies
In addition to exponential (default) and constant reconnection policies, the C# driver also offers a fixed reconnection policy.
Instead of setting a blanket interval for all attempts, the fixed reconnection policy allows you to set a unique interval for each reconnection attempt. For example, you can set the first two intervals to 60 seconds and the third to 1 hour.
For more information, see C# driver reconnection policy and Automatic failover.
GoCQL driver reconnection policies
The GoCQL driver’s default reconnection policy is ConstantReconnectionPolicy, which attempts to reconnect once every 60 seconds for a maximum of 3 attempts.
The driver also offers ExponentialReconnectionPolicy and ReconnectionPolicy.
Java driver reconnection policies
For the DataStax Java driver, use advanced.reconnection-policy to set the reconnection policy for connection pool nodes and the control connection (administration) node.
Available policies include exponential backoff (default), constant, and custom.
If you want the driver to try to reconnect during initial session creation, you can set advanced.reconnect-on-init to true.
Otherwise, session creation fails if the driver can’t connect.
For more information, see Java driver reconnection.
Node.js driver reconnection policies
For the DataStax Node.js driver, use the reconnection module, which has three classes:
-
ConstantReconnectionPolicy -
ExponentialReconnectionPolicy -
ReconnectionPolicy
For more information, see Node.js driver reconnection policy.
PHP driver reconnection policies
Python driver reconnection policies
The DataStax Python driver offers exponential, constant, and custom reconnection policies. For more information, see Reconnecting to dead hosts.
Ruby driver reconnection policies
Monitor node status changes
After establishing the initial connection to the cluster, drivers must monitor node statuses to know when nodes go down and when they come back up. There are two ways that drivers receive node status changes:
- Idle connection timeout
-
To prevent intermediate network devices, like routers and firewalls, from disconnecting a driver from a node, the Cassandra Native Protocol send
OPTIONSrequests to the cluster at constant intervals, also known as a heartbeat. If a node doesn’t respond to the heartbeat, the connection is considered idle and the node is marked as down. - Gossip
-
Gossip is a peer-to-peer communication protocol for exchanging location and state information between nodes. When a cluster reports node statuses through gossip, the driver listens for push notifications from the cluster. These push notifications are not dependent on the heartbeat interval.
If a node goes down, the driver does the following:
-
Removes the down node from the connection pool, and then schedules and runs reconnection jobs based on the reconnection policy.
For connection pool nodes, the driver attempts to reconnect to the same node. For control connection (administration) nodes, drivers attempt to connect to any available node.
-
During the reconnection process, the driver continues to send heartbeats and listen for push notifications for all nodes. This is because heartbeats are sent at constant intervals, and push notifications originate from the cluster, not the driver.
-
Once the node comes back up and reconnects, the driver cancels any remaining scheduled reconnection jobs. If the driver exhausts all reconnection attempts before the node comes back up, then the driver stops trying to reconnect to that node.
When repairing the control connection, if an attempted connection fails, then the driver attempts to connect to the next available node until the connection is restored or the driver has exhausted all available nodes.
-
The reconnection process ends, and the driver returns to the normal node monitoring state, listening for push notifications and sending heartbeats.
Most drivers provide ways to listen, fetch, and log node status changes and other metadata. For more information, see Inspect connected nodes and your driver’s documentation.
