Write request coordination

To propagate a write across a cluster, the coordinator node sends the write request to all replicas that own the row being written. As long as all replica nodes are available, they receive and process the write regardless of the client’s requested write consistency level.

In multiple datacenter deployments, the database optimizes write performance by choosing one coordinator node across all of the datacenters. To minimize cross-datacenter communication, the coordinator node forwards the write request to one replica in each of the other datacenters. The forwarded request includes a special tag to distribute the write to the other local replicas with respect to the consistency level and replication factor.

For more information about the internal write path, see Write request processing.

To respond to the client, the coordinator node waits for successful acknowledgements from the replicas based on the write consistency level. Success means data was written to the commit log and memtable. With the exception of ALL, the coordinator node doesn’t need acknowledgements from all replicas before responding to the client. As soon as the coordinator node receives the minimum number of acknowledgements, it responds to the client, even if the write is still processing or unacknowledged on some replicas.

For example, in a cluster with one datacenter and 6 nodes with a replication factor of 3, an incoming write is distributed to all three nodes that own the requested row. If the requested write consistency level is ONE, the coordinator node responds to the client after receiving a successful acknowledgement from any of the three replicas.

Write consistency levels that require acknowledgement across multiple datacenters inherently increase write latency and complexity by involving more nodes. For faster writes in multiple datacenter deployments, you can write at LOCAL_ONE or LOCAL_QUORUM, if appropriate for your use case. These consistency levels only require acknowledgement from the local datacenter nodes before responding to the client. Other write operations proceed without delaying the response to the client.

At consistency levels other than ALL, it is possible that some replicas can miss a write if they are down when the request is made. If a replica misses a write, the row is made consistent later using one of the built-in repair mechanisms: hinted handoff, read repair, or anti-entropy node repair.

If a write fails on a replica, the coordinator node might wait for other replicas to respond, depending on the failure and the number of additional replicas that are available.

If the coordinator cannot distribute the write to enough replicas to meet the requested consistency level, it returns an Unavailable exception and does not perform any writes. This occurs when too many nodes are down or unreachable by the coordinator node. For example, a write at ALL fails if any node is down.

If there are enough replicas available but the writes aren’t completed within the timeout window, the coordinator returns a WriteTimeout exception.

An OverloadedException error occurs when the coordinator node attempts to write too many hints for hinted handoff.

If a write outright fails on too many replicas, the response to the client depends on the cause of the failure. For more information, see Errors and exception handling in Cassandra drivers.