Repair Cassandra clusters in Kubernetes
This topic describes repair options for Apache Cassandra® clusters that are deployed in Kubernetes.
Use Cassandra Reaper to repair Cassandra clusters
For Cassandra clusters deployed and managed by Kubernetes Operator for Apache Cassandra in Kubernetes, use Cassandra Reaper™ to schedule and orchestrate repairs. This feature is applicable to Cassandra databases only.
|
Reaper support started with the Kubernetes Operator for Apache Cassandra v1.3.0 release (June 30, 2020). If you previously deployed the operator using earlier releases such as v1.2.0, then apply the latest YAML manifest to your Kubernetes (K8) cluster from a local machine where you have an established connection. For more, see Upgrade Kubernetes Operator for Apache Cassandra and related resources in Kubernetes. |
Reaper improves the existing nodetool repair process for Cassandra by:
-
Splitting repair jobs into smaller, tunable segments.
-
Handling back-pressure through monitoring running repairs and pending compactions.
-
Adding the ability to pause or cancel repairs and track progress precisely.
Reaper provides a REST API, a command-line tool, and a web UI. For instructions on how to build, install, and configure Reaper, see the Reaper documentation.
-
To enable Reaper for Kubernetes Operator for Apache Cassandra, update your datacenter configuration. You can edit a local copy and use
kubectl apply -f <filename>, or invoke an editor to perform the changes directly in the Cassandra cluster that is running in Kubernetes. For the latter option, from your local machine where you have already established a connection with your Kubernetes cluster, enter a command such as:kubectl -n cass-operator edit cassdc dc1
-
When the editor starts, add the following YAML lines within the
spec:section, maintaining the YAML indentation:spec: reaper: enabled:trueAfter applying or saving the edited datacenter definition, Kubernetes Operator for Apache Cassandra applies the updated configuration dynamically to the target Cassandra cluster.
-
To check the pod
Readystatus after a few minutes, run:kubectl -n cass-operator get pods
Sample Output:
NAME READY STATUS RESTARTS AGE cass-operator-78884f4f84-25fx5 1/1 Running 0 25d cluster1-dc1-default-sts-0 2/2 Running 0 21d cluster1-dc1-default-sts-1 2/2 Running 0 21d cluster1-dc1-default-sts-2 2/2 Running 0 2m48s
-
To check your YAML file, use a command such as:
kubectl -n cass-operator get pod/cluster1-dc1-default-sts-2 -o yaml
-
To view the log file following a YAML configuration update, use a command such as:
kubectl -n cass-operator logs cluster1-dc1-default-sts-2 server-system-logger
Use NodeSync to repair DSE clusters
For DSE clusters deployed and managed by Kubernetes Operator for Apache Cassandra in Kubernetes, DataStax provides NodeSync, a continuous background repair service that is declarative and self-orchestrating.
After deploying a DSE cluster in your Kubernetes environment, use nodesync enable options on all new tables. The default setting is true.
For details, see nodesync enable.
What’s next?
For information about upgrading Kubernetes Operator for Apache Cassandra and optionally using forceUpgradeRacks in the datacenter configuration, see this topic.
