Repair Cassandra clusters in Kubernetes
This topic describes repair options for Apache Cassandra® clusters that are deployed in Kubernetes.
For Cassandra clusters deployed and managed by Kubernetes Operator for Apache Cassandra in Kubernetes, use Cassandra Reaper™ to schedule and orchestrate repairs. This feature is applicable to Cassandra databases only.
Reaper support started with the Kubernetes Operator for Apache Cassandra v1.3.0 release (June 30, 2020). If you previously deployed the operator using earlier releases such as v1.2.0, then apply the latest YAML manifest to your Kubernetes (K8) cluster from a local machine where you have an established connection. For more, see Upgrade Kubernetes Operator for Apache Cassandra and related resources in Kubernetes.
Reaper improves the existing
nodetool repair process for Cassandra by:
Splitting repair jobs into smaller, tunable segments.
Handling back-pressure through monitoring running repairs and pending compactions.
Adding the ability to pause or cancel repairs and track progress precisely.
Reaper provides a REST API, a command-line tool, and a web UI. For instructions on how to build, install, and configure Reaper, see the Reaper documentation.
To enable Reaper for Kubernetes Operator for Apache Cassandra, update your datacenter configuration. You can edit a local copy and use
kubectl apply -f <filename>, or invoke an editor to perform the changes directly in the Cassandra cluster that is running in Kubernetes. For the latter option, from your local machine where you have already established a connection with your Kubernetes cluster, enter a command such as:
kubectl -n cass-operator edit cassdc dc1
When the editor starts, add the following YAML lines within the
spec:section, maintaining the YAML indentation:
spec: reaper: enabled:true
After applying or saving the edited datacenter definition, Kubernetes Operator for Apache Cassandra applies the updated configuration dynamically to the target Cassandra cluster.
To check the pod
Readystatus after a few minutes, run:
kubectl -n cass-operator get pods
NAME READY STATUS RESTARTS AGE cass-operator-78884f4f84-25fx5 1/1 Running 0 25d cluster1-dc1-default-sts-0 2/2 Running 0 21d cluster1-dc1-default-sts-1 2/2 Running 0 21d cluster1-dc1-default-sts-2 2/2 Running 0 2m48s
To check your YAML file, use a command such as:
kubectl -n cass-operator get pod/cluster1-dc1-default-sts-2 -o yaml
To view the log file following a YAML configuration update, use a command such as:
kubectl -n cass-operator logs cluster1-dc1-default-sts-2 server-system-logger
For DSE clusters deployed and managed by Kubernetes Operator for Apache Cassandra in Kubernetes, DataStax provides
NodeSync, a continuous background repair service that is declarative and self-orchestrating.
After deploying a DSE cluster in your Kubernetes environment, use
nodesync enable options on all new tables. The default setting is
For details, see
For information about upgrading Kubernetes Operator for Apache Cassandra and optionally using
forceUpgradeRacks in the datacenter configuration, see this topic.