Deploying Configuration Changes to a DataStax Enterprise Cluster

DataStax Mission Control is current in Private Preview. It is subject to the beta agreement executed between you and DataStax. DataStax Mission Control is not intended for production use, has not been certified for production workloads, and might contain bugs and other functional issues. There is no guarantee that DataStax Mission Control will ever become generally available. DataStax Mission Control is provided on an “AS IS” basis, without warranty or indemnity of any kind.

If you are interested in trying out DataStax Mission Control please contact your DataStax account team.

To make a configuration change, modify and submit the DSECluster manifest (object).

This section partially covers configuration options and their values. An example workflow and specific guidance for key configuration topics are provided.

K8ssandra enables authentication and authorization by default. It uses the Cassandra default PasswordAuthenticator and CassandraAuthorizer functionality.

Explore topic-specific configuration:

Performance impact

Increasing the heap size parameter that is used by all objects in the cluster may increase the time it takes to find, process, and collect objects included in the heap.

Timeout settings are adjusted to increase throughput and maintain near ideal requests per second, yielding better performance.

Prerequisites

The kubectl CLI tool.
Kubeconfig file` or context pointing to a Control Plane Kubernetes cluster.

Example of a configuration change

This example uses demo-dse.yaml as the filename of the DSECluster manifest. In this file, increase the heap size and adjust both the write_request_timeout_in_ms and read_request_timeout_in_ms settings. Then deploy these configuration changes to an Control Plane Kubernetes cluster.

In this example a Kubernetes cluster exists with a certain number of nodes. The amount of memory allocated to the Java Virtual Machine (JVM) needs to increase. This memory is shared by instances of all the objects created during the runtime of an application.

The heap value is determined by the amount of memory available to the container and should be set to, at a maximum, half of the requested memory. See heap size parameter guidelines and recommendations.

Workflow of user and operators

User submits a modified DSECluster to the Control Plane cluster.
Cluster-level operator detects change, modifies datacenter-level resource (CassandraDatacenter).
Datacenter-level operator detects change, modifies rack-level resource.
Datacenter-level operator terminates instance in rack.
Datacenter-level operator bootstraps replacement node.
Datacenter-level operator repeats steps 3-5 for all racks in the datacenter.
Cluster-level operator repeats steps 2-6 for all datacenters in the cluster.

Procedure

Two Kubernetes clusters exist for this example, a Control Plane cluster and a Data Plane cluster.

Modify the demo-dse.yaml file to define the DSECluster config specification. Set updated heap parameters for the jvmOptions key. Increase both the initial and the maximum heap sizes from 1Gi to 2Gi. Further modify the write_request_timeout_in_ms and read_request_timeout_in_ms properties for the cassandraYaml key as follows:
```
...
config:
    cassandraYaml:
      write_request_timeout_in_ms: 5000
      read_request_timeout_in_ms: 10000
    jvmOptions:
      heap_initial_size: 2Gi
      heap_max_size: 2Gi
```
The DSECluster config: specification supports 3 keys:

cassandraYaml

dseYaml

jvmOptions
Submit the modified DSECluster manifest, demo-dse.yaml, to the Control Plane Kubernetes cluster from a terminal window.
```
kubectl apply -f demo-dse.cassandratask.yaml
```
Monitor the progress of the task from the Data Plane cluster by submitting the following command:
```
kubectl get cassandradatacenter dc1 -o yaml
```
Sample output:
```
status:
  cassandraOperatorProgress: Updating
  conditions:
  - lastTransitionTime: "2022-10-19T19:08:09Z"
    message: ""
    reason: ""
    status: "True"
    type: Healthy
  - lastTransitionTime: "2022-10-19T19:08:10Z"
    message: ""
    reason: ""
    status: "False"
    type: Stopped
...
- lastTransitionTime: "2022-10-19T19:10:43Z"
    message: ""
    reason: ""
    status: "True"
    type: Updating
...
```
The status field is set to "False" when the Updating operation completes.

Changes are applied in the same order in which the racks are declared in the DSECluster manifest.

The configuration changes are applied in a rolling fashion one rack at a time and one DSE node at a time within each rack. The changes also trigger a rolling restart as part of the update. The configuration updates take effect as soon as the update is finished.

Submit the following command to check the cassandra.yaml file and verify that the update is complete:

kubectl exec -it demo-dc1-rack1-sts-0 -c cassandra -- cat resources/cassandra/conf/cassandra.yaml | grep ^'write\|read'_request_timeout_in_ms

Sample output:

write_request_timeout_in_ms: 5000
read_request_timeout_in_ms: 10000

Submit the following command to check the jvm-server.options file and verify that it, too, reflects the update:
```
kubectl exec -it demo-dc1-rack1-sts-0 -c cassandra -- cat resources/cassandra/conf/jvm-server.options | grep ^'-Xmx\|-Xms'
```
Sample output:
```
-Xms2147483648
-Xmx2147483648
```
The values are shown in bytes.

DataStax Mission Control operators perform the conversion from 2Gi to the appropriate number of bytes.

What’s next

Explore configuration examples and keys for common DataStax Enterprise configuration files:
- DSE namespace contains classes for accessing and managing data from DSE clusters.
- Cassandra.yaml
- DSE.yaml
- jvm-server.options