Provision a database cluster

With Mission Control, you can provision a database cluster directly in the Mission Control UI or through the kubectl CLI. With the CLI, you must define a YAML file and submit it with kubectl.

Mission Control reconciles MissionControlCluster resources defined either through the UI or CLI against any currently deployed database instances. These definitions describe the desired state of your database clusters. Using your definitions, Mission Control automates the process of provisioning and configuring resources across your Control and Data Planes.

Prerequisites

  • Because Mission Control organizes clusters by projects, an existing project is required.

Provision and define a cluster

You can provision a database cluster using the Mission Control UI or CLI. If you want to deploy the Data API to your new cluster, you can select it as an option when you use the Mission Control UI.

DataStax recommends the CLI for users who are familiar with kubectl.

  • Mission Control UI

  • CLI

To provision a database cluster in Mission Control, do the following:

  1. In the Mission Control UI, select a project, and then click Create Cluster.

  2. Enter a meaningful, human-readable Cluster Name.

    The Cluster Name can be any string of characters, including international, alphanumeric, punctuation—dashes, spaces, underscores, upper or lower case.

    Cluster names are permanent. You can’t change them after you create the cluster. The name uniquely identifies the cluster across all projects and all environments to prevent a logical cluster from inadvertently joining another.

  3. Select a cluster Type.

  4. Enter a valid Version number, for example, 1.0.0.

  5. Leave the Image field blank. It is for advanced users.

  6. To define the Datacenter configuration, do the following:

    1. Enter a meaningful, human-readable Datacenter Name.

      Datacenter names are permanent. You can’t change them after you create the cluster. The datacenter name:

      • Must start with an alphanumeric character.

      • Must be a single word.

      • Can be any capitalization: upper, lower, or mixed-case.

      • Can include dashes and underscores.

      • Must not include spaces.

    2. Optional: Add the configuration property and its corresponding value in the Add cassandra.yaml Setting sub-section if you require a non-standard Cassandra configuration.

    3. Select the Data Plane Context where you want to deploy the cluster.

      By default, a database cluster will be deployed to the Control Plane. If a Data Plane is deployed on another Kubernetes cluster, you can choose to deploy the database cluster to that context. For more information, see the Planning guide.

    4. Enter a Rack Name for the first rack, for example, rack1.

      Rack names are permanent. You can’t change them after you create the cluster. The rack name:

      • Must start with an alphanumeric character.

      • Must be a single word.

      • Can be any capitalization: upper, lower, or mixed-case.

      • Can have dashes and underscores.

      • Must not include spaces.

      Database pods, or nodes, are scheduled using node affinity.

    5. Add the mission-control.datastax.com/role=database label to the rack configuration to ensure database pods are scheduled on database worker nodes only, not on platform worker nodes.

      • Label: mission-control.datastax.com/role

      • Value: database

        DataStax recommends a minimum of 3 nodes for production clusters to support replication in a datacenter for high availability. With 3 replicas in a datacenter, this configuration can tolerate a failure of one node when using a strong consistency of LOCAL_QUORUM.

        To add another rack, select Add Rack and configure it as you did in the previous steps. Make sure that you add the node affinity label.

    6. For Nodes Per Rack, allocate at least one database node to the rack.

    7. Optional: To create a multi-datacenter cluster, select Add Datacenter and configure it as above.

    8. For Resource Requests, enter the minimum available resources required. DataStax recommends that you allocate the following minimum amounts of memory:

      • 4 GB of RAM for development environments and 8 GB for nodes with Vector Search enabled.

      • 32 GB of RAM to production nodes and 64 GB for nodes with Vector Search enabled.

        For more information, see the HCD, DSE, or Cassandra capacity planning guide.

      • 500 GB of storage for production nodes.

    9. Enter the Storage Amount to allocate.

  7. To add Security Settings, do the following:

    1. Select the Require authentication to access cluster option.

    2. Enter a Superuser Name.

    3. Enter a Superuser Password.

    4. Select the Enable internode encryption option.

      The superuser role is required to provision other roles such as operators and service accounts.

      DataStax recommends that you secure your clusters by enabling authentication and internode encryption, especially for production environments.

  8. To configure Backup/Restore options, do the following:

    1. Optional: Enter a Prefix to use as the name of the top-level folder in the Backup bucket. If you don’t enter a value, Mission Control uses the cluster name.

    2. Select your Backup Configuration.

  9. Under Advanced Settings, for Heap Amount, enter an amount using the following as a guide:

    System memory

    Heap

    8 GB

    4 GB

    32 GB

    8-24 GB

    64 GB

    31 GB

  10. Optional: Under Data API, select Deploy the Data API to deploy the Data API.

  11. Select Create Cluster.

  12. Optional: To monitor the status, track the progress of the database pods provisioned by Mission Control:

    kubectl get pods -n mission-control

    The database pods will have names prefixed by the cluster name. Each node will go through a standard bootstrap sequence that will take approximately 2-3 minutes to complete. When the pods are operational and ready to accept client requests, each pod should show as 2/2 containers READY with a STATUS of Running.

  13. Optional: To inspect pods that aren’t ready, run the following command:

    kubectl describe pod -n mission-control POD_NAME

    Replace POD_NAME with the name of your pod.

Given that your Data Plane clusters have either the appropriate compute capacity or the capability to auto-scale, define a simple MissionControlCluster YAML file and invoke kubectl to create a running HCD, DSE, or Cassandra cluster.

Create a cluster by completing the following define and submit tasks. Review the automatic reconciliation workflow, and then monitor the reconciliation status with one kubectl command.

To define a new MissionControlCluster, start with creating a new YAML file that defines the topology and configuration for the new cluster. This file is an instance of a MissionControlCluster Kubernetes Custom Resource (CR), and it describes the target end-state for the cluster. What follows is a minimal example of a MissionControlCluster instance which creates a 3-node DSE cluster running version 6.9.2. Each node has 5 GB of storage available for data and requests 32 GB of RAM. For more information, see the HCD, DSE, or Cassandra capacity planning guide.

Sample (partial) MissionControlCluster manifest (object):

apiVersion: missioncontrol.datastax.com/v1beta2
kind: MissionControlCluster
metadata:
  name: test
  namespace: PROJECT_SLUG
spec:
  k8ssandra:
    cassandra:
      serverVersion: 6.9.2
      serverType: dse
      storageConfig:
        cassandraDataVolumeClaimSpec:
          storageClassName: standard
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 5Gi
      networking:
        hostNetwork: true
      datacenters:
        - metadata:
            name: dc1
          size: 3
          resources:
            requests:
              memory: 4Gi
...

Replace PROJECT_SLUG with the name of your project.

During Mission Control installation, the user interface shares various configuration settings to review or set. Use the reference catalog to guide your configuration decisions.

  1. Specify certain parameters in this CR file.

    1. The apiVersion and kind parameters indicate what type of resource this file represents. In this example, kind is a MissionControlCluster resource with an apiVersion of v1beta2.

    2. This YAML specification outlines metadata associated with this cluster. At a minimum you must specify a name for your cluster. This value is used in the cluster_name parameter of the cassandra.yaml file.

      Each name must be unique within a project. Submitting two clusters with the same name results in the first cluster being overwritten by the second.

    3. Other fields that may be present in the metadata include annotations or labels to provide additional ancillary data. At this time Mission Control does not use any of these fields, but they may be leveraged by automation within the your environment.

    4. After the metadata block review the spec, or specification, section. spec is the declaration of our target end-state for the cluster. Instead of describing the various steps to create a cluster you simply define what you want your cluster to look like and Mission Control handles reconciling existing or missing resources towards that end-state.

      See the MissionControlCluster reference for a list of options and their descriptions.

  2. The MissionControlCluster is saved to disk as test.missioncontrolcluster.yaml.

    Any filename is valid here. Using <resource_name>.<kind>.yaml allows you to easily differentiate multiple files in a given directory.

  3. Submit the MissionControlCluster YAML file to the Mission Control Control Plane Kubernetes cluster with kubectl.

    kubectl acts as a Kubernetes API client and handles calls to the Kubernetes API server. Advanced users may choose to leverage programmatic clients or GitOps tooling such as Flux instead of the imperative nature of the kubectl CLI.

    Submission of the object is handled with the kubectl apply sub-command.

    For example:

    kubectl apply -f MANIFEST_FILENAME.yaml

    Replace MANIFEST_FILENAME with the name of your file.

    This reads the file specified with the -f flag and submits it to the Control Plane Kubernetes cluster. Should an object exist within the Kubernetes API sharing the same namespace and name it is updated to match the local file. When no file exists a new entry is created. As the new MissionControlCluster object becomes available within the Kubernetes API, Mission Control detects the new resource and immediately begins reconciliation.

Automatic reconciliation steps for MissionControlCluster resources

The following steps describe the automated process for informational purposes only; no user intervention is required.

  1. Cluster-level operators detect a new MissionControlCluster custom resource through the Kubernetes API within the Control Plane.

  2. Cluster-level operators identify which Control Plane or Data Plane clusters should receive datacenters defined within the MissionControlCluster. In this example the east Data Plane cluster is specified so datacenter-level resources are created and reconciled there. If you omit the Data Plane identifier, resources are deployed within the Control Plane.

  3. Datacenter-level operators within the Data Plane clusters detect new datacenter-level custom resources (CRs) through the Kubernetes API.

  4. Datacenter-level operators generate and submit rack-level resources (StatefulSets) to their local Kubernetes API.

  5. Built-in Kubernetes reconciliation loops detect the new rack-level resources and begin creating pods and storage resources representing the underlying HCD, DSE, or Cassandra nodes.

  6. Status of resource creation rises to operators at the datacenter and cluster levels.

  7. When all pods are up and running the cluster-level operator signals the datacenter-level operators to begin bootstrap operations of DSE within the created and running pods.

  8. As pods come online their status is escalated and operations continue until all 3 nodes are up and running with services discoverable via the Kubernetes API.

Monitor bootstrap progress

Monitor the progress of the bootstrap to determine completion status or note any errors.

After you submit the MissionControlCluster custom resource (CR), the operator modifies the resource within the Kubernetes API by adding a status field to the top-level of the resource.

This status field provides valuable insight into the health of the MissionControlCluster as one or more operators detect definition changes. status indicates everything from the reconciliation phase to errors encountered while attempting to create storage.

  1. Run the following command to retrieve the descriptive status for the test MissionControlCluster object:

    kubectl describe mccluster/CLUSTER_NAME

    Replace CLUSTER_NAME with the name of your cluster.

    You can specify MissionControlCluster, missioncontrolcluster, or the short-form mccluster. Additionally, all of the names can be plural.
    Sample results 
    Name:         CLUSTER_NAME
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  missioncontrol.datastax.com/v1beta2
Kind:         MissionControlCluster
Metadata:
  Creation Timestamp:  2023-10-30T11:09:33Z
  Finalizers:
    missioncontrol.datastax.com/finalizer
  Generation:        1
  Resource Version:  105388250
  UID:               57e956f8-1f87-422f-a7f8-b9ec87b956c4
Spec:
  Create Issuer:  true
  Encryption:
    Internode Encryption:
      Certs:
        Cert Template:
          Issuer Ref:
            Name:
          Secret Name:
        Create Certs:   true
      Enabled:          true
  k8ssandra:
    Auth:  true
    Cassandra:
      Datacenters:
        Dse Workloads:
        Metadata:
          Name:  dc1
          Pods:
          Services:
            Additional Seed Service:
            All Pods Service:
            Dc Service:
            Node Port Service:
            Seed Service:
        Per Node Config Init Container Image:  <name>/yq:4
        Per Node Config Map Ref:
        Racks:
          Name:   rack1
        Size:     3
        Stopped:  false
      Metadata:
        Pods:
        Services:
          Additional Seed Service:
          All Pods Service:
          Dc Service:
          Node Port Service:
          Seed Service:
      Per Node Config Init Container Image:  <name>/yq:4
      Resources:
        Requests:
          Memory:      32Gi
      Server Type:     dse
      Server Version:  6.9.2
      Storage Config:
        Cassandra Data Volume Claim Spec:
          Access Modes:
            ReadWriteOnce
          Resources:
            Requests:
              Storage:         5Gi
          Storage Class Name:  standard
      Superuser Secret Ref:
        Name:          my-cluster-superuser
    Secrets Provider:  internal
Status:
  Conditions:
    Last Transition Time:  2023-10-30T14:08:08Z
    Message:
    Reason:                UpdatingIssuers
    Status:                False
    Type:                  UpdatingIssuers
    Last Transition Time:  2023-10-30T14:08:08Z
    Message:
    Reason:                UpdatingCertificates
    Status:                False
    Type:                  UpdatingCertificates
    Last Transition Time:  2023-10-30T14:08:08Z
    Message:
    Reason:                UpdatingReplicatedSecrets
    Status:                False
    Type:                  UpdatingReplicatedSecrets
    Last Transition Time:  2023-10-30T11:16:38Z
    Message:
    Reason:                UpdatingCluster
    Status:                False
    Type:                  UpdatingCluster
    Last Transition Time:  2023-10-30T14:08:08Z
    Message:               Ready
    Reason:                Ready
    Status:                True
    Type:                  Ready
Events:                    <none>

  2. Access operator logs to discover more detail:

    kubectl logs -n mission-control POD_NAME

    Replace POD_NAME with the name of your pod, for example: mission-control-controller.

    The StatefulSet controller is one of the core Kubernetes controllers that create the pods. The number of pods per StatefulSet is calculated by dividing the number of nodes in the datacenter by the number of racks.

    For example, a 3 node cluster with 3 racks has 1 pod per StatefulSet, and a 9 node cluster with 3 racks has 3 pods per StatefulSet.

Next steps

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com