Provision a database cluster

With Mission Control, you can provision a database cluster directly in the Mission Control UI or through the kubectl CLI. With the CLI, you must define a YAML file and submit it with kubectl.

Mission Control reconciles MissionControlCluster resources defined either through the UI or CLI against any currently deployed database instances. These definitions describe the desired state of your database clusters. Using your definitions, Mission Control automates the process of provisioning and configuring resources across your Control and Data Planes.

Prerequisites

  • Because Mission Control organizes clusters by projects, an existing project is required.

Provision and define a cluster

You can provision a database cluster using the Mission Control UI or CLI. The UI offers simple mode and expert mode. DataStax recommends simple mode for users who are new to Kubernetes or Mission Control. It allows you to provision a cluster with a few clicks. Expert mode is for users who are familiar with the Kubernetes API and YAML configuration and allows you to define a cluster with more specific settings.

If you want to deploy the Data API to your new cluster, you can select it as an option when you use the Mission Control UI.

DataStax recommends the CLI for users who are familiar with kubectl.

  • UI simple mode

  • UI expert mode

  • CLI

To provision a database cluster in Mission Control using simple mode, do the following:

  1. In the Mission Control UI, select a project, and then click Create Cluster.

  2. Enter a meaningful, human-readable Cluster Name.

    The Cluster Name can be any string of characters, including international, alphanumeric, punctuation—dashes, spaces, underscores, upper or lower case.

    Cluster names are permanent. You can’t change them after you create the cluster. The name uniquely identifies the cluster across all projects and all environments to prevent a logical cluster from inadvertently joining another.

  3. Select a cluster Type.

  4. Enter a valid Version number, for example, 1.0.0.

  5. Leave the Image field blank. It is for advanced users.

  6. To define the Datacenter configuration, do the following:

    1. Enter a meaningful, human-readable Datacenter Name.

      Datacenter names are permanent. You can’t change them after you create the cluster. The datacenter name:

      • Must start with an alphanumeric character.

      • Must be a single word.

      • Can be any capitalization: upper, lower, or mixed-case.

      • Can include dashes and underscores.

      • Must not include spaces.

    2. Optional: Add the configuration property and its corresponding value in the Add cassandra.yaml Setting sub-section if you require a non-standard Cassandra configuration.

    3. Select the Data Plane Context where you want to deploy the cluster.

      By default, a database cluster will be deployed to the Control Plane. If a Data Plane is deployed on another Kubernetes cluster, you can choose to deploy the database cluster to that context. For more information, see the Planning guide.

    4. Enter a Rack Name for the first rack, for example, rack1.

      Rack names are permanent. You can’t change them after you create the cluster. The rack name:

      • Must start with an alphanumeric character.

      • Must be a single word.

      • Can be any capitalization: upper, lower, or mixed-case.

      • Can have dashes and underscores.

      • Must not include spaces.

      Database pods, or nodes, are scheduled using node affinity.

    5. Add the mission-control.datastax.com/role=database label to the rack configuration to ensure database pods are scheduled on database worker nodes only, not on platform worker nodes.

      • Label: mission-control.datastax.com/role

      • Value: database

        DataStax recommends a minimum of 3 nodes for production clusters to support replication in a datacenter for high availability. With 3 replicas in a datacenter, this configuration can tolerate a failure of one node when using a strong consistency of LOCAL_QUORUM.

        To add another rack, select Add Rack and configure it as you did in the previous steps. Make sure that you add the node affinity label.

    6. For Nodes Per Rack, allocate at least one database node to the rack.

    7. Optional: To create a multi-datacenter cluster, select Add Datacenter and configure it as above.

    8. For Resource Requests, enter the minimum available resources required. DataStax recommends that you allocate the following minimum amounts of memory:

      • 4 GB of RAM for development environments and 8 GB for nodes with Vector Search enabled.

      • 32 GB of RAM to production nodes and 64 GB for nodes with Vector Search enabled.

        For more information, see the HCD, DSE, or Cassandra capacity planning guide.

      • 500 GB of storage for production nodes.

    9. Enter the Storage Amount to allocate.

  7. To add Security Settings, do the following:

    1. Select the Require authentication to access cluster option.

    2. Enter a Superuser Name.

    3. Enter a Superuser Password.

    4. Select the Enable internode encryption option.

      The superuser role is required to provision other roles such as operators and service accounts.

      DataStax recommends that you secure your clusters by enabling authentication and internode encryption, especially for production environments.

  8. To configure Backup/Restore options, do the following:

    1. Optional: Enter a Prefix to use as the name of the top-level folder in the Backup bucket. If you don’t enter a value, Mission Control uses the cluster name.

    2. Select your Backup Configuration.

  9. Under Advanced Settings, for Heap Amount, enter an amount using the following as a guide:

    System memory

    Heap

    8 GB

    4 GB

    32 GB

    8-24 GB

    64 GB

    31 GB

  10. Optional: Under Data API, select Deploy the Data API to deploy the Data API.

  11. Select Create Cluster.

  12. Optional: To monitor the status, track the progress of the database pods provisioned by Mission Control:

    kubectl get pods -n mission-control

    The database pods will have names prefixed by the cluster name. Each node will go through a standard bootstrap sequence that will take approximately 2-3 minutes to complete. When the pods are operational and ready to accept client requests, each pod should show as 2/2 containers READY with a STATUS of Running.

  13. Optional: To inspect pods that aren’t ready, run the following command:

    kubectl describe pod -n mission-control POD_NAME

    Replace POD_NAME with the name of your pod.

DataStax recommends expert mode only for users who are familiar with the Kubernetes API and YAML configuration.

After you create or update a cluster in expert mode, you cannot edit it in simple mode.

For custom resource definitions (CRDs), see Mission Control Custom Resource Definition (CRD) reference.

To provision a database cluster in Mission Control using expert mode, do the following:

  1. In the Mission Control UI, select a project, and then click Create Cluster.

  2. Click Expert. The Create Cluster page displays YAML configuration options.

  3. Edit the YAML configuration to define the cluster.

  4. Click Create Cluster.

When you use expert mode to copy your YAML definition and create a new cluster on another installation, you must **omit the metadata.resourceVersion property. If you include this property, you can’t make updates to the new cluster with kubectl.

Given that your Data Plane clusters have either the appropriate compute capacity or the capability to auto-scale, define a simple MissionControlCluster YAML file and invoke kubectl to create a running HCD, DSE, or Cassandra cluster.

Create a cluster by completing the following define and submit tasks. Review the automatic reconciliation workflow, and then monitor the reconciliation status with one kubectl command.

To define a new MissionControlCluster, start with creating a new YAML file that defines the topology and configuration for the new cluster. This file is an instance of a MissionControlCluster Kubernetes Custom Resource (CR), and it describes the target end-state for the cluster. What follows is a minimal example of a MissionControlCluster instance which creates a 3-node DSE cluster running version 6.9.2. Each node has 5 GB of storage available for data and requests 32 GB of RAM. For more information, see the HCD, DSE, or Cassandra capacity planning guide.

Sample (partial) MissionControlCluster manifest (object):

apiVersion: missioncontrol.datastax.com/v1beta2
kind: MissionControlCluster
metadata:
  name: test
  namespace: PROJECT_SLUG
spec:
  k8ssandra:
    cassandra:
      serverVersion: 6.9.2
      serverType: dse
      storageConfig:
        cassandraDataVolumeClaimSpec:
          storageClassName: standard
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 5Gi
      networking:
        hostNetwork: true
      datacenters:
        - metadata:
            name: dc1
          size: 3
          resources:
            requests:
              memory: 4Gi
...

Replace PROJECT_SLUG with the name of your project.

During Mission Control installation, the user interface shares various configuration settings to review or set. Use the reference catalog to guide your configuration decisions.

  1. Specify certain parameters in this CR file.

    1. The apiVersion and kind parameters indicate what type of resource this file represents. In this example, kind is a MissionControlCluster resource with an apiVersion of v1beta2.

    2. This YAML specification outlines metadata associated with this cluster. At a minimum you must specify a name for your cluster. This value is used in the cluster_name parameter of the cassandra.yaml file.

      Each name must be unique within a project. Submitting two clusters with the same name results in the first cluster being overwritten by the second.

    3. Other fields that may be present in the metadata include annotations or labels to provide additional ancillary data. At this time Mission Control does not use any of these fields, but they may be leveraged by automation within the your environment.

    4. After the metadata block review the spec, or specification, section. spec is the declaration of our target end-state for the cluster. Instead of describing the various steps to create a cluster you simply define what you want your cluster to look like and Mission Control handles reconciling existing or missing resources towards that end-state.

      See the MissionControlCluster reference for a list of options and their descriptions.

  2. The MissionControlCluster is saved to disk as test.missioncontrolcluster.yaml.

    Any filename is valid here. Using <resource_name>.<kind>.yaml allows you to easily differentiate multiple files in a given directory.

  3. Submit the MissionControlCluster YAML file to the Mission Control Control Plane Kubernetes cluster with kubectl.

    kubectl acts as a Kubernetes API client and handles calls to the Kubernetes API server. Advanced users may choose to leverage programmatic clients or GitOps tooling such as Flux instead of the imperative nature of the kubectl CLI.

    Submission of the object is handled with the kubectl apply sub-command.

    For example:

    kubectl apply -f MANIFEST_FILENAME.yaml

    Replace MANIFEST_FILENAME with the name of your file.

    This reads the file specified with the -f flag and submits it to the Control Plane Kubernetes cluster. Should an object exist within the Kubernetes API sharing the same namespace and name it is updated to match the local file. When no file exists a new entry is created. As the new MissionControlCluster object becomes available within the Kubernetes API, Mission Control detects the new resource and immediately begins reconciliation.

Automatic reconciliation steps for MissionControlCluster resources

The following steps describe the automated process for informational purposes only; no user intervention is required.

  1. Cluster-level operators detect a new MissionControlCluster custom resource through the Kubernetes API within the Control Plane.

  2. Cluster-level operators identify which Control Plane or Data Plane clusters should receive datacenters defined within the MissionControlCluster. In this example the east Data Plane cluster is specified so datacenter-level resources are created and reconciled there. If you omit the Data Plane identifier, resources are deployed within the Control Plane.

  3. Datacenter-level operators within the Data Plane clusters detect new datacenter-level custom resources (CRs) through the Kubernetes API.

  4. Datacenter-level operators generate and submit rack-level resources (StatefulSets) to their local Kubernetes API.

  5. Built-in Kubernetes reconciliation loops detect the new rack-level resources and begin creating pods and storage resources representing the underlying HCD, DSE, or Cassandra nodes.

  6. Status of resource creation rises to operators at the datacenter and cluster levels.

  7. When all pods are up and running the cluster-level operator signals the datacenter-level operators to begin bootstrap operations of DSE within the created and running pods.

  8. As pods come online their status is escalated and operations continue until all 3 nodes are up and running with services discoverable via the Kubernetes API.

Monitor bootstrap progress

Monitor the progress of the bootstrap to determine completion status or note any errors.

After you submit the MissionControlCluster custom resource (CR), the operator modifies the resource within the Kubernetes API by adding a status field to the top-level of the resource.

This status field provides valuable insight into the health of the MissionControlCluster as one or more operators detect definition changes. status indicates everything from the reconciliation phase to errors encountered while attempting to create storage.

  1. Run the following command to retrieve the descriptive status for the test MissionControlCluster object:

    kubectl describe mccluster/CLUSTER_NAME

    Replace CLUSTER_NAME with the name of your cluster.

    You can specify MissionControlCluster, missioncontrolcluster, or the short-form mccluster. Additionally, all of the names can be plural.

    Sample results
    Name:         CLUSTER_NAME
    Namespace:    default
    Labels:       <none>
    Annotations:  <none>
    API Version:  missioncontrol.datastax.com/v1beta2
    Kind:         MissionControlCluster
    Metadata:
      Creation Timestamp:  2023-10-30T11:09:33Z
      Finalizers:
        missioncontrol.datastax.com/finalizer
      Generation:        1
      Resource Version:  105388250
      UID:               57e956f8-1f87-422f-a7f8-b9ec87b956c4
    Spec:
      Create Issuer:  true
      Encryption:
        Internode Encryption:
          Certs:
            Cert Template:
              Issuer Ref:
                Name:
              Secret Name:
            Create Certs:   true
          Enabled:          true
      k8ssandra:
        Auth:  true
        Cassandra:
          Datacenters:
            Dse Workloads:
            Metadata:
              Name:  dc1
              Pods:
              Services:
                Additional Seed Service:
                All Pods Service:
                Dc Service:
                Node Port Service:
                Seed Service:
            Per Node Config Init Container Image:  <name>/yq:4
            Per Node Config Map Ref:
            Racks:
              Name:   rack1
            Size:     3
            Stopped:  false
          Metadata:
            Pods:
            Services:
              Additional Seed Service:
              All Pods Service:
              Dc Service:
              Node Port Service:
              Seed Service:
          Per Node Config Init Container Image:  <name>/yq:4
          Resources:
            Requests:
              Memory:      32Gi
          Server Type:     dse
          Server Version:  6.9.2
          Storage Config:
            Cassandra Data Volume Claim Spec:
              Access Modes:
                ReadWriteOnce
              Resources:
                Requests:
                  Storage:         5Gi
              Storage Class Name:  standard
          Superuser Secret Ref:
            Name:          my-cluster-superuser
        Secrets Provider:  internal
    Status:
      Conditions:
        Last Transition Time:  2023-10-30T14:08:08Z
        Message:
        Reason:                UpdatingIssuers
        Status:                False
        Type:                  UpdatingIssuers
        Last Transition Time:  2023-10-30T14:08:08Z
        Message:
        Reason:                UpdatingCertificates
        Status:                False
        Type:                  UpdatingCertificates
        Last Transition Time:  2023-10-30T14:08:08Z
        Message:
        Reason:                UpdatingReplicatedSecrets
        Status:                False
        Type:                  UpdatingReplicatedSecrets
        Last Transition Time:  2023-10-30T11:16:38Z
        Message:
        Reason:                UpdatingCluster
        Status:                False
        Type:                  UpdatingCluster
        Last Transition Time:  2023-10-30T14:08:08Z
        Message:               Ready
        Reason:                Ready
        Status:                True
        Type:                  Ready
    Events:                    <none>
  2. Access operator logs to discover more detail:

    kubectl logs -n mission-control POD_NAME

    Replace POD_NAME with the name of your pod, for example: mission-control-controller.

    The StatefulSet controller is one of the core Kubernetes controllers that create the pods. The number of pods per StatefulSet is calculated by dividing the number of nodes in the datacenter by the number of racks.

    For example, a 3 node cluster with 3 racks has 1 pod per StatefulSet, and a 9 node cluster with 3 racks has 3 pods per StatefulSet.

Next steps

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com