Provision a database cluster
With Mission Control, you can provision a database cluster directly in the Mission Control UI or through the kubectl
CLI.
With the CLI, you must define a YAML file and submit it with kubectl
.
Mission Control reconciles MissionControlCluster
resources defined either through the UI or CLI against any currently deployed database instances.
These definitions describe the desired state of your database clusters.
Using your definitions, Mission Control automates the process of provisioning and configuring resources across your Control and Data Planes.
Prerequisites
-
Because Mission Control organizes clusters by projects, an existing project is required.
-
A prepared environment on either bare-metal/VM or an existing Kubernetes cluster.
-
Access to the Mission Control UI or CLI.
Provision and define a cluster
You can provision a database cluster using the Mission Control UI or CLI. The UI offers simple mode and expert mode. DataStax recommends simple mode for users who are new to Kubernetes or Mission Control. It allows you to provision a cluster with a few clicks. Expert mode is for users who are familiar with the Kubernetes API and YAML configuration and allows you to define a cluster with more specific settings.
If you want to deploy the Data API to your new cluster, you can select it as an option when you use the Mission Control UI.
DataStax recommends the CLI for users who are familiar with kubectl
.
-
UI simple mode
-
UI expert mode
-
CLI
To provision a database cluster in Mission Control using simple mode, do the following:
-
In the Mission Control UI, select a project, and then click Create Cluster.
-
Enter a meaningful, human-readable Cluster Name.
The Cluster Name can be any string of characters, including international, alphanumeric, punctuation—dashes, spaces, underscores, upper or lower case.
Cluster names are permanent. You can’t change them after you create the cluster. The name uniquely identifies the cluster across all projects and all environments to prevent a logical cluster from inadvertently joining another.
-
Select a cluster Type.
-
Enter a valid Version number, for example,
1.0.0
. -
Leave the Image field blank. It is for advanced users.
-
To define the Datacenter configuration, do the following:
-
Enter a meaningful, human-readable Datacenter Name.
Datacenter names are permanent. You can’t change them after you create the cluster. The datacenter name:
-
Must start with an alphanumeric character.
-
Must be a single word.
-
Can be any capitalization: upper, lower, or mixed-case.
-
Can include dashes and underscores.
-
Must not include spaces.
-
-
Optional: Add the configuration property and its corresponding value in the Add cassandra.yaml Setting sub-section if you require a non-standard Cassandra configuration.
-
Select the Data Plane Context where you want to deploy the cluster.
By default, a database cluster will be deployed to the
Control Plane
. If aData Plane
is deployed on another Kubernetes cluster, you can choose to deploy the database cluster to that context. For more information, see the Planning guide. -
Enter a Rack Name for the first rack, for example,
rack1
.Rack names are permanent. You can’t change them after you create the cluster. The rack name:
-
Must start with an alphanumeric character.
-
Must be a single word.
-
Can be any capitalization: upper, lower, or mixed-case.
-
Can have dashes and underscores.
-
Must not include spaces.
Database pods, or nodes, are scheduled using node affinity.
-
-
Add the
mission-control.datastax.com/role=database
label to the rack configuration to ensure database pods are scheduled on database worker nodes only, not on platform worker nodes.-
Label:
mission-control.datastax.com/role
-
Value:
database
DataStax recommends a minimum of 3 nodes for production clusters to support replication in a datacenter for high availability. With 3 replicas in a datacenter, this configuration can tolerate a failure of one node when using a strong consistency of
LOCAL_QUORUM
.To add another rack, select Add Rack and configure it as you did in the previous steps. Make sure that you add the node affinity label.
-
-
For Nodes Per Rack, allocate at least one database node to the rack.
-
Optional: To create a multi-datacenter cluster, select Add Datacenter and configure it as above.
-
For Resource Requests, enter the minimum available resources required. DataStax recommends that you allocate the following minimum amounts of memory:
-
Enter the Storage Amount to allocate.
-
-
To add Security Settings, do the following:
-
Select the Require authentication to access cluster option.
-
Enter a Superuser Name.
-
Enter a Superuser Password.
-
Select the Enable internode encryption option.
The superuser role is required to provision other roles such as operators and service accounts.
DataStax recommends that you secure your clusters by enabling authentication and internode encryption, especially for production environments.
-
-
To configure Backup/Restore options, do the following:
-
Optional: Enter a Prefix to use as the name of the top-level folder in the Backup bucket. If you don’t enter a value, Mission Control uses the cluster name.
-
Select your Backup Configuration.
-
-
Under Advanced Settings, for Heap Amount, enter an amount using the following as a guide:
System memory
Heap
8 GB
4 GB
32 GB
8-24 GB
64 GB
31 GB
-
Optional: Under Data API, select Deploy the Data API to deploy the Data API.
-
Select Create Cluster.
-
Optional: To monitor the status, track the progress of the database pods provisioned by Mission Control:
kubectl get pods -n mission-control
The database pods will have names prefixed by the cluster name. Each node will go through a standard bootstrap sequence that will take approximately 2-3 minutes to complete. When the pods are operational and ready to accept client requests, each pod should show as
2/2
containersREADY
with aSTATUS
ofRunning
. -
Optional: To inspect pods that aren’t ready, run the following command:
kubectl describe pod -n mission-control
POD_NAME
Replace POD_NAME with the name of your pod.
DataStax recommends expert mode only for users who are familiar with the Kubernetes API and YAML configuration.
After you create or update a cluster in expert mode, you cannot edit it in simple mode. |
For custom resource definitions (CRDs), see Mission Control Custom Resource Definition (CRD) reference.
To provision a database cluster in Mission Control using expert mode, do the following:
-
In the Mission Control UI, select a project, and then click Create Cluster.
-
Click Expert. The Create Cluster page displays YAML configuration options.
-
Edit the YAML configuration to define the cluster.
-
Click Create Cluster.
When you use expert mode to copy your YAML definition and create a new cluster on another installation, you must **omit the |
Given that your Data Plane
clusters have either the appropriate
compute capacity or the capability to auto-scale, define a simple MissionControlCluster
YAML file and invoke kubectl
to create a running HCD, DSE, or Cassandra cluster.
Create a cluster by completing the following define and submit tasks. Review the automatic reconciliation workflow, and then monitor the reconciliation status with one kubectl
command.
To define a new MissionControlCluster
, start with creating a new YAML file that defines the topology and configuration for the new cluster. This file is an instance of a MissionControlCluster
Kubernetes Custom Resource (CR), and it describes the target end-state for the cluster. What follows is a minimal example of a MissionControlCluster
instance which creates a 3-node DSE cluster running version 6.9.2. Each node has 5 GB of storage available for data and requests 32 GB of RAM. For more information, see the HCD, DSE, or Cassandra capacity planning guide.
Sample (partial) MissionControlCluster
manifest (object):
apiVersion: missioncontrol.datastax.com/v1beta2
kind: MissionControlCluster
metadata:
name: test
namespace: PROJECT_SLUG
spec:
k8ssandra:
cassandra:
serverVersion: 6.9.2
serverType: dse
storageConfig:
cassandraDataVolumeClaimSpec:
storageClassName: standard
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
networking:
hostNetwork: true
datacenters:
- metadata:
name: dc1
size: 3
resources:
requests:
memory: 4Gi
...
Replace PROJECT_SLUG
with the name of your project.
During Mission Control installation, the user interface shares various configuration settings to review or set. Use the reference catalog to guide your configuration decisions.
-
Specify certain parameters in this CR file.
-
The
apiVersion
andkind
parameters indicate what type of resource this file represents. In this example,kind
is aMissionControlCluster
resource with anapiVersion
ofv1beta2
. -
This YAML specification outlines
metadata
associated with this cluster. At a minimum you must specify aname
for your cluster. This value is used in thecluster_name
parameter of thecassandra.yaml
file.Each name must be unique within a project. Submitting two clusters with the same name results in the first cluster being overwritten by the second.
-
Other fields that may be present in the
metadata
includeannotations
orlabels
to provide additional ancillary data. At this time Mission Control does not use any of these fields, but they may be leveraged by automation within the your environment. -
After the
metadata
block review thespec
, or specification, section.spec
is the declaration of our target end-state for the cluster. Instead of describing the various steps to create a cluster you simply define what you want your cluster to look like and Mission Control handles reconciling existing or missing resources towards that end-state.See the
MissionControlCluster
reference for a list of options and their descriptions.
-
-
The
MissionControlCluster
is saved to disk astest.missioncontrolcluster.yaml
.Any filename is valid here. Using
<resource_name>.<kind>.yaml
allows you to easily differentiate multiple files in a given directory. -
Submit the
MissionControlCluster
YAML file to the Mission ControlControl Plane
Kubernetes cluster withkubectl
.kubectl
acts as a Kubernetes API client and handles calls to the Kubernetes API server. Advanced users may choose to leverage programmatic clients or GitOps tooling such as Flux instead of the imperative nature of thekubectl
CLI.Submission of the object is handled with the
kubectl apply
sub-command.For example:
kubectl apply -f MANIFEST_FILENAME.yaml
Replace
MANIFEST_FILENAME
with the name of your file.This reads the file specified with the
-f
flag and submits it to theControl Plane
Kubernetes cluster. Should an object exist within the Kubernetes API sharing the same namespace and name it is updated to match the local file. When no file exists a new entry is created. As the newMissionControlCluster
object becomes available within the Kubernetes API, Mission Control detects the new resource and immediately begins reconciliation.
Automatic reconciliation steps for MissionControlCluster
resources
The following steps describe the automated process for informational purposes only; no user intervention is required.
-
Cluster-level operators detect a new
MissionControlCluster
custom resource through the Kubernetes API within theControl Plane
. -
Cluster-level operators identify which
Control Plane
orData Plane
clusters should receive datacenters defined within theMissionControlCluster
. In this example theeast
Data Plane cluster is specified so datacenter-level resources are created and reconciled there. If you omit theData Plane
identifier, resources are deployed within theControl Plane
. -
Datacenter-level operators within the
Data Plane
clusters detect new datacenter-level custom resources (CRs) through the Kubernetes API. -
Datacenter-level operators generate and submit rack-level resources (StatefulSets) to their local Kubernetes API.
-
Built-in Kubernetes reconciliation loops detect the new rack-level resources and begin creating pods and storage resources representing the underlying HCD, DSE, or Cassandra nodes.
-
Status of resource creation rises to operators at the datacenter and cluster levels.
-
When all pods are up and running the cluster-level operator signals the datacenter-level operators to begin bootstrap operations of DSE within the created and running pods.
-
As pods come online their status is escalated and operations continue until all 3 nodes are up and running with services discoverable via the Kubernetes API.
Monitor bootstrap progress
Monitor the progress of the bootstrap to determine completion status or note any errors.
After you submit the MissionControlCluster
custom resource (CR), the operator modifies the resource within the Kubernetes API by adding a status
field to the top-level of the resource.
This status
field provides valuable insight into the health of the MissionControlCluster
as one or more operators detect definition changes. status
indicates everything from the reconciliation phase to errors encountered while attempting to create storage.
-
Run the following command to retrieve the descriptive status for the
test
MissionControlCluster
object:kubectl describe mccluster/CLUSTER_NAME
Replace
CLUSTER_NAME
with the name of your cluster.You can specify
MissionControlCluster
,missioncontrolcluster
, or the short-formmccluster
. Additionally, all of the names can be plural.Sample results
Name: CLUSTER_NAME Namespace: default Labels: <none> Annotations: <none> API Version: missioncontrol.datastax.com/v1beta2 Kind:
MissionControlCluster
Metadata: Creation Timestamp: 2023-10-30T11:09:33Z Finalizers: missioncontrol.datastax.com/finalizer Generation: 1 Resource Version: 105388250 UID: 57e956f8-1f87-422f-a7f8-b9ec87b956c4 Spec: Create Issuer: true Encryption: Internode Encryption: Certs: Cert Template: Issuer Ref: Name: Secret Name: Create Certs: true Enabled: true k8ssandra: Auth: true Cassandra: Datacenters: Dse Workloads: Metadata: Name: dc1 Pods: Services: Additional Seed Service: All Pods Service: Dc Service: Node Port Service: Seed Service: Per Node Config Init Container Image: <name>/yq:4 Per Node Config Map Ref: Racks: Name: rack1 Size: 3 Stopped: false Metadata: Pods: Services: Additional Seed Service: All Pods Service: Dc Service: Node Port Service: Seed Service: Per Node Config Init Container Image: <name>/yq:4 Resources: Requests: Memory: 32Gi Server Type: dse Server Version: 6.9.2 Storage Config: Cassandra Data Volume Claim Spec: Access Modes: ReadWriteOnce Resources: Requests: Storage: 5Gi Storage Class Name: standard Superuser Secret Ref: Name: my-cluster-superuser Secrets Provider: internal Status: Conditions: Last Transition Time: 2023-10-30T14:08:08Z Message: Reason: UpdatingIssuers Status: False Type: UpdatingIssuers Last Transition Time: 2023-10-30T14:08:08Z Message: Reason: UpdatingCertificates Status: False Type: UpdatingCertificates Last Transition Time: 2023-10-30T14:08:08Z Message: Reason: UpdatingReplicatedSecrets Status: False Type: UpdatingReplicatedSecrets Last Transition Time: 2023-10-30T11:16:38Z Message: Reason: UpdatingCluster Status: False Type: UpdatingCluster Last Transition Time: 2023-10-30T14:08:08Z Message: Ready Reason: Ready Status: True Type: Ready Events: <none> -
Access operator logs to discover more detail:
kubectl logs -n mission-control POD_NAME
Replace
POD_NAME
with the name of your pod, for example:mission-control-controller
.The
StatefulSet
controller is one of the core Kubernetes controllers that create the pods. The number of pods perStatefulSet
is calculated by dividing the number of nodes in the datacenter by the number of racks.For example, a 3 node cluster with 3 racks has 1 pod per
StatefulSet
, and a 9 node cluster with 3 racks has 3 pods perStatefulSet
.
Next steps
-
Explore the
MissionControlCluster
reference documentation for a complete listing of all fields and values. -
Terminate the newly-created cluster.