Install and deploy DataStax Enterprise (DSE) on a cloud platform
You can deploy DataStax Enterprise (DSE) on major cloud providers.
Cloud deployment guides are provided by the DataStax Partner Network (DSPN).
|
Always test your configuration before deploying to production. |
Install DSE on Google Compute Engine
You can find DSE on the Google Cloud Platform (GCP) Marketplace.
For assistance with this installation, see the DSPN DSE Google Deployment Guide or contact IBM Support.
Install DSE on Microsoft Azure
You can find DSE on the Microsoft Azure Marketplace.
For assistance with this installation, see the DSPN DSE Azure Deployment Guide or contact IBM Support.
Install DSE on Amazon EC2
Before your deploy a DataStax Enterprise (DSE) cluster on Amazon EC2, make sure you are familiar with Amazon EC2 and the available EC2 instance types. Then, use the following guidelines to ensure your EC2 instance type and configuration are appropriate for your DSE workloads.
These are general guidelines that you might need to adjust for your use cases and workloads. For more specific guidance, contact IBM Support.
Use AMIs from trusted sources
Only use AMIs for supported platforms from a trusted source, such as the following:
Untrusted AMIs can be insecure and perform poorly due to the configuration required for the EC2 installation.
DataStax no longer hosts the ComboAMI. Instead, use an AMI for a supported platform from a trusted source.
Plan your EC2 deployment
- General instance type and size recommendations
-
Cost and performance can vary for different EC2 instance types. Make sure you choose an appropriately sized instance type for your workloads. For example:
-
m4.2xlarge: This is the absolute minimum for DSE. It is suitable for development environments and production environments with extremely light workloads on transaction (Cassandra) nodes only. Not suitable for DSE advanced workloads.
-
i3.4xlarge: Suitable for production environments with moderate workloads.
-
i3.8xlarge: Suitable for large production environments and heavy workloads.
-
i3.4xlarge or i3.8xlarge: Recommended for DSE Search and DSE Analytic nodes.
In EC2 instances, each vCPU is a hyperthread of an Intel Xeon core. This means that two virtual cores exist on one physical core. For example, an i3.8xlarge instance type has 32 vCPUs, which is equivalent to 16 physical cores.
-
- Minimum of 10,000 IOPS
-
DSE requires at least 10,000 IOPS per node. Make sure the storage configuration for your EC2 instance supports the required IOPS. For example:
-
EBS General Purpose SSD (gp2) volumes: To achieve the required I/O, you must use 3.5 TB volumes regardless of the actual space used. This is because gp2 provides 3 IOPS per GB.
gp2 volumes always achieve 10,000 IOPS or better when volumes larger than 3.5 TB are attached to instances.
-
Amazon EBS Provisioned IOPS SSD (io1) volumes: EBS io1 with 10,000 provisioned IOPS (PIOPS) provides the same performance level as gp2 using smaller volumes but at a higher cost.
-
Directly attached local SSDs: Also known as ephemeral or instance SSDs, this storage type can provide a better balance of cost and performance for i3 instances.
-
- EBS volumes for m4 instance types
-
SSD-backed general purpose volumes (gp2) or provisioned IOPS volumes (io1) are suitable for m4 instances running minimal production workloads on transaction nodes only (no DSE advanced workloads). These volume types are designed to deliver consistent, low latency performance (single-digit millisecond latencies).
gp2 volumes are suitable for most workloads, and they always meet the minimum required IOPS when volumes larger than 3.5 TB are attached to instances. These volumes are designed to deliver the provisioned performance 99.0% of the time.
io1 with 10,000 provisioned IOPS (PIOPS) provides equivalent performance as gp2 using smaller volumes, but it has higher costs. These volumes are designed to deliver the provisioned performance 99.9% of the time.
- Avoid EBS magnetic volumes
-
DataStax doesn’t recommend EBS magnetic volumes for DSE data storage for the following reasons:
-
EBS magnetic volumes compete with standard packets for network throughput. This means that EBS throughput can fail when a network link is saturated.
-
EBS magnetic volumes have unreliable performance, and I/O performance can be exceptionally slow. This can cause the system to queue reads and writes indefinitely until the entire cluster becomes unresponsive.
-
You cannot scale up capacity by increasing the number of EBS volumes per host. Attempting to do so can overload the system’s ability to maintain effective buffer caches while concurrently serve requests for all data that it manages.
Use only ephemeral
instance-storeor the recommended EBS volume types for data storage. -
- Initialize drives to optimize disk performance
-
For the best disk performance with mounted drives, DataStax recommends that you pre-warm your drives. To do this, write once to every drive location before production use. Depending on EC2 conditions, this practice can improve throughput exponentially. For more information, see Initialize instance store volumes on EC2 instances.
- Storage recommendations
-
DSE supports Just a Bunch of Disks (JBOD), which has high tolerance for partial failures in a disk array. This is dependent on the
disk_failure_policyparameter incassandra.yaml.JBOD support means you can use standard disks. Typically, RAID isn’t required. For certain workloads, RAID-0 can provide better throughput through parallel writes on disk. For more information, see RAID on data disks.
- EC2 security group network configuration
-
When deploying a cluster on EC2, you must create security rules that open DSE ports to other nodes in the same security group.
An EC2 security group acts as a firewall. You specify the protocols and ports that are open in your cluster as a range of IP addresses or by security group.
If the source IP is set to
0.0.0.0/0, all externally accessible ports are open to incoming traffic from any IP address. This is a significant security risk that can result in data loss and unauthorized access.For more information, see Amazon EC2 security groups for your EC2 instances.
Generally, when you have firewalls between machines, it is difficult to run JMX across a network and maintain security. This is because JMX connects on port 7199, handshakes, and then uses any port within the range of 1024 and higher. Instead, you can use SSH to execute commands remotely to connect to JMX.
- Multi-region and multi-AZ EC2 deployments
-
For multi-region and multi-availability zone (AZ) deployments, use any supported platform on each node. DataStax recommends using the same platform on all nodes.
Then, configure the cluster as a multiple datacenter cluster using the Amazon EC2 multi-region snitch.
If you are scaling up a legacy cluster based on the DataStax ComboAMI, use Ubuntu for additional nodes.
Deploy your EC2 instance and install DSE
-
-
Apply the recommended configuration for your DSE workloads and cluster architecture, as described in Plan your EC2 deployment.
-
Install DSE using the installation method that corresponds with your AMI’s platform.
For all installation methods, see Choose an installation method.
You can also use DSE OpsCenter Lifecycle Manager (LCM) to provision and configure your cluster.
For more information on DSE EC2 deployments, see the following: