Planning a DataStax Enterprise cluster on Amazon EC2 

Information about deploying a production DataStax Enterprise cluster on Amazon EC2.

Before planning an Amazon EC2 cluster, read Amazon EC2 - Virtual Server Hosting.

DataStax AMI deployments 

DataStax no longer hosts the DataStax ComboAMI. You can install DataStax Enterprise in two ways:

  • Create your instances using an AMI for a supported platform and from a trusted source. Then use the appropriate install method for your platform.
  • Use the Lifecycle Manager in OpsCenter to easily provision a DataStax Enterprise cluster for versions 4.7 and later:
    1. Create your instances using an AMI for a supported platform and from a trusted source.
    2. Use the Lifecycle Manager to provision and configure your cluster.

Use AMIs from trusted sources 

Use only AMIs for supported platforms and from a trusted source. Random AMIs pose a security risk and may perform slower than expected due to the way the EC2 install is configured. The following are examples of trusted AMIs:

EC2 deployments for multiple regions/availability zones 

For these deployments use any of the supported platforms on each node:

It is best practice to use the same platform on all nodes. If your cluster was instantiated using the DataStax AMI (no longer supported), use Ubuntu for the additional nodes. Configure the cluster as a multiple datacenter cluster using the Ec2MultiRegionSnitch.

Guidelines for production clusters 

For production clusters on EC2, use these guidelines for choosing the instance types:

  • Development and light production: m3.large
  • Moderate production: m3.xlarge
  • SSD production with light data: c3.2xlarge
  • Largest heavy production: m3.2xlarge (PV) or i2.2xlarge (HVM)
  • Micro, small, and medium types are not supported.

EBS volumes recommended for production 

SSD-backed general purpose volumes (GP2) or provisioned IOPS volumes (PIOPS) are suitable for production workloads. These volume types are designed to deliver consistent, low latency performance:
  • The best choice for most workloads and have the advantage of guaranteeing 10,000 IOPS when volumes larger than 3.5TB are attached to instances.
  • Designed to deliver single-digit millisecond latencies.
  • Designed to deliver the provisioned performance 99.0% of the time.
  • Designed to deliver single-digit millisecond latencies.
  • Designed to deliver the provisioned performance 99.9% of the time.

Disk Performance Optimization 

To ensure high disk performance to mounted drives, it is recommended that you pre-warm your drives by writing once to every drive location before production use. Depending on EC2 conditions, you can get moderate to enormous increases in throughput. See Optimizing Disk Performance in the Amazon Elastic Compute Cloud Documentation.

Storage recommendations 

DataStax Enterprise supports JBOD (just a bunch of disks). JBOD excels at tolerating partial failures in a disk array. Configure using the disk_failure_policy in the cassandra.yaml file. Addition information is available in the Handling Disk Failures In Cassandra 1.2 blog and Recovering from a single disk failure using JBOD.
Note: JBOD support allows you to use standard disks. However, RAID0 may provide better throughput because it splits every block to be on another device. This means that writes are written in parallel fashion instead of written serially on disk.

EC2 security group 

When deploying DataStax Enterprise on EC2, you must create security rules that open ports to other nodes in the same security group. An EC2 Security Group acts as a firewall that allows you to choose which protocols and ports are open in your cluster. You can specify the protocols and ports either by a range of IP addresses or by security group. For more information, see the Amazon EC2 help on Security Groups.

Warning: Specifying a Source IP of opens externally accessible ports to incoming traffic from any IP address. The risk of data loss is high.

The Securing DataStax Enterprise ports table provides a list of ports that should be opened to internode and client communications.

Note: Generally, when you have firewalls between machines, it is difficult to run JMX across a network and maintain security. This is because JMX connects on port 7199, handshakes, and then uses any port within the 1024+ range. Instead use SSH to execute commands remotely to connect to JMX locally or use the DataStax OpsCenter.

Other resources