Planning an Amazon EC2 cluster

Use AMIs from trusted sources

Use only AMIs from a trusted source. Random AMIs pose a security risk and may perform slower than expected due to the way the EC2 install is configured. The following are examples of trusted AMIs:

EC2 clusters spanning multiple regions and availability zones

When creating an EC2 cluster that spans multiple regions and availability zones, use any of the supported platforms and install Cassandra on each node. It is best practice to use the same platform on all nodes. If your cluster was instantiated using the DataStax AMI, use Ubuntu for the additional nodes. Configure the cluster as a multiple datacenter cluster using the Ec2MultiRegionSnitch.

Production Cassandra clusters on EC2

For production Cassandra clusters on EC2, use these guidelines for choosing the instance types:

Development and light production: m3.large
Moderate production: m3.xlarge
SSD production with light data: c3.2xlarge
Largest heavy production: m3.2xlarge (PV) or i2.2xlarge (HVM)

Note: The main difference between m1 and m3 instance types for use with Cassandra is that m3 instance types have faster, smaller SSD drives and m1 instance types have slower, larger rotational drives. Use m1 instance types when you have higher tolerance for latency SLAs and you require smaller cluster sizes, or both. For more aggressive workloads use m3 instance types with appropriately sized clusters.

EBS volumes recommended for production

SSD-backed general purpose volumes (GP2) or provisioned IOPS volumes (PIOPS) are suitable for production workloads. These volume types are designed to deliver consistent, low latency performance:


GP2	PIOPS
The best choice for most workloads and have the advantage of guaranteeing 10,000 IOPS when volumes larger than 3.5TB are attached to instances. Designed to deliver single-digit millisecond latencies. Designed to deliver the provisioned performance 99.0% of the time.	Designed to deliver single-digit millisecond latencies. Designed to deliver the provisioned performance 99.9% of the time.

EBS magnetic volumes not recommended

EBS magnetic volumes are not recommended for Cassandra data storage volumes for the following reasons:

EBS magnetic volumes contend directly for network throughput with standard packets. This contention means that EBS throughput is likely to fail if you saturate a network link.
EBS magnetic volumes have unreliable performance. I/O performance can be exceptionally slow, causing the system to back load reads and writes until the entire cluster becomes unresponsive.
Adding capacity by increasing the number of EBS volumes per host does not scale. You can easily surpass the ability of the system to keep effective buffer caches and concurrently serve requests for all of the data it is responsible for managing.

Note: Use only ephemeral instance-store or the recommended EBS volume types for Cassandra data storage.

For more information and graphs related to ephemeral versus EBS performance, see the blog article Systematic Look at EC2 I/O.

Disk Performance Optimization

To ensure high disk performance to mounted drives, it is recommended that you pre-warm your drives by writing once to every drive location before production use. Depending on EC2 conditions, you can get moderate to enormous increases in throughput. See Optimizing Disk Performance in the Amazon Elastic Compute Cloud Documentation.

Storage recommendations for Cassandra 1.2 and later

Cassandra 1.2 and later supports JBOD (just a bunch of disks). JBOD excels at tolerating partial failures in a disk array. Configure using the disk_failure_policy in the cassandra.yaml file. Addition information is available in the Handling Disk Failures In Cassandra 1.2 blog and Recovering from a single disk failure using JBOD.

Note: Cassandra JBOD support allows you to use standard disks. However, RAID0 may provide better throughput because it splits every block to be on another device. This means that writes are written in parallel fashion instead of written serially on disk.

The location of the cassandra.yaml file depends on the type of installation:

Package installations	/etc/cassandra/cassandra.yaml
Tarball installations	`install_location`/resources/cassandra/conf/cassandra.yaml

Storage recommendations for Cassandra 1.1 and earlier

RAID 0 the ephemeral disks. Then put both the data directory and the commit log on that volume. This has proved to be better in practice than putting the commit log on the root volume, which is also a shared resource. For more data redundancy, consider deploying your Cassandra cluster across multiple availability zones or using EBS volumes to store your Cassandra backup files.