Configure compression

Compression maximizes the storage capacity of DataStax Enterprise (DSE) nodes by reducing the volume of data on disk and disk I/O, particularly for read-heavy workloads. When processing a read request, the database finds the location of rows in the SSTable index, and then decompresses the relevant data chunks. In addition to compression, the DSE storage engine is designed to reduce disk volume automatically.

In DSE, compression doesn’t degrade write performance like it does in relational databases where writes must decompress, overwrite, and recompress data on disk. In DSE, data is written to disk and compressed when it is flushed from memtables to SSTables. SSTables are immutable, so there is no recompression cycle for writes. Optionally, you can also compress the commit log, which is used for durability of writes before they are flushed to disk.

Which tables to compress

Compression is most effective on larger tables where each row contains values for the same set of columns (or the same number of columns) as all other rows. For example, a table containing many rows of user data such as username, email, and state is a good candidate for compression.

Compression ratios and read performance improvements are better when rows have similar data across columns. Tables where the rows aren’t consistently populated, or the values in each cells are dramatically different, aren’t recommended for compression. Additionally, very small tables aren’t recommended for compression because there is little benefit to compressing such a small amount of data.

If a table is ideal for compression, enabling compression can provide the following benefits:

25 to 33 percent reduction in data size.
25 to 35 percent performance improvement on reads.
5 to 10 percent performance improvement on writes.

If you use DSE Search, DataStax recommends enabling compression to improve DSE Search read performance.

For tables that use Transparent Data Encryption (TDE), DataStax recommends that you also enable compression.

Compression algorithms

The following algorithms are available for SSTable and commit log compression:

LZ4 ('class' : 'LZ4Compressor'): This is the default compression algorithm. It provides a good balance of compression efficiency and CPU load required to deserialize and read compressed blocks.
Snappy ('class' : 'SnappyCompressor'): The Snappy algorithm’s compression ratio is similar to LZ4 but much slower.

If you are upgrading or migrating from an earlier version where Snappy was the default compression algorithm, DataStax recommends altering your tables to use LZ4.
Deflate ('class' : 'DeflateCompressor'): Provides a better compression ratio than other compression algorithms, but it generates much more CPU load and is much slower. Run performance tests before using this algorithm in production.
None ('class' : ''): Disables compression.

Configure SSTable compression

Compression is configured on individual tables with the CREATE TABLE and ALTER TABLE commands.

Compression is enabled by default with the LZ4Compressor algorithm, 64 KB chunk size, and 100 percent CRC check chance. Both of the following table definitions use the default compression configuration:

CREATE TABLE Dogs ( block_id uuid, name text, breed text, color text, age int,
    PRIMARY KEY (block_id)
)

CREATE TABLE Dogs ( block_id uuid, name text, breed text, color text, age int,
    PRIMARY KEY (block_id)
) WITH compression = { 'enabled' : true, 'class' : 'LZ4Compressor', 'chunk_length_in_kb' : 16Kib, 'crc_check_chance' : 1.0 };

You can set the following compression properties when you create or alter a table:

enabled: Must be true to use compression.

Set to false to disable compression:
```
ALTER TABLE Dogs
   WITH compression = { 'enabled' : false };
```
class: Specifies the compression algorithm to use.
chunk_length_in_kb: Specifies the size in KB of compressed blocks on disk. The default is 16 KB (16KiB), which is sufficient for most tables.

Avoid increasing this value without proper testing. Larger values can improve the compression rate but increases the amount of data that must be deserialized and read from disk during a read operation. Adjust this value based on your table’s access patterns, such as the amount of data typically requested by reads and the average row size.

Smaller values increase the amount of data stored on disk due to more blocks, which effectively negates the purpose of compression.

When using compression with encryption, the default chunk size is 64 KB.
crc_check_chance: Specifies the probability of checking compressed block checksums during reads to detect disk rot and avoid propagating corrupted blocks to other replicas. The default is 1.0 (100 percent, always check).

For more information and examples of compression table properties, see the CQL table options.

Rewrite SSTables after altering compression

Compression configuration changes only apply to SSTables created after you alter a table. Existing SSTables on disk aren’t rewritten with the new compression configuration until the next compaction event. You can force existing SSTables to be rewritten and recompressed using nodetool upgradesstables or nodetool scrub.

For example:

nodetool upgradesstables -a KEYSPACE_NAME TABLE_NAME

Configure commit log compression

To enable commit log compression, set the commitlog_compression parameter in cassandra.yaml.