Read operations

All databases read fastest when the most frequently accessed data fits into memory.

Generally, the chunk cache (file cache) is best at improving performance. The chunk cache is similar to an OS page cache, and it is a central component of the DSE asynchronous thread-per-core (TPC) architecture. In some cases, the row cache can improve workloads where the overwhelming majority of requests are reads (approximately 95 percent). Row cache isn’t recommended for write-heavy workloads.

If you enable the row cache, it stores a subset of the partition data on disk in SSTables in memory. If this data is needed for a read, it is read from the row cache, eliminating the need to retrieve that data from disk. If a row must be retrieved from disk (SSTables), the compiled row is added to the row cache in case it is needed again.

The row cache is stored in fully off-heap memory using an implementation that relieves garbage collection pressure in the Java Virtual Machine (JVM). Data stored in the row cache uses a configurable amount of memory for a specified period of time. When the cache is full, the row cache uses least-recently-use (LRU) eviction to reclaim memory. This inherently results in the row cache populating with the most frequently read rows.

You can configure the row cache size and the number of rows to store. Storing a set number of rows is a useful feature that can make queries like "Last 10 items" very fast to read.

The row cache is not write-through. If a write mutates a cached row, the cached row is invalidated, and it isn’t cached again until the row is read again. Similarly, if a partition is updated, the entire partition is removed from the cache. If required partition data isn’t found in the row cache, then the database checks bloom filters.

Each SSTable has an associated bloom filter that can establish whether certain partition data is present in that SSTable. Bloom filters improve the performance of partition key lookups during reads by narrowing the pool of keys for a given SSTable. Bloom filters are used for index scans, not range scans.

The database checks the bloom filter to discover which SSTables are likely to have the requested partition data. If the bloom filter doesn’t explicitly exclude a particular SSTable, then the database checks the partition index.

Not all SSTables identified by a bloom filter will have the predicted data. Because the bloom filter is a probabilistic function, it can sometimes return false positives.

The bloom filter is stored in off-heap memory, and it can use as much as 1 to 2 GB per billion partitions. In an extreme case, each row can have a partition, so one machine could have billions of these entries.

Tuning bloom filter settings can help balance memory usage and performance (successful lookups). For more information, see Tune bloom filters.

The partition index maps partition keys to a row index. It supports iteration from a partially specified partition position if required.

Typically, database partitions vary in size. The partition index trie data structure uses unique byte-ordered partition key prefixes to point to the following:

Upon reaching a leaf of the trie, the partition key prefix matches some content in the file, but the database cannot be certain that this is a full match for the partition key. The leaf node points to a place in the row index (Row.Db) or data file (Data.Db). In either case, the first bytes at the specified position contain a serialization of the partition key, which is compared to the mapped key. If it matches, the database has found the partition. If it doesn’t match, the database knows that the SSTable has no data for this partition because the stored partition key prefixes are unique.

The chunk cache buffers data before it is compressed and written to an SSTable. When reading data, uncompressed data is checked first. If the data is compressed and on disk, the read path uses the compression offset map to locate the data on disk and uncompress the SSTable into memory.

The chunk cache is similar to an OS page cache. For more information about how the chunk cache uses memory and recommended settings for the chunk cache, see Recommended production settings.

The compression offset map stores pointers to the exact location on disk where the desired partition data is stored. This location is stored in off-heap memory, and it is accessed by the partition index.

Once the database knows the location on disk, it fetches the compressed partition data from the relevant SSTables. Then, the database prepares the result set for the original read request. Depending on caching configuration, the database also uncompresses retrieved data and stores it in memory.

The compression offset map can increase as much as 1 to 3 GB for each TB of compressed data. More compressed data requires more compressed blocks, resulting in a larger compression offset table to map the blocks.

In DSE, compression is enabled by default even though finding blocks in the compression offset map consumes CPU resources. The chunk cache is more effective with compression enabled, typically resulting in faster reads.

Write operations can affect the performance of read operations in a cluster. For example, compaction strategies that are best for read-heavy workloads minimize the number of SSTables and make reads more efficient because there are fewer files to check on the read path. In contrast, strategies that are best for write-heavy workloads create more SSTables, which can make reads take longer to complete.