Data maintenance through compaction
When processing writes, Astra DB Serverless stores data on disk in immutable files called SSTables.
Because the database cannot edit existing SSTables, new writes are always written to new SSTables, regardless of the contents of existing SSTables. Over time, the SSTables can accumulate multiple versions of the same row, and partitions can be stored across multiple SSTables. The number of SSTables directly impacts read request latency because the database must access multiple files to retrieve a complete row.
To prevent unbounded SSTable growth and maintain read performance, the database periodically merges SSTables into fewer files by reconciling the writes and discarding redundant data. This process is called compaction.
To maintain data integrity during reads and compaction, all SSTable writes are timestamped, and deletes are marked with tombstones. This means the database can determine the most recent write, and deleted data is removed in a consistent and deliberate way.
How compaction works
Compaction runs on multiple SSTables. From these SSTables, compaction finds all versions of each uniquely identified row, and then assembles a complete, unified version of each row. Redundant mutations are resolved using the most recent write timestamp for each of the row’s columns.
The merge process is performant because rows are sorted by partition key within each SSTable and the merge process does not use random I/O.
The new version of each row is written to a new SSTable. The old versions, along with any rows that are ready for deletion, are left in the old SSTables, and are deleted when any pending reads are completed.
Compaction causes a temporary spike in disk space usage and disk I/O while old and new SSTables co-exist. As it completes, compaction frees up disk space occupied by old SSTables. It improves read performance by incrementally replacing old SSTables with compacted SSTables. The database can read data directly from the new SSTable even before it finishes writing, instead of waiting for the entire compaction process to finish.
Astra provides predictable high performance even under heavy load by replacing old SSTables with new ones in memory as it processes writes and reads. Caching the new SSTable, while directing reads away from the old one, is incremental and doesn’t cause a performance degradation event known as a dramatic cache miss storm.
Routine compaction maintenance on a per-table level is considered minor compaction. When compaction runs on all tables in a keyspace, this is considered major compaction.
Compaction strategies
Compaction strategies determine which SSTables are chosen for compaction and how the compacted rows are sorted into new SSTables.
Astra supports only the UnifiedCompactionStrategy (UCS). This strategy unifies and improves upon SizeTieredCompactionStrategy (STCS) and LeveledCompactionStrategy (LCS). It also supports time series data. This all-inclusive compaction strategy works well for Astra DB use cases.
