Testing DSE Tiered Storage configurations
DataStax recommends local configuration testing before deploying cluster wide, or for storage configurations that do not map cleanly to the database with CREATE
or ALTER
table statements.
Test adjustments on one or two nodes before deploying cluster wide.
You can add local configuration options to overwrite the tiered storage settings in the table schema.
You cannot overwrite the class or the tiered storage configuration name.
Prerequisites
Complete DSE Tiered Storage configuration steps before you adjust and test the configurations.
Procedure
Where is the dse.yaml
file?
The location of the dse.yaml
file depends on the type of installation:
Installation Type | Location |
---|---|
Package installations + Installer-Services installations |
|
Tarball installations + Installer-No Services installations |
|
-
To overwrite the settings in the table schema In the local
dse.yaml
file, add alocal_options
key to an existing tiered storage configuration.For example, for this
dse.yaml
configuration:tiered_storage_options: strategy1: tiers: - paths: - /mnt1 - paths: - /mnt2 - paths: - /mnt3
And this existing table schema with
'max_tier_ages': '3600,7200'
:CREATE TABLE ks.tbl (k INT, c INT, v INT, PRIMARY KEY (k, c)) WITH COMPACTION={'class':'TieredCompactionStrategy', 'tiering_strategy': 'TimeWindowStorageStrategy', 'config': 'strategy1', 'max_tier_ages': '3600,7200'};
You can adjust the
max_tier_ages
value to 7200,10800 on a single node, by adding thelocal_options
key like this:tiered_storage_options: strategy1: local_options: max_tier_ages: "7200, 10800" tiers: - paths: - /mnt1 - paths: - /mnt2 - paths: - /mnt3
-
Restart the node.
After the node starts, the tiered storage strategies that use
strategy1
now usemax_tier_ages
values "7200,10800", instead of "3600,7200" as configured on the table schema. -
To monitor the tiered storage behavior of individual tables, use the
dsetool tieredtablestats
command:dsetool tieredtablestats
ks.tbl Tier 0: Summary: max_data_age: 1449178580284 max_timestamp: 1449168678515945 min_timestamp: 1449168678515945 reads_120_min: 5.2188117172945374E-5 reads_15_min: 4.415612774014863E-7 size: 4839 SSTables: /mnt2/ks/tbl-257cecf1988311e58be1ff4e6f1f6740/ma-3-big-Data.db: estimated_keys: 256 level: 0 max_data_age: 1449178580284 max_timestamp: 1449168678515945 min_timestamp: 1449168678515945 reads_120_min: 5.2188117172945374E-5 reads_15_min: 4.415612774014863E-7 rows: 1 size: 4839 Tier 1: Summary: max_data_age: 1449178580284 max_timestamp: 1449168749912092 min_timestamp: 1449168749912092 reads_120_min: 0.0 reads_15_min: 0.0 size: 4839 SSTables: /mnt3/ks/tbl-257cecf1988311e58be1ff4e6f1f6740/ma-4-big-Data.db: estimated_keys: 256 level: 0 max_data_age: 1449178580284 max_timestamp: 1449168749912092 min_timestamp: 1449168749912092 reads_120_min: 0.0 reads_15_min: 0.0 rows: 1 size: 4839