Testing DSE Tiered Storage configurations

DataStax recommends local configuration testing before deploying cluster wide, or for storage configurations that do not map cleanly to the database with CREATE or ALTER table statements. Test adjustments on one or two nodes before deploying cluster wide. You can add local configuration options to overwrite the tiered storage settings in the table schema. You cannot overwrite the class or the tiered storage configuration name.

Prerequisites

Complete DSE Tiered Storage configuration steps before you adjust and test the configurations.

Procedure

Where is the dse.yaml file?

The location of the dse.yaml file depends on the type of installation:

Installation Type Location

Package installations + Installer-Services installations

/etc/dse/dse.yaml

Tarball installations + Installer-No Services installations

<installation_location>/resources/dse/conf/dse.yaml

  1. To overwrite the settings in the table schema In the local dse.yaml file, add a local_options key to an existing tiered storage configuration.

    For example, for this dse.yaml configuration:

    tiered_storage_options:
        strategy1:
          tiers:
            - paths:
              - /mnt1
            - paths:
              - /mnt2
            - paths:
              - /mnt3

    And this existing table schema with 'max_tier_ages': '3600,7200':

    CREATE TABLE ks.tbl (k INT, c INT, v INT, PRIMARY KEY (k, c))
    WITH COMPACTION={'class':'TieredCompactionStrategy',
    'tiering_strategy': 'TimeWindowStorageStrategy',
        'config': 'strategy1',
        'max_tier_ages': '3600,7200'};

    You can adjust the max_tier_ages value to 7200,10800 on a single node, by adding the local_options key like this:

    tiered_storage_options:
        strategy1:
          local_options:
            max_tier_ages: "7200, 10800"
          tiers:
            - paths:
              - /mnt1
            - paths:
              - /mnt2
            - paths:
              - /mnt3
  2. Restart the node.

    After the node starts, the tiered storage strategies that use strategy1 now use max_tier_ages values "7200,10800", instead of "3600,7200" as configured on the table schema.

  3. To monitor the tiered storage behavior of individual tables, use the dsetool tieredtablestats command:

    dsetool tieredtablestats
    ks.tbl
      Tier 0:
        Summary:
          max_data_age: 1449178580284
          max_timestamp: 1449168678515945
          min_timestamp: 1449168678515945
          reads_120_min: 5.2188117172945374E-5
          reads_15_min: 4.415612774014863E-7
          size: 4839
        SSTables:
          /mnt2/ks/tbl-257cecf1988311e58be1ff4e6f1f6740/ma-3-big-Data.db:
            estimated_keys: 256
            level: 0
            max_data_age: 1449178580284
            max_timestamp: 1449168678515945
            min_timestamp: 1449168678515945
            reads_120_min: 5.2188117172945374E-5
            reads_15_min: 4.415612774014863E-7
            rows: 1
            size: 4839
      Tier 1:
        Summary:
          max_data_age: 1449178580284
          max_timestamp: 1449168749912092
          min_timestamp: 1449168749912092
          reads_120_min: 0.0
          reads_15_min: 0.0
          size: 4839
        SSTables:
          /mnt3/ks/tbl-257cecf1988311e58be1ff4e6f1f6740/ma-4-big-Data.db:
            estimated_keys: 256
            level: 0
            max_data_age: 1449178580284
            max_timestamp: 1449168749912092
            min_timestamp: 1449168749912092
            reads_120_min: 0.0
            reads_15_min: 0.0
            rows: 1
            size: 4839

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com