dsetool utility 

The dsetool utility for DataStax Enterprise (DSE) performs operations, including creating system keys, encrypting sensitive configuration information, temporarily changing running parameters for the performance service, and listing node subranges of data in a keyspace.

Use the dsetool utility for DataStax Enterprise (DSE) operations, including creating system keys, encrypting sensitive configuration information, temporarily changing running parameters for the performance service, and listing node subranges of data in a keyspace.

Synopsis 

dsetool [connection_options] command command_args
Legend
Syntax conventions Description
Italics Variable value. Replace with a user-defined value.
[ ] Optional. Square brackets ( [ ] ) surround optional command arguments. Do not type the square brackets.
( ) Group. Parentheses ( ( ) ) identify a group to choose from. Do not type the parentheses.
| Or. A vertical bar ( | ) separates alternative elements. Type any one of the elements. Do not type the vertical bar.
[ -- ] Separate the command line options from the command arguments with two hyphens ( -- ). This syntax is useful when arguments might be mistaken for command line options.
JMX authentication is supported by some dsetool commands. Other dsetool commands authenticate with the user name and password of the configured user.
Note: Authentication credentials can be provided in several ways, see Connecting to authentication enabled clusters.

To enable dsetool to use Kerberos authentication, see Using dsetool with Kerberos enabled cluster.

These connection options specify how to connect and authenticate the dsetool command:
Short form Long form Description
-f arg --config-file=arg Path to configuration file that stores credentials. The credentials in this configuration file override the ~/.dserc credentials.
-l arg --username=arg Role to authenticate with the configured role.
-p arg --password=arg Password to authenticate with the configured role.
-a arg --jmxusername=arg User name for authenticating with secure JMX.
-b arg --jmxpassword=arg Password for authenticating with secure JMX.
-c arg --cassandra_port=arg DSE port number.
-h arg --host=arg Node hostname or IP address.

Authentication credentials can be provided in several ways, see Connecting to authentication enabled clusters.

To enable dsetool to use Kerberos authentication, see Using dsetool with Kerberos enabled cluster.

-j arg --jmxport=arg Remote JMX agent port number.
-s arg --port=arg The Apache Solr™ port number.
This table describes optional dsetool arguments to configure SSL for native client connections:
Argument Description
--ssl=true|false true - use SSL for native client connections.

false - do not use SSL for native client connections.

--ssl-protocol=ssl_protocol The SSL protocol to use for connection to DSE when SSL is not enabled. For example, --ssl-protocol=ssl4.

--sslauth=true|false true - use SSL client authentication.

false - do not use SSL client authentication.

--cipher-suites=ssl_cipher_suites A comma-separated list of SSL cipher suites for connection to DSE when SSL is enabled. For example, --cipher-suites=c1,c2,c3.
--keystore-password=ssl_keystore_password The keystore password for connection to DSE when SSL client authentication is enabled.
--keystore-path=ssl_keystore_path The path to the keystore for connection to DSE when SSL client authentication is enabled. For example, --keystore-path=/path/to/ks.
--keystore-type=ssl_keystore_type The keystore type for connection to DSE when SSL client authentication is enabled. For example, --keystore-type=jks1.
--truststore-password=ssl_truststore_password The truststore password to use for connection to DSE when SSL is enabled.
--truststore-path=ssl_truststore_path ssl_truststore_path - the path to the truststore to use for connection to DSE when SSL is enabled. For example, --truststore-path=/path/to/ts.
--truststore-type=ssl_truststore_type The truststore type for connection to DSE when SSL is enabled. For example, --truststore-type=jks2.

dsetool commands  

autojt (deprecated) 
This command is deprecated. Job Trackers are managed automatically.
checkcfs cfs:///|filepath| 
Checks a single Cassandra File System (CFS) file or the whole CFS using these options:
  • cfs:/// - Scan the entire CFS for corrupted files
  • filepath - Get details about a particular file that has been corrupted.
See checking the CFS using dsetool.
core_indexing_status keyspace_name.table_name [--all] [--progress] 
Retrieves the dynamic indexing status (INDEXING, FINISHED, or FAILED) of the specified core or cores in a DSE Search node, and optionally displays the percent complete and an estimated completion time in milliseconds.
$ dsetool -h IP_address core_indexing_status core_name|--all
where:
  • IP_address is the IP address of the host that is output by the dsetool ring command.
    If you do not specify the IP address, the default is the local node. For example:
    $ dsetool core_indexing_status wiki.solr
    wiki.solr: INDEXING
  • core_name is the name of the search index

    or

    --all to retrieve the dynamic indexing status of all search cores.

  • --progress to display the percent complete and an estimated completion time in milliseconds.
See Checking the indexing status using dsetool.
create_core keyspace_name.table_name [options] 
Supports DSE authentication with [-l username -p password].
Creates the search core and optionally generates resources automatically. Auto-generated schemas have default DocValues enabled. The case of keyspace and table names is preserved. You must use the correct case for the keyspace and table names. The search core is created with the specified keyspace and table name and following options:
Option Description
schema=filepath Path of the schema file. Cannot be specified when generateResources=true.
solrconfig=filepath Path of the solrconfig.xmlfile. Cannot be specified when generateResources=true.
distributed=true | false
  • true - distributes and applies the operation to all nodes in the local DC. Default.
  • false - applies the operation only to the node it was sent to. Works only when recovery=true.
recovery=true | false
  • true - if the search core is unable to load due to corrupted index, recovers it by deleting and recreating the index.
  • false - no recovery. Default.
reindex=true | false Observed only on auto-core creation (generateResources=true); otherwise, always reindexes on core creation. Reindex works on a datacenter (DC) level. Run this command once per search enabled DC.
  • true - reindexes the data. Keeps the current index (accepting reads) while the new index is building.
  • false - does not reindex the data. Default.
generateResources=true | false If true, automatically generates resources. Cannot be used with schema= and solrconfig=. Default: false.

The
    generateResources=true option does not generate resources if resources exist on
    solr_resources.

coreOptions= Path to the YAML-formatted options file when generateResources=true. See Customizing automatic resource generation.
coreOptionsInline=options Accepts the same options that can be specified in the YAML file that is specified with coreOptions. See Customizing automatic resource generation. Use the following syntax: key1:value1#key2:value2#.
Examples:
coreOptionsInline=include_columns:id,name,body#rt:true
You must remove any spaces when using double quotes:
coreOptionsInline="generate_DocValues_for_fields:'*'"
This dsetool create_core command is node-specific. The cluster-wide CQL command to create a search core is CREATE SEARCH INDEX.
createsystemkey 
Creates a local encryption key or a new encryption key on a remote KMIP host. Use the following syntax:
dsetool createsystemkey ['cipher' [key_length]] 
([filename] | -k=kmip_groupname [-t kmip_template] [-n namespace])
  • 'cipher' [length]: Sets the cipher algorithm name, mode, padding and length of the encryption key. DSE supports the following JCE ciphers:
    Cipher algorithms values
    cipher length
    AES/CBC/PKCS5Padding 128, 192, or 256
    AES/ECB/PKCS5Padding 128, 192, or 256
    DES/CBC/PKCS5Padding 56
    DESede/CBC/PKCS5Padding 112 or 168
    Blowfish/CBC/PKCS5Padding 32-448
    RC2/CBC/PKCS5Padding 40-128
    For example, to create a new key using the strongest available AES algorithm:
    dsetool createsystemkey 'AES/CBC/PKCS5Padding' 256

    The default cipher is 'AES/CBC/PKCS5Padding' 128.

  • filename: Specify the name for the encryption key file. Use a filename or the -k option.

    The default is system_key.

  • -k = kmip_groupname: Specify a KMIP group name that is defined in kmip_hosts section in the dse.yaml file. When using KMIP, returns the key URL (kmip://group_name/key_id).
    • -t kmip_template: Use a template from the KMIP server to generate a new key.
    • -n namespace: Creates a new key in an existing KMIP namespace.
    Use a filename or the -k option.
Note: When both the filename and KMIP host options are excluded, dsetool creates local file named system_key.
encryptconfigvalue 
Prompts for an text entry, and returns the encrypted string to use in configuration file properties. Encrypts the text using the config_encryption_key_name.
get_core_config keyspace_name.table_name [current=true|false] 
Outputs the latest uploaded solrconfig.xml resource file for the specified core. If current is set to true, returns the current live solrconfig.
This dsetool get_core_config command is node-specific. You can use the CQL shell command DESCRIBE SEARCH INDEX CONFIG command to view the search index configuration XML file.
get_core_schema keyspace_name.table_name [current=true|false] 
Supports DSE authentication with [-l username -p password].
Outputs the latest uploaded search schema. If current is set to true, returns the current live schema.
This dsetool get_core_schema command is node-specific. You can use the CQL shell command DESCRIBE SEARCH INDEX SCHEMA to view the search index schema XML file.
index_checks true|false 
Optional and experimental: When true, run index check to verify index integrity. Reads the full index and has performance impact. Run only when index is inactive; no writes are allowed while index check is running. No repairs or fixes occur. Default: false
Note: Running this index check is time consuming and implies a hard commit.
index_checks_stop true|false 
Requests to stop the current index checks task.
infer_solr_schema keyspace_name.table_name [coreOptions path_to_options_file] 
Supports DSE authentication with [-l username -p password].
Automatically infers and proposes a schema that is based on the specified keyspace and table. The search cores are not modified. The search schema is inferred with the specified coreOptions YAML file or specified on the command line with coreOptionsInline:
Option Description
coreOptions Path to the YAML-formatted options file when generateResources=true. See Customizing automatic resource generation.
coreOptionsInline=options Accepts the same options that can be specified in the YAML file that is specified with coreOptions. Use the following syntax: key1:value1#key2:value2#. See Customizing automatic resource generation.
Examples:
coreOptionsInline=include_columns:id,name,body#rt:true
You must remove any spaces when using double quotes:
coreOptionsInline="generate_DocValues_for_fields:'*'"
inmemorystatus [keyspace_name.table_name] 
Provides the memory size, capacity, and percentage for this node and the amount of memory each table is using. To get information about a single table, specify the keyspace and table. The unit of measurement is MB. Bytes are truncated.
list_index_files keyspace_name.table_name [--index directory] 
Lists all DSE Search index files for the specified search core on the local node with the following option:
  • --index directory - specifies the data directory that contains the Solr index files. When not specified, the default directory is inferred from the search core name.
The index file is encrypted only when the backing CQL table is encrypted and the search core uses EncryptedFSDirectoryFactory; otherwise, the index file is decrypted.
list_subranges keyspace_name.table_name keys_per_range start_token, end_token 
Divides a token range for a given keyspace/table into a number of smaller subranges of approximately keys_per_range. To be useful, the specified range should be contained by the target node's primary range. See Listing sub-ranges using dsetool.
listjt 
Lists all Job Tracker nodes grouped by the datacenter that is local to them.
managekmip (help|list|expirekey|revoke|destroy) kmip_group_name [command_arguments] 
Remotely manage KMIP encryption keys from a DSE node. Define each KMIP host under a user-defined group name in the kmip_hosts section of the dse.yaml. DSE can proxy the following KMIP commands:
list kmip_group_name [namespace=kmip_namespace]
Returns a detailed list of encryption keys from the KMIP host. Also supports listing keys that are in a specified KMIP namespace.
expirekey kmip_group_name key_id [datetime]
Sets the date and time to expire an encryption key on the remote KMIP host. Expire a key to change the encryption key without re-encrypting existing data. After the next refresh interval (key_cache_millis), new data is encrypted using a new key that is automatically issued by the KMIP server. The database uses an expired key to decrypt existing data. The KMIP host can reactivate expired keys.

When no date and time is specified, the key expires immediately and a new key is used the next time the key cache refreshes.

Format datetime as YYYY-MM-DD HH:MM:SS:T. For example, to expire the key on April 13th 2016 at 8:05pm specify the date and time as 2016-04-13 20:05:00:0.

revoke kmip_group_name key_id
Permanently deactivates the encryption key. The database continues to use the key until after the key cache interval, key_cache_millis (default five minutes). Perform a rolling restart after revoking the key to refresh the key cache. Revoked keys are available for data decryption but cannot be reactivated by the KMIP server.
destroy kmip_group_name key_id
Permanently remove the encryption key from the remote KMIP server. After a key is destroyed, it cannot be regenerated by the KMIP server and is no longer available for encryption or decryption.
CAUTION:
Existing data becomes inaccessible when you destroy a key without rekeying the SSTables. Follow the steps in Rekeying tables using a new key to remove KMIP keys without losing data.
node_health -h IP_address [-all] 
Retrieves a dynamic score between 0 and 1 that describes the health of a DataStax Enterprise node. If you do not specify the IP address, the default is the local DataStax Enterprise node. A higher score indicates better node health. Nodes that have a large number of dropped mutations and nodes that are just started have a lower health score.
$ dsetool -h IP_address node_health 
where IP_address is the IP address that is output by the dsetool ring command.
For example:
$ dsetool -h 200.192.10.11 node_health 
Node Health: 0.7
Specify -all to retrieve the node health scores for all nodes:
$ dsetool node_health -all
partitioner 
Returns the fully qualified classname of the IPartitioner that is in use by the cluster.
perf subcommand 
Modifies performance object settings as described in the subcommand section.
read_resource keyspace_name.table_name name=resfilename 
Supports DSE authentication with [-l username -p password].
rebuild_indexes keyspace_name.table_name [idx1,idx2,...] 
Rebuilds specified secondary indexes for specified keyspace/table. To rebuild all indexes, do not specify indexes and use only rebuild_indexes keyspace_name.table_name. This dsetool rebuild_indexes command is node-specific. The cluster-wide CQL command to rebuild a search index is REBUILD SEARCH INDEX.
reload_core keyspace_name.table_name [option ...] 
Supports DSE authentication with [-l username -p password].
Reloads a search core with the specified keyspace and table name. After you modify the search schema.xml or solrconfig.xml or upload resource files (like a synonym file), you must reload a search core to recognize the changes. The case of keyspace and table names is preserved. You must use the correct case for the keyspace and table names. Reloads a core with the following options:
Option Description
schema=filepath Path of the search index schema file
solrconfig=filepath Path of the search configuration solrconfig.xml file
distributed=true | false
  • true - distributes and applies the reload operation to all nodes in the local DC. Default.
  • false - applies the reload operation only to the node it was sent to.
Warning: Distributing a re-index to an entire datacenter degrades performance severely in that datacenter.
reindex=true | false Works on a datacenter level. Run once per search-enabled datacenter.
  • true - reindexes the data.
  • false - does not reindex the data. Default.
deleteAll=true | false
  • true - deletes the already existing index before reindexing; search results will return either no or partial data while the index is rebuilding.
  • false - does not delete the existing index, causing the reindex to happen in-place; search results will return partially incorrect results while the index is updating. Default.
Note: To reload the core and prevent reindexing, accept the default values reindex=false and deleteAll=false.
This dsetool reload_core command is node-specific. The cluster-wide CQL command to reload a search index is RELOAD SEARCH INDEX.
repaircfs [file_system] 
Repairs the file system from orphan blocks, where file_system specifies a CFS file system. Scans the sblocks table and deletes the data blocks that are not referenced from the inode table. Orphan blocks cannot be distinguished from a block that is being written. Do not use this command when data is being written to CFS.
The default value is cfs:/. To repair a CFS file system other than the default, specify the CFS file system. For example:
dsetool repaircfs othercfs:/
Restriction:
  • If replication factor (RF) is a value other than 1, you must run nodetool repair before you run dsetool repaircfs.
  • Do not run analytics jobs while dsetool repaircfs is running.
ring 
Lists the nodes in the ring sorted by token, including their node type. Datacenters with heterogeneous workloads are noted.
sparkworker restart 
Manually restarts the Spark Worker on the selected node, without restarting the node.
status 
Lists the nodes in their ring, including the node type and node health. When the datacenter workloads are the same type, the workload type is listed. When the datacenter workloads are heterogeneous, the workload type is mixed. Similar to the output of ring command.
stop_core_reindex keyspace_name.table_name [timeout] 
Stops the search core reindexing for the specified keyspace and table on the node where the command is run. Optionally, specify a timeout in minutes so that the core waits to stop reindexing until the specified timeout is reached, then gracefully stops the indexing. The default timeout is 1 minute. You can change the timeout for dsetool commands for DSE Search.
tieredtablestats [keyspace.table] [-v] 
Outputs tiered storage informationtiered storage information, including SSTables, tiers, timestamps, and sizes. Provides information on every table that uses tiered storage.
$ dsetool tieredtablestats foo.bar -v
  • -v Output statistics for each SSTable, in addition to the tier summaries.
  • keyspace.table Outputs statistics only for the specified keyspace and table.
tsreload client|server 
Reloads the node's truststores without a restart. Specify client or server:
  • client - Reloads the client truststore that is used for encrypted client-to-node communications.
  • server - Reloads the server truststore that is used for encrypted node-to-node SSL (internode) communications.
unload_core keyspace_name.table_name [option ...] 
Supports DSE authentication with [-l username -p password].
Removes a search core with the specified keyspace and table name. The case of keyspace and table names is preserved. You must use the correct case for the keyspace and table names. Unloads a core with the following options:
Option Description
deleteDataDir=true | false If true, deletes index data and any other artifacts in the solr.data directory. It does not delete DataStax Enterprise data. Default: false
deleteResources=true | false If true, deletes the resources associated with the search index. For example, solrconfig.xml and schema.xml.
distributed=true | false If true, deletes search data and resources across the cluster, depending on the values of deleteDataDir and deleteResources. Default: true

The removal of the secondary index from the table schema is always distributed.

upgrade_index_files keyspace_name.table_name -h IP_address [-c cassandra_port] [--backup] [--workspace directory] [--index directory] 
The node that contains the encryption configuration must be running. The local node is offline. The user that runs this command must have read and write permissions to the directory that contains the index files. Upgrades all DSE Search index files for the specified search core on the local node with the following options:
  • -h IP_address - Required. Node hostname or IP address of the remote node that contains the encryption configuration that is used for index encryption. The remote node must be running.
  • -c cassandra_port - The DSE port on the remote node that contains the encryption configuration.
  • --backup - Preserves the index files from the current index as a backup after successful upgrade. When not specified, index files from the current index are deleted.
  • --workspace directory - Specifies the workspace directory for the upgrade process. The upgraded index is created in this directory. When --backup is specified, the preserved index file backup is moved here. When not specified, the default directory is the same directory that contains the Solr index files.
  • --index directory - Specifies the data directory that contains the Solr index files. When not specified, the default directory is inferred from the search core name.
Index files are encrypted only when the backing CQL table is encrypted and the search core uses EncryptedFSDirectoryFactory; otherwise, the index file is decrypted.
write_resource keyspace_name.table_name name=uploaded_name file=path_to_file_to_upload 
Supports DSE authentication with [-l username -p password].
Uploads the specified DSE Search resource file.
$ dsetool write_resource keyspace.table name=ResourceFile.xml file=schemaFile.xml
You can specify a file name for the uploaded resource file and the path to the resource file to upload. For example, stopwords.txt.
$ dsetool write_resource keyspace.table name=ResourceFile.xml file=myPath1/myPath2/schemaFile.xml
Resource files are stored in the DSE database. To view the resources, use dsetool read_resource or use the Solr Admin interface. You can configure the maximum resource file size or disable resource upload with the DSE Search resource upload limit option in dse.yaml.

Checking the CFS using dsetool 

Use the dsetool checkcfs command to scan the Cassandra File System (CFS) for corrupted files. For example:
$ dsetool checkcfs cfs:///
Use the dsetool checkcfs command to get details about a particular file that has been corrupted. For example:
$ dsetool checkcfs /tmp/myhadoop/mapred/system/jobtracker.info

Listing sub-ranges using dsetool 

The dsetool command syntax for listing subranges of data in a keyspace is:
$ dsetool [-h hostname ] list_subranges keyspace table rows_per_subrange start_token end_token
  • rows_per_subrange - The approximate number of rows per subrange.
  • start_partition_range - The start range of the node.
  • end_partition_range - The end range of the node.
Note: Run nodetool repair on a single node using the output of list_subranges. The output must be partition ranges that are used on that node.
Example
$ dsetool list_subranges Keyspace1 Standard1 10000 113427455640312821154458202477256070485 0

Output

The output lists the subranges to use as input to the nodetool repair command. For example:
Start Token                             End Token                               Estimated Size
------------------------------------------------------------------------------------------------
113427455640312821154458202477256070485 132425442795624521227151664615147681247 11264
132425442795624521227151664615147681247 151409576048389227347257997936583470460 11136
151409576048389227347257997936583470460 0                                       11264

Nodetool repair command options

You must use the nodetool utility to work with sub-ranges. The start partition range (-st) and end partition range (-et) options specify the portion of the node that needs repair. You get values for the start and end tokens from the output of dsetool list_subranges command. The nodetool repair syntax for using these options is:
$ nodetool repair keyspace table -st start_token -et end_token
Example
$ nodetool repair Keyspace1 Standard1 -st 113427455640312821154458202477256070485 -et 132425442795624521227151664615147681247 
$ nodetool repair Keyspace1 Standard1 -st 132425442795624521227151664615147681247 -et 151409576048389227347257997936583470460
$ nodetool repair Keyspace1 Standard1 -st 151409576048389227347257997936583470460 -et 0

These commands begins an anti-entropy node repair from the start partition range to the end partition range.

Performance object subcommands 

The dsetool perf subcommands temporarily change the running parameters without changing the dse.yaml file do not require a restart:

Subcommand name Possible values Description
clustersummary enable|disable Toggle cluster summary statistics. See Collecting database summary diagnostics.
cqlslowlog threshold Set the CQL slow log threshold as a percentile of the actual request times:
  • [0,1] is a percentile threshold
  • >1 is an absolute threshold in milliseconds
  • 1.0 logs no queries
  • 99.9 logs 0.1% of the slowest queries
  • 95.0 logs 5% of the slowest queries
  • 50.0 logs 50% of the slowest queries
  • 0.0 logs all queries
See Collecting slow queries.
cqlslowlog enable|disable Toggle the CQL slow log collection.
cqlslowlog skip_writing_to_db skip_writing_to_db - Keeps slow queries in-memory only. Temporary equivalent of dse.yaml:cql_slow_log_options.skip_writing_to_db: true
cqlslowlog write_to_db Writes data to the database. Temporary equivalent of dse.yaml: cql_slow_log_options.skip_writing_to_db: false When data writes to the database, the threshold must be >= 2000 ms to prevent a high load on database.
cqlslowlog set_num_slowest_queries Set the number of slow queries to keep in-memory.
cqlslowlog recent_slowest_queries Retrieve the specified number of the most recent slow queries.
cqlsysteminfo enable|disable Toggle CQL system information statistics. See Collecting system level diagnostics.
dbsummary enable|disable Toggle database summary statistics. See Collecting database summary diagnostics.
histograms enable|disable Toggle table histograms. See Collecting histogram diagnostics.
resourcelatencytracking enable|disable Toggle resource latency tracking. See Collecting system level diagnosticsCollecting system level diagnostics.
solrcachestats enable|disable Toggle Solr cache statistics.
solrindexingerrorlog enable|disable Toggle Solr indexing error log.
solrindexstats enable|disable Toggle Solr index statistics.
solrlatencysnapshots enable|disable Toggle Solr latency snapshots.
solrrequesthandlerstats enable|disable Toggle Solr request handler statistics.
solrslowlog threshold

enable|disable

Set the Solr slow log threshold in milliseconds.

Toggle Solr slow sub-query log. See Collecting slow queries.

solrupdatehandlerstats enable|disable Toggle Solr update handler statistics.
userlatencytracking enable|disable Toggle user latency tracking. See Collecting user activity diagnostics.
Note: Changes made with performance object subcommands do not persist between restarts and are useful only for short-term diagnostics. To make these settings permanent you must change the dse.yaml options, see CQL Performance Service options.
The location of the dse.yaml file depends on the type of installation:

Package installations
Installer-Services installations

/etc/dse/dse.yaml

Tarball installations
Installer-No Services installations

installation_location/resources/dse/conf/dse.yaml