dse.yaml configuration file

Configures DseAuthenticator to authenticate users when the authenticator option in cassandra.yaml is set to com.datastax.bdp.cassandra.auth.DseAuthenticator. Authenticators other than DseAuthenticator are not supported.

Enables user authentication.

The first scheme to validate a user against when the driver does not request a specific scheme.

List of schemes that are checked if validation against the first scheme fails and no scheme was specified by the driver.

Determines if roles need to have permission granted to them to use specific authentication schemes. These permissions can be granted only when the DseAuthorizer is used.

Controls whether DIGEST-MD5 authentication is allowed with Kerberos. Kerberos uses DIGEST-MD5 to pass credentials between nodes and jobs. The DIGEST-MD5 mechanism is not associated directly with an authentication scheme.

Controls how the DseAuthenticator responds to plain text authentication requests over unencrypted client connections.

Sets transitional mode for temporary use during authentication setup in an established environment.

Configures the DSE Role Manager. To enable role manager, set:

Manages granting and revoking of roles.

Set to true, to enable logging of DSE role creation and modification events in the dse_security.role_stats system table. All nodes must have the stats option enabled, and must be restarted for the functionality to take effect.

SELECT * FROM dse_security.role_stats;

 role  | created                         | password_changed
-------+---------------------------------+---------------------------------
 user1 | 2020-04-13 00:44:09.221000+0000 |                            null
 user2 | 2020-04-12 23:49:21.457000+0000 | 2020-04-12 23:49:21.457000+0000

(2 rows)

Configures the DSE Authorizer to authorize users when the authorization option in cassandra.yaml is set to com.datastax.bdp.cassandra.auth.DseAuthorizer.

Enables the DSE Authorizer for role-based access control (RBAC).

Allows the DSE Authorizer to operate in a temporary mode during authorization setup in a cluster.

Enables row-level access control (RLAC) permissions. Use the same setting on all nodes. See Setting up Row Level Access Control (RLAC).

Configures security for a DataStax Enterprise cluster using Kerberos.

The filepath of dse.keytab.

The service_principal that the DataStax Enterprise process runs under must use the form <dse_user>/_HOST@<REALM>, where:

Used by the Tomcat application container to run DSE Search. The Tomcat web server uses the GSSAPI mechanism (SPNEGO) to negotiate the GSSAPI security mechanism (Kerberos). <REALM> is the name of your Kerberos realm. In the Kerberos principal, <REALM> must be uppercase.

A comma-delimited list of Quality of Protection (QOP) values that clients and servers can use for each connection. The client can have multiple QOP values, while the server can have only a single QOP value.

Configures LDAP security when the authenticator option in cassandra.yaml is set to com.datastax.bdp.cassandra.auth.DseAuthenticator.

A comma separated list of LDAP server hosts.

The port on which the LDAP server listens.

Enable hostname verification. The following conditions must be met:

Distinguished name (DN) of an account with read access to the user_search_base and group_search_base. For example:

The password of the search_dn account.

Enables an SSL-encrypted connection to the LDAP server.

Enables STARTTLS connections to the LDAP server on the unencrypted port.

The filepath to the SSL certificates truststore.

The password to access the truststore.

Valid types are JKS, JCEKS, or PKCS12.

Distinguished name (DN) of the object to start the recursive search for user entries for authentication and role management memberof searches.

Identifies the user that the search filter uses for looking up usernames.

Contains a list of group names. Role manager assigns DSE roles that exactly match any group name in the list. Required when managing roles using group_search_type: memberof_search with LDAP (role_manager.mode:ldap). The directory server must have memberof support, which is a default user attribute in Microsoft Active Directory (AD).

Option to define additional search bases for users. If the user is not found in one search base, DSE attempts to find the user in another search base, until all search bases have been tried. See also user_search_base, group_search_base, and extra_group_search_bases.

Defines how group membership is determined for a user. Required when managing roles with LDAP (role_manager.mode: ldap).

The unique distinguished name (DN) of the group record from which to start the group membership search.

Set to any valid LDAP filter.

The attribute in the group record that contains the LDAP group name. Role names are case-sensitive and must match exactly on DSE for assignment. Unmatched groups are ignored.

Option to define additional search bases for groups. DSE merges all groups found in all the defined search bases. See also group_search_base, user_search_base, and extra_user_search_bases.

A credentials cache improves performance by reducing the number of requests that are sent to the internal or LDAP server. See Defining an LDAP scheme.

Configures a search cache to improve performance by reducing the number of requests that are sent to the internal or LDAP server.

Configures the connection pool for making LDAP requests.

The maximum number of active connections to the LDAP server.

The maximum number of idle connections in the pool awaiting requests.

Sets the encryption settings for system resources that might contain sensitive information, including the system.batchlog and system.paxos tables, hint files, and the database commit log.

Enables encryption of system resources. See Encrypting system resources.

The name of the JCE cipher algorithm used to encrypt system resources.

Default: AES

Length of key to use for the system resources. See Supported cipher algorithms names.

Optional. Size of SSTable chunks when data from the system.batchlog or system.paxos are written to disk.

KMIP key provider to enable encrypting sensitive system data with a KMIP key. Comment out if using a local encryption key.

The KMIP key server host. Set to the <kmip_group_name> that defines the KMIP host in kmip_hosts section. DSE requests a key from the KMIP host and uses the key generated by the KMIP provider.

Path to the directory where local encryption key files are stored, also called system keys. Distributes the system keys to all nodes in the cluster. Ensure the DSE account is the folder owner and has read/write/execute (700) permissions.

Enables encryption on sensitive data stored in tables and in configuration files.

The local encryption key filename or KMIP key URL to use for configuration file property value decryption.

Configures connections for key servers that support the KMIP protocol.

A user-defined name for a group of options to configure a KMIP server or servers, key settings, and certificates. For each KMIP key server or group of KMIP key servers, you must configure options for a <kmip_groupname> section. Using separate key server configuration settings allows use of different key servers to encrypt table data and eliminates the need to enter key server configuration information in Data Definition Language (DDL) statements and other configurations. DDL statements are database schema change commands like CREATE TABLE. Multiple KMIP hosts are supported.

A comma-separated list of KMIP hosts (<host>[:<port>]) using the FQDN (Fully Qualified Domain Name). Add KMIP hosts in the intended failover sequence because DSE queries the host in the listed order.

The path to a Java keystore created from the KMIP agent PEM files.

Valid types are JKS, JCEKS, PKCS11, and PKCS12. For file-based keystores, use PKCS12.

Password used to protect the private key of the key pair.

The path to a Java truststore that was created using the KMIP root certificate.

Milliseconds to locally cache the encryption keys that are read from the KMIP hosts. The longer the encryption keys are cached, the fewer requests to the KMIP key server are made and the longer it takes for changes, like revocation, to propagate to the DSE node. DataStax Enterprise uses concurrent encryption, so multiple threads fetch the secret key from the KMIP key server at the same time. DataStax recommends using the default value.

Socket timeout in milliseconds.

Tunes encryption of search indexes.

Allocates shared DSE Search decryption cache off JVM heap.

The maximum size of the shared DSE Search decryption cache in megabytes (MB).

Configures the schedulers in charge of querying for expired records, removing expired records, and the execution of the checks.

Time interval in seconds to check for expired data in seconds.

The number of seconds to delay the first TTL check to speed up start-up time.

The maximum number of documents to check and delete per batch by the TTL rebuild thread. All expired documents are deleted from the index during each check. To avoid memory pressure, their unique keys are retrieved and then deletes are issued in batches.

The maximum number of search indexes (cores) that can execute TTL cleanup concurrently. Manages system resource consumption and prevents many search cores from executing simultaneous TTL deletes.

For DSE Search, configure whether to asynchronously reindex bootstrapped data.

The maximum time in milliseconds to wait for all rows to be read from the database during CQL Solr queries.

Configures the maximum file size of the search index config or schema. Resource files can be uploaded, but the search index config and schema are stored internally in the database after upload.

Fault tolerance option for internode communication between DSE Search nodes.

Timeout behavior during distributed queries. The internal timeout for all search queries to prevent long running queries. The client request timeout is the maximum cumulative time (in milliseconds) that a distributed search request will wait idly for shard responses.

The maximum number of queued partitions during search index rebuilding and reindexing. This maximum number safeguards against excessive heap use by the indexing queue. If set lower than the number of threads per core (TPC), not all TPC threads can be actively indexing.

The maximum time, in minutes, to wait for the flushing of asynchronous index updates that occurs at DSE Search commit time or at flush time.

The maximum time, in minutes, to wait for each DSE Search index to load on startup or create/reload operations. This advanced option should be changed only if exceptions happen during search index loading.

Whether to apply the configured disk failure policy if IOExceptions occur during index update operations.

The directory to store index data. See Managing the location of DSE Search data. By default, each DSE Search index is saved in <solr_data_dir>/<keyspace_name>.<table_name> or as specified by the dse.solr.data.dir system property.

The Apache Lucene® field cache is deprecated. Instead, for fields that are sorted, faceted, or grouped by, set docValues="true" on the field in the search index schema. Then reload the search index and reindex.

Global Lucene RAM buffer usage threshold for heap to force segment flush. Setting too low can cause a state of constant flushing during periods of ongoing write activity. For near-real-time (NRT) indexing, forced segment flushes also de-schedule pending auto-soft commits to avoid potentially flushing too many small segments.

Global Lucene RAM buffer usage threshold for offheap to force segment flush. Setting too low can cause a state of constant flushing during periods of ongoing write activity. For NRT, forced segment flushes also de-schedule pending auto-soft commits to avoid potentially flushing too many small segments. When not set, the default is 1024.

Number of background threads used by the performance service under normal conditions.

Maximum number of background threads used by the performance service.

Allowed number of queued tasks in the backlog when the number of performance_max_threads are busy.

Graph event information.

Number of seconds a record survives before it is expired.

Configures reporting distributed sub-queries for search (query executions on individual shards) that take longer than a specified period of time.

The threshold in milliseconds or as a percentile.

The initial number of queries before activating the percentile filter.

Number of seconds a slow log record survives before it is expired.

Keeps slow queries only in-memory and does not write data to database.

The number of slow queries to keep in-memory.

Configures collection of system-wide performance information about a cluster.

Enables collection of system-wide performance information about a cluster.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

Configures collection of object I/O performance statistics.

Enables collection of object input output performance statistics.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

Configures collection of summary statistics at the database level.

Enables collection of database summary performance information.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

Configures collection of statistics at a cluster-wide level.

Enables collection of statistics at a cluster-wide level.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

Configures collection of data associated with Spark cluster and Spark applications.

spark_cluster_info_options:
    enabled: false
    refresh_rate_ms: 10000

Enables collection of Spark performance statistics.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

Histogram data for the dropped mutation metrics are stored in the dropped_messages table in the dse_perf keyspace.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

Default: 3

User-resource latency tracking settings.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

The maximum number of individual metrics.

Default: false

See Collecting slow search queries.

The number of seconds a record survives before it is expired.

The number of server threads dedicated to writing in the log. More than one server thread might degrade performance.

Default: 3000

Options to collect search index direct update handler statistics over time.

Options to collect search index request handler statistics over time.

Options to record search index statistics over time.

See Collecting cache statistics.

See Collecting Apache Solr performance statistics.

Collection of Spark application metrics.

The length of the sampling period in milliseconds; the frequency to update the performance statistics.

Collection that configures collection of metrics at the Spark Driver.

Enables collecting Spark Cassandra Connector metrics at the Spark Driver.

Enables collection of JVM heap and garbage collection (GC) metrics from the Spark Driver.

Enables collection of application state metrics at the Spark Driver.

Configures collection of metrics at Spark executors.

Enables collecting metrics collected at Spark executors.

Enables collection of Spark Cassandra Connector metrics at Spark executors.

Enables collection of JVM heap and GC metrics at Spark executors.

The length of a shared secret used to authenticate Spark components and encrypt the connections between them. This value is not the strength of the cipher for encrypting connections.

When DSE authentication is enabled with authentication_options, Spark security is enabled regardless of this setting.

When DSE authentication is enabled with authentication_options, Spark security encryption is enabled regardless of this setting.

Time interval in milliseconds between subsequent retries by the Spark plugin for Spark Master and Worker readiness to start.

Controls the physical resources used by Spark applications on this node. Optionally add named workpools with specific dedicated resources. See Core management.

Configures the amount of system resources that are made available to the Spark Worker.

The number of total system cores available to Spark.

The amount of total system memory available to Spark.

A collection of named workpools that can use a portion of the total resources defined under worker_options.

The name of the workpool. A workpool named alwayson_sql is created by default for AlwaysOn SQL. By default, the alwayson_sql workpool is configured to use 25% of the resources available to Spark.

The number of system cores to use in this workpool expressed as an absolute value or a decimal value. This option follows the same rules as cores_total.

The amount of memory to use in this workpool expressed as either an absolute value or a decimal value. This option follows the same rules as memory_total.

Configures encryption for Spark Master and Spark Worker UIs. These options apply only to Spark daemon UIs, and do not apply to user applications even when the user applications are run in cluster mode.

The source for SSL settings.

When encryption: custom, configures encryption for HTTPS of Spark Master and Worker UI.

Enables Spark encryption for Spark client-to-Spark cluster and Spark internode communication.

The keystore for Spark encryption keys.

The password to access the keystore.

Enables custom truststore for client authentication.

The filepath to the truststore for Spark encryption keys if require_client_auth: true.

The password to access the truststore.

The Transport Layer Security (TLS) authentication protocol. The TLS protocol must be supported by JVM and Spark. TLS 1.2 is the most common JVM default.

The key manager algorithm.

Valid types are JKS, JCEKS, PKCS11, and PKCS12. For file-based keystores, use PKCS12.

Valid types are JKS, JCEKS, and PKCS12.

A comma-separated list of cipher suites for Spark encryption. Enclose the list in square brackets.

Configures how Spark driver and executor processes are created and managed. See Running Spark processes as separate users.

When runner_type: run_as, Spark applications run as a different OS user than the DSE service user.

The list slot users to separate Spark processes users from the DSE service user.

The number of hours to retain audit events by supporting loggers for the CassandraAuditWriter.

Audit writer options.

The mode the writer runs in.

Available only when mode: async. Must be greater than 0.

Available only when mode: async.

The size of the queue feeding the asynchronous audit log writer threads.

The consistency level that is used to write audit events.

The directory to store the log file that reports dropped events.

The time interval in milliseconds between changing nodes to spread audit log information across multiple nodes. For example, to change the target node every 12 hours, specify 43200000 milliseconds.

The top-level configurations in Gremlin Server.

The available communications port for Gremlin Server.

The number of worker threads that handle non-blocking read and write (requests and responses) on the Gremlin Server channel, including routing requests to the right server operations, handling scheduled jobs on the server, and writing serialized responses back to the client.

This pool represents the workers available to handle blocking operations in Gremlin Server.

Configures gremlin server scripts.

Configures for gremlin-groovy scripts.

Configures gremlim groovy sandbox.

Configures sandbox rules.

List of packages, one package per line, to whitelist.

The fully qualified package name.

List of types, one type per line, to whitelist.

The fully qualified type name.

List of super classes, one class per line, to whitelist.

The fully qualified class name.

List of packages, one package per line, to blacklist.

The fully qualified package name.

List of super classes, one class per line, to blacklist. Retain the hyphen before the fully qualified class name.

The fully qualified class name.

System-level configuration options and options that are shared between graph instances. Add an option if it is not present in the provided dse.yaml file.

Maximum time to wait for an OLAP analytic (Spark) traversal to evaluate.

Maximum time to wait for an OLTP real-time traversal to evaluate.

Maximum time to wait for the database to agree on schema versions before timing out.

Maximum time to wait for a graph system-based request to execute, like creating a new graph.

The amount of ram to allocate to each graph’s adjacency (edge and property) cache.

The amount of ram to allocate to the index cache.

The maximum number of parameters that can be passed on a graph query request for TinkerPop drivers and drivers using the Cassandra native protocol. Passing very large numbers of parameters on requests is an anti-pattern, because the script evaluation time increases proportionally. DataStax recommends reducing the number of parameters to speed up script compilation times. Before you increase this value, consider alternate methods for parameterizing scripts, like passing a single map. If the graph query request requires many arguments, pass a list.