New features in DSE OpsCenter 6.1

Changes in features, configuration files, metrics, and APIs in OpsCenter 6.1.

New features

cluster_name.conf

The location of the cluster_name.conf file depends on the type of installation:

Package installations: /etc/opscenter/clusters/cluster_name.conf
Tarball installations: install_location/conf/clusters/cluster_name.conf

cassandra-env.sh

The location of the cassandra-env.sh file depends on the type of installation:

Package installations Installer-Services installations	/etc/dse/cassandra/cassandra-env.sh
Tarball installations Installer-No Services installations	`installation_location`/resources/cassandra/conf/cassandra-env.sh

The cassandra-env.sh file is located in the installation_location/conf directory.

jvm.options

The location of the jvm.options file depends on the type of installation:

Package installations Installer-Services installations	/etc/dse/cassandra/jvm.options
Tarball installations Installer-No Services installations	`installation_location`/resources/cassandra/conf/jvm.options

address.yaml

The location of the address.yaml file depends on the type of installation:

Package installations: /var/lib/datastax-agent/conf/address.yaml
Tarball installations: install_location/conf/address.yaml

opscenterd.conf

The location of the opscenterd.conf file depends on the type of installation:

Package installations: /etc/opscenter/opscenterd.conf
Tarball installations: install_location/conf/opscenterd.conf

The following new and improved features are highlighted for the current DataStax Enterprise (DSE) OpsCenter version 6.1 release.


DataStax Enterprise 5.1	Support for provisioning and monitoring DataStax Enterprise 5.1 clusters: Added support for DSE Unified Authentication for JMX for DSE 5.1 and later. See Configuring a JMX Connection using LCM. Added LCM support for row-level access control (RLAC). Added support in LCM Config Profiles to configure the jvm.options file for DSE 5.1 and later. The options were moved from cassandra-env.sh to jvm.options. See Configuring JVM options in LCM. The schema viewer in the Data area of the OpsCenter Monitoring UI now displays materialized views, secondary indexes, and user-defined types used on a table. Users with permissions can view query statements on the new User-Defined Types, User-Defined Functions, and User-Defined Aggregates tabs. Removed support for Hadoop; deprecated in DSE 5.0 and 6.1 of OpsCenter.
Repair Service	Overhauled the OpsCenter Repair Service, including refactoring for reliability and performance: New Repair Service Status dashboard for complete transparency into progress of subrange and incremental repair processes, including remaining time and data to complete a repair cycle. Improved handling around repairs when nodes are down. Restrict incremental repairs by datacenter and racks for multi-DC clusters with replicated keyspaces. Ignore specific keyspaces and tables from the subrange repair process. Automatically throttles subrange repairs when appropriate. Exposed and improved error messages. Fixed critical bugs as noted in the 6.1.10 Release Notes.
Backup Service	Added features for flexible backup and restore workflows: Ability to enable backup failure alerts as an event to the Event log, which can be configured to alert as an email or posted URL to a chat room. Back up and restore a datacenter. Added a rule to the Best Practice Service when Commit Log Archiving is inconsistent. Added an unset throttle warning when restoring a backup. Cloning a backup from a Local FS location. Point-in-time restores from a Local FS location.
DataStax agents improvements	Improved flexibility for managing DataStax agents: Added backward-compatibility with agent versions that allow similar but different agent versions to co-exist with the installed OpsCenter version without loss of feature service. This feature provides flexibility with regard to maintaining a DSE cluster within OpsCenter. Exposed the DataStax Agent API in a Swagger UI console. See Enable and access the DataStax Agent API. Enhanced UI design for conveying Agents Status: Vastly improved visibility of agent version and agent compatibility, including better built-in guidance on when it is imperative that all agents get upgraded. Tooltips explain each column purpose in the Agent Status. A red icon appears on the Agents tab for increased visibility when agents are unhealthy. Replaced text in each status column and row with intuitive icons with tooltips for faster scanning of agent status.
New metrics	Added metrics for DSE Graph and messaging latency: Added OpsCenter support for DSE Graph metrics for dashboard graphs and alerts. Available for DSE versions 5.0 and later. Messaging latency metrics for both dashboard graphs and alerts at the datacenter and node levels.
Best Practice Service	Added a rule for Commit Log Archiving Setting Enabled Consistency Best Practice Rule. The rule ensures new nodes added to a cluster with Commit Log Archiving (CLA) enabled also have CLA enabled, which helps prevent data loss particularly during a Point-in-Time restore using the Backup Service. The rule checks to make sure all nodes within a cluster have Commit Log Archiving enabled.
Access Logs from the OpsCenter UI	Access critical logs from within the Node Details dialog in OpsCenter monitoring. These logs are a labs feature. See Viewing logs from node details.
Lifecycle Manager (LCM)	LCM now runs jobs concurrently across different clusters for faster job completion.

Updates from OpsCenter 6.0

The following changes are updates from the OpsCenter 6.0 major releases.

Support for using EC2 IAM roles for OpsCenter S3 backups

Added support to use system default credentials for Amazon S3 backups as described in Working with AWS Credentials. See Adding an Amazon S3 backup location for instructions on adding an Amazon S3 backup location.

Support for multiple user roles using LDAP authentication

Users can have multiple roles when using LDAP authentication. If the list of a user's groups map to more than one role in OpsCenter, the user will be granted each of the listed roles, and their resulting OpsCenter permissions will be the merging of permissions for all of their roles. See Adding a role for an LDAP user.

Important: In OpsCenter 6.1.10 and later and OpsCenter 6.5.3 and later, you must update custom scripts and applications that use the OpsCenter API if you want to use multiple user roles with LDAP authentication. If a custom script or application that uses the OpsCenter API did not account for multiple user roles, and a user has multiple roles, the script or application will fail because the role attribute cannot be found. The single role attribute will be provided for users that have only one role. If your application or script has users with only one role, then updates are not required for continued use.

Hadoop deprecated

DataStax has deprecated support for Hadoop as of DSE 5.1 and OpsCenter 6.1. DataStax Support can discuss alternative options for your organization.

Attempts to import a DSE cluster that has Hadoop enabled will fail. Lifecycle Manager can only import clusters that do not have Hadoop datacenter workloads. LCM cannot run jobs on clusters, datacenters, or nodes that have Hadoop enabled. Please contact DataStax Support for assistance with disabling Hadoop before retrying an import or attempting to run the job again.

The hive-site.xml configuration file has been relocated to the Spark section of the Configuration Profiles in LCM. The Hadoop section has been removed from LCM Config Profiles.

The Vnodes enabled on Hadoop nodes Best Practice Rule has been removed from OpsCenter monitoring.

Deprecated Hadoop-related configuration

The following port configuration option has been removed from opscenterd.conf: [hadoop] base_job_tracker_proxy_port.

The following configuration options have been removed from :

[hadoop] job_tracker_http_port
[hadoop] job_tracker_port
[hadoop] job_tracker_proxy_port
[kerberos] job_tracker_client_principal

The following agent configuration options have been removed from :

hadoop_conf_location
hadoop_log_location

Repair Service configuration changes

The Repair Service features have been significantly simplified and improved from both a usability and reliability standpoint. For complete details, see the Repair Service documentation in OpsCenter.

The following configuration options have been removed:

incremental_range_repair
ks_update_period
max_err_threshold
alert_on_repair_failure
repair_estimation_factor

The following configuration options have been added:

incremental_sleep
incremental_threshold
prioritization_page_size
offline_splits
ignore_keyspaces
ignore_table
time_to_completion_target_percentage
incremental_repair_datacenters
tokenranges_http_timeout
max_down_node_retry

Backup Service configuration changes

OpsCenter uses the Amazon SDK for S3 functionality. The [cloud] accepted_certs configuration option is no longer necessary and has been removed from opscenterd.conf.

The backup file queue has been modified into a backup job queue. The backup_file_queue_size configuration option has been removed from address.yaml.

The following configuration options were added to opscenterd.conf under [backup_service] for the Amazon S3 proxy:

s3_proxy_host
s3_proxy_port

Improvements to the S3 throttle feature available in the Add Location dialog in the Backup Service have deprecated the following options:

max-seconds-to-sleep
seconds-to-read-kill-channel
multipart-chunk-size
read-buffer-size
write-buffer-size

The options have been removed from address.yaml.

Best Practice Service Rule changes

The following changes were made to the Best Practice Service:

The Commit Log Archiving Setting Enabled Consistency rule has been added. The rule ensures new nodes added to a cluster with Commit Log Archiving (CLA) enabled also have CLA enabled to prevent any potential data loss, particularly during a Point-in-Time restore using the Backup Service. If the Commit Log Archiving setting is not consistent across all nodes in a cluster, the rule fails. For more information, see Configuring commit log backups.
The Vnodes enabled on Hadoop nodes Best Practice Rule has been removed.

Deprecated agent install script and configuration option

The following files related to legacy agent install features are no longer needed in OpsCenter 6.x and have been removed:

./agent/pkg/bin/install_agent.sh
./opscenterd/agent/bin/install_agent.sh

The agent install process is now internal to Opscenter, and no external script is necessary.

The path_to_installscript configuration option has been removed from the [agents] section of opscenterd.conf. The option if present in your configuration is ignored for OpsCenter 6.1 and later. Updating your configuration to remove the path_to_installscript is not necessary.

DataStax Agent backward compatibility

DataStax Agent backward-compatibility starts with OpsCenter 6.1 going forward. The DataStax Agent has backward compatibility with the installed OpsCenter version, which provides flexibility with the timing of upgrading agents. Before DataStax Agent backward compatibility, the OpsCenter version and the DataStax Agent version were required to match exactly. The DataStax Agent Status View lists any potential features that will not operate for a specific agent version and provides guidance on when it is imperative to upgrade agents.

Note: The OpsCenter 6.1 server is not compatible with DataStax Agent versions earlier than 6.1.0. All DataStax Agents must be upgraded to version 6.1.0.

Check the OpsCenter release notes and known issues for any changes that break DataStax Agent backward-compatibility for features.

Definitions files directory relocation

OpsCenter definitions are no longer considered configuration files, and have been moved from /etc/opscenter/definitions to /var/lib/opscenter/definitions. Existing definitions files will be left in place in /etc/opscenter/definitions, but will be ignored by OpsCenter. The existing definitions files in /etc/opscenter/definitions may optionally be deleted manually.