Controlling data collection

To control consumption of disk space, OpsCenter limits the growth of OpsCenter performance data by:

Excluding specified keyspaces and tables from performance data collection
Shortening the time period after which performance data automatically expires

Exporting metrics

Use Lifecycle Manager (LCM) to export metrics collection from DSE Metrics Collector to a supported monitoring tool.

For clusters not managed by LCM, see the DSE Metrics Collector documentation to configure metrics collection and exporting to a supported monitoring tool.

Excluding keyspaces and tables from data collection

By default, OpsCenter does not collect performance data for its own keyspace or the Cassandra system keyspace. You can manually add any other keyspaces or tables that you do not want to monitor in the [cassandra_metrics] section of the cluster_name.conf configuration file. For example, to prevent data collection for the keyspace test as well as the table Keyspace1.Standard1, update the OpsCenter cluster configuration cluster_name.conf file. The location of this file depends on the type of installation:

Package installations: /etc/opscenter/clusters/cluster_name.conf
Tarball installations: install_location/conf/clusters/cluster_name.conf

Uncomment and edit the following values in cluster_name.conf:

[cassandra_metrics]
ignored_keyspaces = system, OpsCenter, test
ignored_column_families = Keyspace1.Standard1

Tables (formerly column families) are specified in the format:

<keyspace_name>.<column_family_name>.

Configuration properties still refer to tables as column families in OpsCenter.

Metrics Collection Properties

[cassandra_metrics] ignored_keyspaces

A list of keyspaces to not collect metrics for, separated by commas. The default value is system, system_traces, system_auth, dse_auth, OpsCenter, system_views and system_virtual_schema.
[cassandra_metrics] ignored_column_families

A list of tables to not collect metrics for, separated by commas. Each entry should be of the form "ks.cf". For example: metrics_ignored_column_families = system.NodeInfo, system.Schema, Keyspace1.Standard1
[cassandra_metrics] ignored_solr_cores

A list of solr cores to not collect metrics for, separated by commas. Each entry should be of the form "ks.cf". For example: metrics_ignored_solr_cores = Keyspace1.Standard1, solr.wiki.

These properties set the expiration time for data stored in the OpsCenter keyspace. Each time period for rolling up data points into summary views has a separate expiration threshold, or time-to-live (ttl) value expressed in seconds. By default, shorter time periods have lower values that result in more efficient expiration and compaction of the relatively larger volumes of data. Uncomment these properties to change the default expiration periods for performance data. Properties and default values are:

[cassandra_metrics] 1min_ttl

Sets the time in seconds to expire 1 minute data points. The default value is 604800 (7 days).
[cassandra_metrics] 5min_ttl

Sets the time in seconds to expire 5 minute data points. The default value is 2419200 (28 days).
[cassandra_metrics] 2hr_ttl

Sets the time in seconds to expire 2 hour data points. The default value is 31536000 (365 days).
[cassandra_metrics] 24hr_ttl

Sets the time to expire 24 hour data points. The default value is 0, or never.

Rollup tables

OpsCenter rollup tables store the metrics that are displayed in the OpsCenter monitoring graphs. Each rollup table collects data for the period identified in the rollup table. For example, rollups60 stores an average or aggregate data point for a 60-second period.

The rollup tables are stored within the OpsCenter keyspace in the OpsCenter keyspace on the monitored DataStax Enterprise (DSE) cluster or a separate OpsCenter storage cluster.

Configure time-to-live (TTL) value for the collected data to remain in each rollup table in the [agent_config] section of cluster_name.conf.

[rollup_state_ttl] The time-to-live (TTL) for data points stored in rollup states before data is collected in the rollup60 table. Default: (* 86400 3) Example: rollup_state_ttl: 259200
[rollups60_ttl] Sets time-to-live (TTL) for rollups60 in seconds. A value of '0' prevents the data from expiring. Setting '-1' disables this rollup and prevents storing any data for it which might help reduce used disk space. Default: 604800 Example: rollups60_ttl: 604800
[rollups300_ttl] Sets time-to-live (TTL) for rollups300 in seconds. A value of '0' prevents the data from expiring. Setting '-1' disables this rollup and prevents storing any data for it which might help reduce used disk space. Default: 604800 Example: rollups300_ttl: 2419200
[rollups7200_ttl] Sets time-to-live (TTL) for rollups7200 in seconds. A value of '0' prevents the data from expiring. Setting '-1' disables this rollup and prevents storing any data for it which might help reduce used disk space. Default: 31536000 Example: rollups7200_ttl: 31536000
[rollups86400_ttl] Sets time-to-live (TTL) for rollups86400 in seconds. A value of '0' prevents the data from expiring. Setting '-1' disables this rollup and prevents storing any data for it which might help reduce used disk space. Default: 0 Example: rollups86400_ttl: 0

Related information

Excluding keyspaces and tables from data collection :page-aliases: configure/opscChangingPerformanceDataExpiration_t.adoc

Changing performance data expiration times

About this task

Performance data stored in OpsCenter expires after configurable time periods. The default values are designed to provide efficient compaction and eventual deletion of the data, with faster expiration times for the more granular, larger-volume data roll-ups.

One-minute roll-ups (1min_ttl) expire after one week, or 604800 seconds.
Five-minute roll-ups (5min_ttl) expire after four weeks, or 2419200 seconds.
Two-hour roll-ups (2hr_ttl) expire after one year, or 31536000 seconds.

To change expiration time period:

In this example, the one-minute and five-minute roll-ups are set to expire twice as fast as the defaults, two-hour roll-ups are set to be kept indefinitely (expiration is disabled), and the 24 hour roll-ups are not going to be stored.

Procedure

Locate the cluster_name.conf file. The location of this file depends on the type of installation:
- Package installations: /etc/opscenter/clusters/cluster_name.conf
- Tarball installations: install_location/conf/clusters/cluster_name.conf
Edit the cluster_name.conf file.
Add the following time-to-live (ttl) values in seconds under a [cassandra_metrics] section:
```
[cassandra_metrics]

1min_ttl = 302400
5min_ttl = 1209600
2hr_ttl = 0
24hr_ttl = 0
```
Restart OpsCenter.

Data collected after restarting OpsCenter expires according to the new settings. The data collected before restarting OpsCenter expires according to the setting in effect when it was collected.