Metrics

Quick overview

  • advanced.metrics in the configuration. All metrics disabled by default. To enable, select the metrics library to use, then define which individual metrics to activate.
  • some metrics are per node, others global to the session, or both.
  • unlike driver 3, JMX is not provided out of the box. You need to add the dependency manually.

The driver is able to report measurements of its internal behavior to a variety of metrics libraries, and ships with bindings for three popular ones: Dropwizard Metrics , Micrometer Metrics and MicroProfile Metrics.

Selecting a Metrics Library

Dropwizard Metrics

Dropwizard is the driver’s default metrics library; there is no additional configuration nor any extra dependency to add if you wish to use Dropwizard.

Micrometer

To use Micrometer you must:

  1. Define MicrometerMetricsFactory as the metrics factory to use in the driver configuration: 
datastax-java-driver.advanced.metrics {
  factory.class = MicrometerMetricsFactory
}
  1. Add a dependency to java-driver-metrics-micrometer in your application. This separate driver module contains the actual bindings for Micrometer, and depends itself on the Micrometer library:
<dependency>
  <groupId>com.datastax.oss</groupId>
  <artifactId>java-driver-metrics-micrometer</artifactId>
  <version>${driver.version}</version>
</dependency>
  1. You should also exclude Dropwizard and HdrHistogram, which are two transitive dependencies of the driver, because they are not relevant when using Micrometer:
<dependency>
  <groupId>com.datastax.oss</groupId>
  <artifactId>java-driver-core</artifactId>
  <exclusions>
    <exclusion>
      <groupId>io.dropwizard.metrics</groupId>
      <artifactId>metrics-core</artifactId>
    </exclusion>
    <exclusion>
      <groupId>org.hdrhistogram</groupId>
      <artifactId>HdrHistogram</artifactId>
    </exclusion>
  </exclusions>
</dependency>

MicroProfile Metrics

To use MicroProfile Metrics you must:

  1. Define MicroProfileMetricsFactory as the metrics factory to use in the driver configuration:
datastax-java-driver.advanced.metrics {
  factory.class = MicroProfileMetricsFactory
}
  1. Add a dependency to java-driver-metrics-microprofile in your application. This separate driver module contains the actual bindings for MicroProfile, and depends itself on the MicroProfile Metrics library:
<dependency>
  <groupId>com.datastax.oss</groupId>
  <artifactId>java-driver-metrics-microprofile</artifactId>
  <version>${driver.version}</version>
</dependency>
  1. You should also exclude Dropwizard and HdrHistogram, which are two transitive dependencies of the driver, because they are not relevant when using MicroProfile Metrics:
<dependency>
  <groupId>com.datastax.oss</groupId>
  <artifactId>java-driver-core</artifactId>
  <exclusions>
    <exclusion>
      <groupId>io.dropwizard.metrics</groupId>
      <artifactId>metrics-core</artifactId>
    </exclusion>
    <exclusion>
      <groupId>org.hdrhistogram</groupId>
      <artifactId>HdrHistogram</artifactId>
    </exclusion>
  </exclusions>
</dependency>

Other Metrics libraries

Other metrics libraries can also be used. However, you will need to provide a custom metrics factory. Simply implement the com.datastax.oss.driver.internal.core.metrics.MetricsFactory interface for your library of choice, then pass the fully-qualified name of that implementation class to the driver using the advanced.metrics.factory.class option. See the reference configuration.

You will certainly need to add the metrics library as a dependency to your application as well. It is also recommended excluding Dropwizard and HdrHistogram, as shown above.

Enabling specific driver metrics

Now that the metrics library is configured, you need to activate the driver metrics you are interested in.

There are two categories of driver metrics:

  • session-level: the measured data is global to a Session instance. For example, connected-nodes measures the number of nodes to which we have connections.
  • node-level: the data is specific to a node (and therefore there is one metric instance per node). For example, pool.open-connections measures the number of connections open to this particular node.

To find out which metrics are available, see the reference configuration. It contains a commented-out line for each metric, with detailed explanations on its intended usage.

By default, all metrics are disabled. You can turn them on individually in the configuration, by adding their name to these lists:

datastax-java-driver.advanced.metrics {
  session.enabled = [ connected-nodes, cql-requests ]
  node.enabled = [ pool.open-connections, pool.in-flight ]
}

If you specify a metric that doesn’t exist, it will be ignored, and a warning will be logged.

Finally, if you are using Dropwizard or Micrometer and enabled any metric of timer type, such as cql-requests, it is also possible to provide additional configuration to fine-tune the underlying histogram’s characteristics and precision, such as its highest expected latency, its number of significant digits to use, and its refresh interval. Again, see the reference configuration for more details.

Selecting a metric identifier style

Most metric libraries uniquely identify a metric by a name and, optionally, by a set of key-value pairs, usually called tags.

The advanced.metrics.id-generator.class option is used to customize how the driver generates metric identifiers. The driver ships with two built-in implementations:

  • DefaultMetricIdGenerator: generates identifiers composed solely of (unique) metric names; it does not generate tags. All metric names start with the name of the session (see session-name in the configuration), and in the case of node-level metrics, this is followed by .nodes., followed by a textual representation of the node’s address. All names end with the metric distinctive name. See below for examples. This generator is mostly suitable for use with metrics libraries that do not support tags, like Dropwizard.

  • TaggingMetricIdGenerator: generates identifiers composed of a name and one or two tags. Session-level metric names start with the session. prefix followed by the metric distinctive name; node-level metric names start with the nodes. prefix followed by the metric distinctive name. Session-level tags will include a session tag whose value is the session name (see session-name in the configuration); node-level tags will include the same session tag, and also a node tag whose value is the node’s address. See below for examples. This generator is mostly suitable for use with metrics libraries that support tags, like Micrometer or MicroProfile Metrics.

For example, here is how each one of them generates identifiers for the session metric “bytes-sent”, assuming that the session is named “s0”:

  • DefaultMetricIdGenerator:
    • name:s0.bytes-sent
    • tags: {}
  • TaggingMetricIdGenerator:
    • name: session.bytes-sent
    • tags: { "session" : "s0" }

Here is how each one of them generates identifiers for the node metric “bytes-sent”, assuming that the session is named “s0”, and the node’s broadcast address is 10.1.2.3:9042:

  • DefaultMetricIdGenerator:
    • name : s0.nodes.10_1_2_3:9042.bytes-sent
    • tags: {}
  • TaggingMetricIdGenerator:
    • name nodes.bytes-sent
    • tags: { "session" : "s0", "node" : "\10.1.2.3:9042" }

As shown above, both built-in implementations generate names that are path-like structures separated by dots. This is indeed the most common expected format by reporting tools.

Finally, it is also possible to define a global prefix for all metric names; this can be done with the advanced.metrics.id-generator.prefix option.

The prefix should not start nor end with a dot or any other path separator; the following are two valid examples: cassandra or myapp.prod.cassandra.

For example, if this prefix is set to cassandra, here is how the session metric “bytes-sent” would be named, assuming that the session is named “s0”:

  • with DefaultMetricIdGenerator: cassandra.s0.bytes-sent
  • with TaggingMetricIdGenerator: cassandra.session.bytes-sent

Here is how the node metric “bytes-sent” would be named, assuming that the session is named “s0”, and the node’s broadcast address is 10.1.2.3:9042:

  • with DefaultMetricIdGenerator: cassandra.s0.nodes.10_1_2_3:9042.bytes-sent
  • with TaggingMetricIdGenerator: cassandra.nodes.bytes-sent

Using an external metric registry

Regardless of which metrics library is used, you can provide an external metric registry object when building a session. This allows the driver to transparently export its operational metrics to whatever reporting system you want to use.

To pass a metric registry object to the session, use the CqlSessionBuilder.withMetricRegistry() method:

CqlSessionBuilder builder = CqlSession.builder();
builder.withMetricRegistry(myRegistryObject);
CqlSession session = builder.build();

Beware that the driver does not inspect the provided object, it simply passes it to the metrics factory in use; it is the user’s responsibility to provide registry objects compatible with the metrics library in use. For reference, here are the expected base types for the three built-in metrics libraries:

  • Dropwizard: com.codahale.metrics.MetricRegistry
  • Micrometer: io.micrometer.core.instrument.MeterRegistry
  • MicroProfile: org.eclipse.microprofile.metrics.MetricRegistry

NOTE: MicroProfile requires an external instance of its registry to be provided. For Micrometer, if no registry object is provided, Micrometer’s globalRegistry will be used. For Dropwizard, if no registry object is provided, an instance of MetricRegistry will be created and used (in which case, it can be retrieved programmatically if needed, see below).

Programmatic access to driver metrics

Programmatic access to driver metrics is only available when using Dropwizard Metrics. Users of other libraries are encouraged to provide an external registry when creating the driver session (see above), then use it to gain programmatic access to the driver metrics.

The Dropwizard MetricRegistry object is exposed in the driver API via session.getMetrics().getRegistry(). You can retrieve it and, for example, configure a Reporter to send the metrics to a monitoring tool.

NOTE: Beware that session.getMetrics() is not available when using other metrics libraries, and will throw a NoClassDefFoundError at runtime if accessed in such circumstances.

Exposing driver metrics with JMX

Unlike previous driver versions, JMX support is not included out of the box.

The way to add JMX support to your application depends largely on the metrics library being used. We show below instructions for Dropwizard only. Micrometer also has support for JMX: please refer to its official documentation.

Dropwizard Metrics

Add the following dependency to your application (make sure the version matches the metrics-core dependency of the driver):

<dependency>
  <groupId>io.dropwizard.metrics</groupId>
  <artifactId>metrics-jmx</artifactId>
  <version>4.1.2</version>
</dependency>

Then create a JMX reporter for the registry:

MetricRegistry registry = session.getMetrics()
    .orElseThrow(() -> new IllegalStateException("Metrics are disabled"))
    .getRegistry();

JmxReporter reporter =
    JmxReporter.forRegistry(registry)
        .inDomain("com.datastax.oss.driver")
        .build();
reporter.start();

Note: by default, the JMX reporter exposes all metrics in a flat structure (for example, pool.open-connections and pool.in-flight appear as root elements). If you prefer a hierarchical structure (open-connections and in-flight nested into a pool sub-domain), use a custom object factory:

import com.codahale.metrics.jmx.JmxReporter;
import com.codahale.metrics.jmx.ObjectNameFactory;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;

ObjectNameFactory objectNameFactory = (type, domain, name) -> {
  StringBuilder objectName = new StringBuilder(domain).append(':');
  List<String> nameParts = Splitter.on('.').splitToList(name);
  int i = 0;
  for (String namePart : nameParts) {
    boolean isLast = (i == nameParts.size() - 1);
    String key =
        isLast ? "name" : Strings.padStart(Integer.toString(i), 2, '0');
    objectName.append(key).append('=').append(namePart);
    if (!isLast) {
      objectName.append(',');
    }
    i += 1;
  }
  try {
    return new ObjectName(objectName.toString());
  } catch (MalformedObjectNameException e) {
    throw new RuntimeException(e);
  }
};

JmxReporter reporter =
    JmxReporter.forRegistry(registry)
        .inDomain("com.datastax.oss.driver")
        .createsObjectNamesWith(objectNameFactory)
        .build();
reporter.start();

Exporting metrics with other protocols

Dropwizard Metrics has built-in reporters for other output formats: JSON (via a servlet), stdout, CSV files, SLF4J logs and Graphite. Refer to their manual for more details.