Query metrics MBean
The query metrics MBean is useful for troubleshooting query performance, tuning the Solr configuration, such as the schema and caches, and tuning server resources, such as the JVM heap.
The query metrics MBean is useful for troubleshooting query performance, tuning the Solr configuration, such as the schema and caches, and tuning server resources, such as the JVM heap. The query metrics MBean records the amount of time spent to execute several main phases of a distributed query on the index.
Main operational phases
The main phases of a distributed query operation are:
- COORDINATE
Comprises the total amount of time spent by the coordinator node to distribute the query and gather/process results from shards. This value is computed only on query coordinator nodes.
- EXECUTE
Comprises the time spent by a single shard to execute the actual index query. This value is computed on the local node executing the shard query.
- RETRIEVE
Comprises the time spent by a single shard to retrieve the actual data from Cassandra. This value will be computed on the local node hosting the requested data.
Query metrics MBean operations
The query metrics MBean measures latency in microseconds. Metrics can be grouped by query, by providing an additional query.name parameter. For example, assuming you are using a Solr core named demo.solr and have indexed a field named type, this URL provides the additional query.name parameter:
http://localhost:8983/solr/demo.solr/select/?q=type:1&query.name=myquery
All metrics collected under a given query name are recorded and retrieved separately, as shown in the following list of operations. If no query name is provided, all metrics are recorded together.
Operations are:
- setEnabled(boolean enabled)
Enables/disables metrics recording. Enabled by default.
- isEnabled()
Checks if metrics recording is enabled.
- getLatencyPercentile(String phase, String query, double percentile)
Gets a query latency percentile by its query name, which is optional and can be null, and phase.
- getRecordedLatencyCount(String phase, String query)
Gets the total count of recorded latency metrics by its query name, which is optional and can be null, and phase.
- getUnrecordedLatencyCount()
Gets the total count of unrecorded latency values due to exceeding the maximum tracked latency, which is 10 minutes.
- resetLatency(String query)
Resets latency metrics for the given query name, which is optional and can be null.
- resetLatencies()
Resets all latency metrics.
Query metrics MBean operations use the phase names previously listed.
Using MBeans to evaluate performance
The following example shows how to use the MBeans on Linux to obtain information about performance while running the DataStax Solr stress test demo.
- Start a single DSE Search node.
- Start jconsole using the PID of the DSE Search node: For
example:
sudo jconsole 1284
- Change to the demos directory. The default location of the demos directory depends on the type of installation:
Installer-Services and Package installations /usr/share/dse/demos Installer-No Services and Tarball installations install_location/demos - Make demos/solr_stress your current directory.
- Execute this script to create the
schema:
where the script options are:./create-schema.sh [options]
- CQL table creation options
--ssl
use SSL for Cassandra table creation over cqlsh- Solr HTTP options
-e CA_CERT_FILE
use HTTPS with the provided CA certificate
The script creates the Cassandra schema and posts the solrconfig.xml and schema.xml files to these locations:The script then creates the core/index by posting to the following location:- http://localhost:8983/solr/resource/demo.solr/solrconfig.xml
- http://localhost:8983/solr/resource/demo.solr/schema.xml
- http://localhost:8983/solr/admin/cores?action=CREATE&name=demo.solr
-x schemafile.xml -t tableCreationFile.cql -r solrCofgFile.xml -k solrCore
- Execute this script to run the
benchmark:
where the script options are:./run-benchmark.sh [--clients=clients_count] [--loops=loops_count] [--fetch=fetch_size] [--solrCore=solr_core] [--testData=test_data_file] [--url=url1,url2,url3,...] [--qps=qps] [--stats=true|false] [--seed=seed_value]
- --clients
- The number of client threads to create.
- --loops
- The number of times the commands list gets executed if running sequentially or the number of commands to run if running randomly.
- --fetch
- Fetch size for CQL pagination (disabled by default). Only the first page is retrieved.
- --solrCore
- Solr core name to run the benchmark against.
- --testData
- Name of the file that contains the test data.
- --seed
- Value to set the random generator seed to.
- --qps
- Maximum number of queries per second allowed.
- --stats
- Specifies whether to gather statics during runtime and create a csv file with the recorded values.
- --url
- A comma delimited list of servers to run the benchmark against. For
example:
--url=http://localhost:8983,http://192.168.10.45:8983,http://192.168.10.46:8983
The demo creates a Solr core named demo.solr and indexes 50,000 documents.
Example CQL commands:./run-benchmark.sh --url=http://localhost:8983 --testData=resources/testCqlQuery.txt --solrCore=demo.solr
./run-benchmark.sh --url=http://localhost:8983 --testData=resources/testCqlWrite.txt --solrCore=demo.solr
See /demos/solr_stress/README.txt for execution modes and sample script commands.
- In jconsole, expand
The CommitMetrics and QueryMetrics MBean items appear.
. - In jconsole, in getLatencyPercentile button.
The Operation return value, 582 microseconds, appears:
, type EXECUTE in the p0 text entry box and
0.95 in the p1 text entry box. Click the
- Click OK.
- Query Solr 20,000 times using the query.name parameter. For
example:
curl "http://localhost:8983/solr/demo.solr/select/?q=type:1&query.name=myquery" curl "http://localhost:8983/solr/demo.solr/select/?q=type:3&query.name=myquery"
- In jconsole, in Operations
getLatencyPercentile, type EXECUTE
in the p0 text entry box, myquery in the p1 text entry box, and 95.0 in the P2
text entry box.
The Operation return value, 97 microseconds, appears.