Shard transport options for DSE Search/Solr communications

A custom, TCP-based communications layer for Solr is the default type in DataStax Enterprise. To improve Solr inter-node communications and avoid distributed deadlock during queries, switch from the HTTP-based communications to the netty non-blocking communications layer.

Available in DataStax Enterprise 4.0 is a custom, TCP-based communications layer for Solr. The new communications layer is an alternative to the default HTTP-based, Tomcat-backed interface, which customers say is slow and resource intensive. The new communications layer improves Solr inter-node communications in several ways:

  • Lowers latency
  • Reduces resource consumption
  • Increases throughput even while handling thousands of concurrent requests
  • Provides nonblocking I/O processing

To avoid distributed deadlock during queries, switch from the HTTP-based communications to the new non-blocking communications layer.

The TCP-based communications layer for Solr supports client-to-node and node-to-node encryption using SSL, but does not support Kerberos.

Configure the shard transport options in the dse.yaml file to select HTTP- or TCP-based communication. The dse.yaml file is located in the following directories:

  • Packaged installs: /etc/dse/cassandra
  • Tarball installs: install_location/resources/dse/conf

The shard_transport_options in the dse.yaml file for managing inter-node communication between Solr nodes are:

  • type: http or netty

    The default type, http, configures plain old Solr communication that uses the standard HTTP-based communications interface. Choosing the netty type configures TCP-based Solr communications. If you chose netty, the following netty options are applicable.

  • netty_server_port: 8984

    The TCP listen port, mandatory if use the netty type, or if you want to migrate to the netty type from the http type later. If you plan to use the http type indefinitely, either comment netty_server_port or set it to -1.

  • netty_server_acceptor_threads

    The number of server acceptor threads. The default is number of available processors.

  • netty_server_worker_threads

    The number of server worker threads. The default is number of available processors times 8.

  • netty_client_worker_threads

    The number of client worker threads. The default is number of available processors times 8.

  • netty_client_max_connections

    The maximum number of client connections. The default is 100.

  • netty_client_request_timeout

    The client request timeout in milliseconds. The default is 60000.