DataStax Enterprise 4.8 release notes

Requirement for Uniform Clusters

All Nodes in each Cluster must be uniformly licensed to use the same Subscription. For example, if a Cluster contains 5 Nodes, all 5 Nodes within that Cluster must be either DataStax Basic, or all 5 Nodes must be DataStax Enterprise. Mixing different Subscriptions within a Cluster is not permitted. “Cluster” means a collection of Nodes running the Software which communicate with one another via Gossip, and “Gossip” means the mechanism within the Software enabling related Nodes to communicate with one another. For more information, see Enterprise Terms.

Before you upgrade

The latest version of DataStax Enterprise 4.8 is 4.8.16.

Be sure to read the DataStax Enterprise Upgrade Guide.
Be sure to check driver compatibility.

DSE 4.8.16

24 April 2018

4.8.16 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.19.1484
Apache Solr 4.10.3.1.2261
Apache Spark 1.4.2.10
Hive Connector 0.2.23

4.8.16 Changes and enhancements

DSE demos use Jetty Runner 9.4.8. (DSP-14772)
To preserve existing agents, the DataStax Installer no longer installs the OpsCenter DataStax Agent (datastax_agent). See Installing DataStax Agents. (DSP-13047)
DSE Search: Indexing is optimized for SSDs by default. Spinning disk detection logic is removed. (DSP-13924)
DSE Analytics: Improved troubleshooting capabilities by including keyspace name in Hive Connector CL log warning message. (DSP-14184)
DSE Search: Maximum number of entries in SolrFilterCache is limited to 32K. (DSP-14534)
Protect against overflow of local expiration time. (DSP-15412)
Improve live-node-replacement. (DSP-15412)
Protect against BigDecimals with large scale. (DSP-15698)

4.8.16 Resolved issues

Fix for possible scenario where newly-added nodes can have a schema mismatch for system keyspaces. (DSP-11787)
Apache HttpClient directory traversal through malformed URI. (DSP-13580)
DSE Search: Text field does not work for group by operations; unexpected docvalues type SORTED_SET error message for text fields. (DSP-14106)
SSL parameters are not passed to JVM when using sstableloader. (DSP-14202)
Memory leak causes executor descriptions to accumulate in DSE process. (DSP-14868)
JMX SSL is supported for use with dsetool and nodetool. See Setting up SSL for nodetool and dsetool. (DSP-14200)
Add metrics on coordination of read commands. (DSP-14775)
CVE-2017-7525: FasterXML Jackson-databind is prone to a remote-code execution vulnerability. (DSP-14784)
CVE-2018-1308: XML External Entity (XXE) processing attack through Apache Solr DataImportHandler dataConfig request parameter. (DSP-15902)
CVE-2017-7525: jackson-databind is vulnerable to remote code execution (RCE) attacks. (DSP-15096)
CVE-2016-1000031: Security vulnerability in Apache Commons FileUpload. (DSP-15908)
LDAP library issue. (DSP-15927)

4.8.16 Apache Cassandra™ changes

DataStax Enterprise 4.8.16 certifies Apache 2.1.19 with additional production-certified Cassandra changes.

DSE 4.8.15

19 October 2017

4.8.15 Components

Apache Solr™ 4.10.3.1.1982

4.8.15 Highlight

A single change for DSE Search:

Due to CVE-2017-12629, added Solr XMLParser protection from XML External Entity (XXE) attacks and removed Solr RunExecutableListener to harden security for DSE Search enabled clusters. (DSP-14618)

DSE 4.8.14

Important: DataStax recommends installing the latest patch release.

4 August 2017

4.8.14 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.19.1463
Apache Solr 4.10.3.1.1831
Hive Connector 0.2.22

4.8.14 Changes and enhancements

DSE Search: Display reindexing progress with dsetool core_indexing_status --progress option. (DSP-12617)
New nodetool sjk command for troubleshooting and monitoring that runs Swiss Java Knife (SJK) on the local node. (DSP-13838)

4.8.14 Resolved issues

Error out on tuple subfield sorting and faceting. (DSP-11907)
Snappy libraries have wide open permissions. (DSP-11668)
CVE-2017-7957 xstream-core is vulnerable to Denial of Service (DoS) attacks. (DSP-13419)
Spark-sql throws error when using an existing but hidden keyspace. (DSP-13556)
Allow aggressive expiration of fully expired sstables without timestamp/key overlap checks. (DSP-13987)
Unable to create Solr core with Kerberos enabled. (DSP-14069)

4.8.14 Known issue

Error messages show in the logs are caused by a race condition:
```
Error closing IndexWriter java.nio.channels.ClosedByInterruptException
```
Operations are not affected. You can ignore these messages. (DSP-12686)

4.8.14 Apache Cassandra™ changes

DataStax Enterprise 4.8.14 certifies Apache Cassandra™ 2.1.19 with additional production-certified Cassandra changes.

DSE 4.8.13

Important: DataStax recommends installing the latest patch release.

17 May 2017

4.8.13 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.17.1448
Apache Solr 4.10.3.1.1703
Apache Tomcat® 6.0.53
Hive Connector 0.2.21

4.8.13 Highlights

Executive summary highlights for DSE 4.8.13:

DSE Analytics 4.8.13 fixes a performance regression that was introduced in 4.8.11 where startup times for Spark SQL, including Spark SQL Thrift server, become very slow as the total number of tables in DSE increased (for example, into the hundreds). Versions 4.8.0 to 4.8.10 are not affected. (DSP-12754)
DSE Search 4.8.13 improvements include:
- Search result consistency by introducing the runtime node blacklisting. (DSP-12965)
- Indexing performance by removing the unnecessary Cassandra read in the scheduled Solr doc expiration. (DSP-12927)
- Heap usage while reindexing wide partitions. (DSP-12547)
- Searching performance for UDT sub-fields. (DSP-12812)

4.8.13 Changes and enhancements

Default batch size for the search TTL process is changed. (DSP-11493)
When a value is not specified for ttl_index_rebuild_options.max_docs_per_batch in dse.yaml, the default is changed from 100 to 4096.
When using the Solr Admin UI, users require SELECT permissions on any search core that they view. Specific permissions are required for all core operations. (DSP-11910)
When using the Solr Admin UI, users require SELECT permissions on any search core that they view. Specific permissions are required for all core operations. (DSP-11910)
Improve default selection for dse.yaml and solrconfig.xml write path configuration. See Configuring and tuning indexing performance. (DSP-12491)
Runtime node blacklisting for distributed search queries; the EndpointStateTracker MBean now has Blacklisted boolean attribute. (DSP-12965)

4.8.13 Resolved issues

DSE Search: Remove unnecessary Solr resource repair on core creation and reloading. (DSP-10270)
CQL Search queries time out when a column has a colon (:) in it. Solr field name policy applies to DSE Search field names. (DSP-11296)
DSE Search upgrade to 4.8.x with kerberos does not allow to reload core on node0. (DSP-11462)
DSE Search: Make UnInvertedField disabled by default. (DSP-12184)
DSE Search: Combining normal column + 2 UDT subfields search requires 'fq' syntax. (DSP-12347)
DSE Search: DistributedRequestException isn't created with a detail message. (DSP-12493)
DSE Search: CQL should show error on using unsupported column aliases with solr_query queries. (DSP-12500)
DSE Search: Heap is exhausted while search re-indexes very wide partitions. New IndexPool MBean attributes. (DSP-12547)
Improved error handling for authentication and authorization of Solr HTTP requests and Solr Admin UI. (DSP-12550)
Requests that fail due to lack of permissions return a 403 error, not a 401 error that was returned in earlier versions.
Passing credentials through DSE_USERNAME and DSE_PASSWORD is broken when dserc file exists. (DSP-12602)
DSE Search: Solr queries don't work with partition restriction that has a space. (DSP-12610)
HiveMetastore starts slow. (DSP-12754)
LDAP causes high CPU use. (DSP-12787)
DSE Search: Poor performance when searching with UDT sub-fields. (DSP-12812)
Better TTL logging. (DSP-12885)
DSE Search: Remove unnecessary database reads in the scheduled Solr document expiration process. (DSP-12927)
DSE Search: The TTL task is never de-scheduled. (DSP-12960)
Don't assume PlainTextAuthProvider when using cqlsh SOURCE command. (DSP-13077)
dsetool logs clear credentials on logs. (DSP-12985)
CassandraFSPlugin fails occasionally on startup, preventing the node from booting up. (DSP-13222)

4.8.13 Known issues

Default distributed keyspaces, such as system_auth, can have their replication factor reset when adding a new datacenter. (DSP-13987)
Unable to create Solr core using demo with Kerberos enabled. (DSP-14069)

4.8.13 Apache Cassandra™ changes

DataStax Enterprise 4.8.13 certifies Apache Cassandra™ 2.1.17 with additional production-certified Cassandra changes.

DSE 4.8.12

Important: DataStax recommends installing the latest patch release.

30 January 2017

4.8.12 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.17.1439
Apache Solr™ 4.10.3.1.1511
Hive Connector 0.2.19
Spark Connector 1.4.5

4.8.12 Changes and enhancements

Log the max allowed key length to identify whether the JCE extensions are installed. (DSP-11177)
Stored=true copy fields are not supported and result in warning in the logs. (DSP-11087)

4.8.12 Resolved issues

Default values for SPARK_WORKER_MEMORY and SPARK_WORKER_CORES should take SPARK_WORKER_INSTANCES into account. (DSP-8493)
Verbose errors/warnings in Spark shells (dse spark, dse spark-sql). (DSP-10261)
Nodetool uses cassandra-env.sh MAX_HEAP_SIZE value if set. (DSP-11285)
Add missing Tika 1.13 dependencies and replace unsafe version of Jackson. (DSP-11311)
Rebuild of node/dc some DSE made keyspaces have their replication strategy reset to default replication. (DSP-11381)
Search startup/reload performance issue resolved by turning off the reader and merge warming. (DSP-11498)
To keep segment warming, add mergedSegmentWarmer to solrConfig.xml.
dse cassandra-stop gives errors when using Spark mode. (DSP-11507)
Selecting binary fields and tuples doesn't work with single pass queries. (DSP-11511)
Excessive live docs filters cause slow queries. (DSP-11512)
dsetool create_core fails with Lock on opening new Searcher after unsuccessful dsetool reload_core. (DSP-11573)
Thrift server fails to start on multiDC environment if a DC is down. (DSP-11617)
solr_stress and CQL should work with Kerberos. (DSP-11650)
Tombstones are skipped and not indexed during repair. (DSP-11798)
Live docs query is not removed when reader is closed, causing excessive heap usage. (DSP-11819)
Note: Reverting to an earlier DSE release requires a full reindex.
Issue with Spark collection types with UDTs - SPARKC-363. (DSP-11991)
Secure upgrade failed with secure password: SolrStress run-benchmark read-cql failed with password: java.lang.RuntimeException: javax.security.auth.login.LoginException: No LoginModules configured for DseClient. (DSP-12057)
Solr query fails with "Lucene document does not contain unique field" when the unique key is not composite and is not stored. (DSP-12106)
Incorrect "range.facet.other" results for distributed range facet queries. (DSP-12198)
Core is not correctly unloaded on restarted nodes. (DSP-12434)
After providing wrong schema next reload fails with mappingVersion. (DSP-12908)
Weather sensor demo website not graphing all data values. (DSP-13041)
Solr is accepting HTTP requests before all cores have loaded. (DSP-13270)

4.8.12 Known issues

DataStax Enterprise is not compatible with Oracle Java SE Runtime Environment 7 or OpenJDK 7. Use Java 8 instead. (DSP-13260)
Default distributed keyspaces, such as system_auth, can have their replication factor reset when adding a new datacenter. (DSP-13987)

4.8.12 Apache Cassandra™ changes

DataStax Enterprise 4.8.12 certifies Apache Cassandra 2.1.16 with additional production-certified Cassandra changes.

DSE 4.8.11

Important: DataStax recommends installing the latest patch release.

1 November 2016

4.8.11 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.17.1428
Apache Solr™ 4.10.3.1.1402
Apache Spark™ 1.4.2.7
Hive Connector 0.2.18

4.8.11 Changes and enhancements

Make shard token range selections optionally visible to the client. (DSP-10506)
Allow the prepared statement cache size to be changed. (DSP-11420)

4.8.11 Resolved issues

dse cassandra-stop can return before cassandra exits. (DSP-8257)
Error InvalidRequestException thrown in spark-sql queries. (DSP-9159)
Spark app should not run when driver metadata persistence fails. (DSP-10324)
Delayed failure when creating core with RAMDirectoryFactory, because RAMDirectoryFactory and other non-persistent directory factories are unsupported features. (DSP-10379)
Lucene should do parallel delete processing for NRT and parallel creation of FrozenBufferedUpdates for RT. New indexConfig values. (DSP-10416)
service dse start timing out before JVM starts. (DSP-10478)
Make the error on tokenized unique key optional. (DSP-10657)
Preflight check is incorrect about vm.max_map_count. (DSP-10706)
Throw error for frozen collection in search. (DSP-10772)
Slow query log NPEs instead of logging slow statement. (DSP-10930)
Spark Master in EverywhereStrategy keyspace has availability issues. (DSP-10941)
SchemaTool does not handle empty bytebuffer values for timestamp/datetype. (DSP-11091)
Fixes for round robin assignment of drivers to Spark Workers. (DSP-11004)
cqlsh NoHostsAvailable/AuthenticationFailure when sourcing a file with COPY commands. (DSP-11111)
Solr unnecessarily reads stored fields in HTTP queries with an explicit field list. (DSP-11469)
Add TermsComponent back to stress solrconfig.xml. (DSP-11495)
nodetool startup script assumes JMX_PORT. (DSP-11590)

4.8.11 Apache Cassandra™ changes

DataStax Enterprise 4.8.11 certifies Apache Cassandra 2.1.16 with additional production-certified Cassandra changes.

DSE 4.8.10

Important: DataStax recommends installing the latest patch release.

31 August 2016

4.8.10 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.15.1423
Apache Spark™ 1.4.2.6
Apache Solr™ 4.10.3.1.1298
Hive Connector 0.2.16

4.8.10 Changes and enhancements

Standardize Solr metrics. (DSP-8566)
Better default threshold for Solr slow sub-query log. (DSP-9969)
Add time-based execution to solr_stress. (DSP-10059)
Maximum size of an uploaded Solr resource is configurable in dse.yaml. (DSP-10253)
Solr Stress supports password authentication and SSL for CQL. (DSP-10411)
Give custom field types control over the type of their serialized value. (DSP-10418)
Fixed mis-configured logging for spark-beeline. (DSP-10594)

4.8.10 Resolved issues

"select null" query does not work with SparkSQL Thriftserver, but does with spark-sql, shark, and hive. (DSP-6620)
NPE during reindexing non-live cell that is no longer part of CFMetaData. (DSP-8804)
Fix ulimit issues (DSP-9128)
Delegation token renewal does not work for DSE Spark applications. (DSP-9445)
Report configuration errors properly. (DSP-9605)
Fix the error when DC name has apostrophe/single quote. (DSP-9754)
logback-spark-server.xml has wrong fileNamePattern. (DSP-9890)
BYOH: dse hive-schema exception when rpc_address is 0.0.0.0 (DSP-9947)
Reduce logging on opening a new IndexSearcher. (DSP-9955)
DSE tarball installs do not use $CASSANDRA_HOME when data directory locations are not set in cassandra.yaml. (DSP-10034)
Security issues with Hadoop delegation tokens. (DSP-10082)
Add support for specifying consistency levels in solr_stress (DSP-10174)
Delegation token is exposed in processes list. (DSP-10227)
When using HiveContext with SSL in Spark, the keystore password gets copied from truststore password. (DSP-10286)
Reintroduce the Solr validation log configuration. (DSP-10334)
Solr query results in NPE because of sort in requestHandler. (DSP-10362)
Make sure Solr field validation errors are logged only to the validation log. (DSP-10388)
Properly check Hive connector version number. (DSP-10409)
Single threaded Lucene parallel delete causes Out of Memory error when Live Indexing is enabled. (DSP-10415)
Search failover handles multiple failure responses from a node incorrectly. (DSP-10454)
The dsetool should set storagedir when missing in configuration (DSP-10470)
Number of server query worker threads is hard coded. (DSP-10519)
Log peer in the event of a streaming error. (DSP-10678)
Native driver connections in dsetool aren't isolated to specified host. (DSP-12438)
Concurrent sorting issue with RT. (DSP-12600)

4.8.10 Apache Cassandra™ changes

DataStax Enterprise 4.8.10 certifies Apache Cassandra™ 2.1.15 with additional production-certified Cassandra changes.

DSE 4.8.9

Important: DataStax recommends installing the latest patch release.

1 July 2016

4.8.9 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.15.1403
Apache Solr 4.10.3.1.1170

4.8.9 Changes and enhancements

Additional logging for analyzing core loading performance. (DSP-6560)
DataStax Enterprise fails to start after upgrading Hadoop with Kerberos enabled. (DSP-8912)
Fix Solr Data Import Handler Deprecation Message. (DSP-10121)
Support for CentOs and RHEL 7.2 (DSP-10276)
Give custom field types control over the type of their serialized value. (DSP-10418)

4.8.9 Resolved issues

sstablemetadata estimated droppable tombstones does not take partition level tombstones into account (DSP-10335)
Duplicate documents caused by Tuple/UDT field updates. (DSP-10177)
SparkWorker error logs during shutdown. (DSP-10128)
Add a dse.yaml configuration option to avoid waiting for post-bootstrap reindexing so that the node is not marked down. (DSP-10067)
CVE 2015-5262 - Update http-client used in Spark streaming demo to 4.5.2. (DSP-10041)
Spark options in dse.yaml are not ignored even though SPARK_ENABLED=0. (DSP-10023)
Fixed CVE-2016-2175 - Apache PDFBox before 1.8.12 and 2.x before 2.0.1 does not properly initialize the XML parsers. (DSP-9975)
CVE-2014-4715 and CVE 2014-4611 - Driver vulnerability due to LZ4 and xxHash 1.2.0. (DSP-9910)
CVE 2015-3250 - Transitive dependency on Apache directory LDAP API Model 1.0.0-M24. (DSP-9909)
solr_stress does not work if the schema does not have default field. (DSP-9897)
Upgrade unsafe Apache Tika Version Solr pulls into DataStax Enterprise. (DSP-9811)
CVE-2014-0114 - Apache Commons BeanUtils. (DSP-9628)
Missing index updates after core failure. (DSP-8502)
Weather Sensor Demo throws SASL errors on CentOS. (DSP-8309)

4.8.9 Apache Cassandra™ changes

DataStax Enterprise 4.8.9 certifies Apache Cassandra™ 2.1.15 with additional production-certified Cassandra changes.

DSE 4.8.8

Important: DataStax recommends installing the latest patch release.

31 May 2016

4.8.8 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.14.1346
Apache Solr 4.10.3.1.1060
Apache Spark Connector 1.4.4

4.8.8 Changes and enhancements

DSE Analytics: Add dsetool cleanup_leases to cleanup old LeaderManager entries. (DSP-9647)
Improve multi-threaded processing of long running queries for search. (DSP-9685)
Improve security: CVE-2014-0114 (commons-beanutils-core). (DSP-9628)

4.8.8 Resolved issues

Fixed search core closing exceptions on node shutdown. (DSP-9738)
Rows with non-indexed UDT fields are not searchable. (DSP-9711)
Properly quote secondary index names for Solr cores. (DSP-9705)
Bound multithreaded deletes only with live indexing. (DSP-9620)
dsetool create_core should accept generateResources=false. (DSP-9555)
Error out on multiple 'q' for solr_queries. (DSP-9517)
Properly quote CQL statements when unloading search cores. (DSP-9345)
Change the log level of the slow query log. (DSP-9308)
Co-located Spark Master and cluster mode driver cannot fail over reliably. (DSP-9158)
Sqoop SSL encryption is broken for CQL imports. (DSP-8061)
dsetool create_core generateResources=false should not allow coreOptions. (DSP-7918)
Empty options file for resource generation causes NPE and better error handling. (DSP-7606)

4.8.8 Apache Cassandra™ changes

DataStax Enterprise 4.8.8 certifies Apache Cassandra™ 2.1.14 with additional production-certified Cassandra changes.

DSE 4.8.7

Important: DataStax recommends installing the latest patch release.

6 May 2016

4.8.7 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.14.1272
Apache Solr 4.10.3.1.1021
Hive Connector 0.2.14
Spark Cassandra Connector 1.4.3

4.8.7 Changes and enhancements

Add version of Spark job server and Spark connector to system.log/output.log. (DSP-9355)
Improved Solr stress capabilities and metrics reporting. Merge geo names loader into Solr stress with improvements to geonames. (DSP-8602)
Clarify resource generation when static resources are specified in dsetool. If generateResources=true and resources already exist, then fail gracefully. (DSP-7799)

4.8.7 Resolved issues

Overly granular node health scores isolate request submission to too few replicas. (DSP-9512)
TTL collector shouldn't read whole rows from Cassandra. (DSP-9446)
Improve the dynamic field warning for incorrectly stored map data in Cql3CassandraRowReader. (DSP-9394)
Cannot geodist() sort with RPT fields on multi-node clusters. (DSP-9392)
Spark history server creates extra system.log. (DSP-9364)
Hive create external table location keyword does not work. (DSP-9359)
CFS keyspace warning about phantom datacenter DC1 or UNKNOWN-DC. (DSP-9246)
Copy field returns no result if the source field is non-indexed. (DSP-9157)
NPE When spark_master_recovery table does not exist. (DSP-9140)
Spark History Server can't start when auth is enabled. (DSP-9134)
Spark workers will hang if can't reach consistency level for Cassandra read/write. (DSP-9061)
NullPointerException in Spark SQL Thriftserver. (DSP-8647)
Spark Master/Worker can shut down the entire node. (DSP-8324)
Multiple CQL Solr query fq clauses should work. (DSP-7975)
The install-jts.sh script doesn't work. You must manually install the JTS Topology Suite. (DSP-7403)
Node reaches NORMAL status before internode transport is ready. (DSP-7331)
Error out on unrecognized facet parameters. (DSP-7212)

4.8.7 Apache Cassandra™ changes

DataStax Enterprise 4.8.7 certifies Apache Cassandra™ 2.1.14 with additional production-certified Cassandra changes.

DSE 4.8.6

1 April 2016

4.8.6 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.13.1218 with additional production-certified Cassandra changes
Hive Connector 0.2.13
Apache Solr 4.10.3.1.415
Apache Spark 1.4.2.4
Apache Tomcat 6.0.45

4.8.6 Changes and enhancements

Support for the Solr time zone (TZ) parameter for search queries with JSON. (DSP-8377)
TomcatSolrRunner should get the cipher suites for SSL connections from cassandra.yaml. (DSP-8370)
Make EverywhereStrategy return the correct primary endpoint from calculateNaturalEndpoints. (DSP-8507)

4.8.6 Resolved issues

Remove DataStax Enterprise dependency on OpenJDK. (DSP-4934)
bulkCassandraWriter failed in retry tasks when directory already exists. (DSP-5234)
Wrong timeout value logged following flush timeout exception. (DSP-6971)
Netty EventExecutorGroup shutdown can occur before channel shutdown. (DSP-8074)
Support all inter-node (node-to-node) encryption options in DSE Search legacy netty transport. Use server_encryption_options properties in cassandra.yaml. (DSP-8374)
Slow start of spark-sql-thriftserver due to reconnection in SchemaManagerService. (DSP-8470)
Named parameters in simple statements don't work. (DSP-8495)
Allow core admin to direct user to use dsetool create core. (DSP-8518)
LeaderManager should ignore decommissioned nodes. (DSP-8532)
Node should exit when plugin fails to start (deadlock in PluginManager.activate/deactivate). (DSP-8648)
Do not log passwords in log files or show in Spark Web UI. Environment variables DSE_USERNAME and DSE_PASSWORD are supported for all Spark commands. Only DataStax Enterprise user has permissions (600) to the log file created by Spark SQL Thrift Server. (DSP-8674)
Services Only and Services Start doesn't work in Mac standalone installer. (DSP-8710)
Can't set SPARK_DAEMON_MEMORY that is used by Spark-Thriftserver and Spark history server. (DSP-8760)
SolrJ-Auth dependencies are now publicly available on the datastax-public-releases-local repository. (DSP-8798)
dsetool search commands work with hostname and IP. (DSP-8832)
Avoid reading from Cassandra during first phase of CQL Solr Query without lazy field loading. (DSP-8833)
Solr rounds the timestamp key to second if used as primary key when distributed. (DSP-8879)
Document deletion fails due to incorrect escaping of Solr special characters in unique key. (DSP-8884)
Make dsetool search commands work with SSL + Kerberos. (DSP-8896)
Fix duplicate results from Solr query. (DSP-8936)
NoSuchElementException key not found due to concurrency bug in Spark Thriftserver. (DSP-9031)

4.8.6 Apache Cassandra™ changes

DataStax Enterprise 4.8.6 certifies Apache Cassandra™ 2.1.13 with additional production-certified Cassandra changes.

DSE 4.8.5

29 February 2016

4.8.5 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.13.1131 with additional production-certified Cassandra changes
Netty 4.0.34.Final
Spark Cassandra Connector 1.4.2
Apache Solr 4.10.3.1.358

4.8.5 Changes and enhancements

Configure the uptime required for a node to reach full health. (DSP-6883)
Certified with Centos 6.7. (DSP-7852)
Provide user credentials in a configuration file for use with dse and dsetool. (DSP-8128)
Enable some dsetool commands to work with Cassandra password authentication. User credentials can be provided in several ways, see Providing credentials for authentication. (DSP-8251)

4.8.5 Resolved issues

Throw error if Solr schema unique key has a tokenized field. (DSP-5431)
dse.in.sh should set CASSANDRA_LOG_DIR only if it isn't already set. (DSP-5514)
Solr slow sub-query request modification consumes POST content. (DSP-6609)
Can't select inet/varint type in spark-beeline when there is a null inet/varint collection in the table. (DSP-7002)
Print out warning or error message when kerberos keytab file is not readable. (DSP-7296)
Solr and demo apps should be served from CATALINA_HOME. (DSP-7414)
Tarball installs: resources/cassandra/bin/nodetool fails if DSE_HOME is set. (DSP-7445)
Add a warning after dsetool create_core if reindex=true is not specified. (DSP-7631)
TDseClientTransportFactory is hard coded for cassandra.thrift.framed.size_mb. Respect the thrift_framed_transport_size_in_mb value in Cassandra.yaml. (DSP-7719)
No milliseconds in stdout log appender for Cassandra. (DSP-7827)
Bootstrapping a new data center reverts dse_perf keyspace to SimpleStrategy. (DSP-7845)
The dsetool utility does not have argument for non-default Cassandra port. (DSP-7856)
dsetool core_indexing_status does not accept hostname (accepts only IP) (DSP-7985)
Fix live indexing sorted terms seek and iteration. (DSP-8009)
dsetool ring reports clusters as using vnodes when they aren't. (DSP-8148)
EverywhereStrategy should calculate natural endpoints in sorted order. (DSP-8023)
Spark job has retries infinitely, RDD partitions out of order, related to CassandraRDDPartitioner violates Partition Contract SPARKC-323. (DSP-8033)
Fix example JMX remote password file location in cassandra-env.sh. (DSP-8245)
Cannot query non-stored copy field with tuple sub-field source. (DSP-8471)
FuzzySet assertion for large segments. (DSP-8479)
Stop optimizing in the commit that occurs at the end of re-indexing a core. (DSP-8613)

4.8.5 Apache Cassandra™ changes

DataStax Enterprise 4.8.5 certifies Apache Cassandra™ 2.1.13 with additional production-certified Cassandra changes.

DSE 4.8.4

13 January 2016

4.8.4 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.12.1046
Apache Solr 4.10.3.1.287
Hive Connector 0.2.11
Netty 4.0.33.dse

4.8.4 Changes and enhancements

Enable the Spark history server. (DSP-4532)
Support for CentOS 7.1. (DSP-7548)

4.8.4 Resolved issues

dse.in.sh silently fails if not all debian packages are installed. (DSP-5322)
Incremental repair fails on large LCS column families. (DSP-7124)
Solr/Lucene versions information is missing in Solr Admin UI. (DSP-7234)
Re-index doesn't correctly page through wide partitions. (DSP-7591)
Fix passing security credentials for new style of start/stop Spark SQL Thrift Server. (DSP-7644)

4.8.4 Apache Cassandra™ changes

DataStax Enterprise 4.8.4 certifies Apache Cassandra™ 2.1.12 with additional production-certified Cassandra changes.

DSE 4.8.3

9 December 2015

4.8.3 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.11.969
Apache Spark 1.4.2.2
Apache Solr 4.10.3.1.263
Spark Connector 1.4.1

4.8.3 Changes and enhancements

New dsetool write_resource and read_resource commands support uploading extra DSE Search resource files. (DSP-7011)
Global scheduler for running Solr TTLs. (DSP-7275)
To manage system resource consumption and prevent many Solr cores from executing simultaneous TTL deletes, a new thread_pool_size option in dse.yaml sets the maximum number of threads that concurrently execute TTL deletes.

4.8.3 Resolved issues

Search Solr NPE in Lucene TermHandler. (DSP-6908)
Fix consistency for start/stop spark-sql-thriftserver dse commands. (DSP-6922)
Removed byoh-env.sh: PIG_PARTITIONER='dsetool partitioner' call to speed up BYOH start. (DSP-6962)
After creating Thrift table for data import handler, unable to create valid Solr core. (DSP-6982)
Conversion of unordered docids into bitsets for a Solr join query. (DSP-7021)
Spark Jobserver does not start on package installs. (DSP-7052)
Upgrade Apache HttpComponents library to 4.4.1. (DSP-7194)
Gossip race condition causing unavailable ranges during queries. (DSP-7230)
Cannot write UDT types from Pyspark using DFs. (DSP-7235)
User latency tracking no longer performs a reverse DNS lookup. (DSP-7252)
Exclude hadoop-client lib from Spark. (DSP-7253)
Various streaming errors with out-of-order keys. (DSP-7424)
If you upgraded to 4.8.1 or 4.8.2 and started receiving streaming errors similar to the following errors, you must scrub the affected table after upgrading. These errors are triggered when range tombstones are streamed incorrectly, and can prevent critical operations like repair and bootstrapping from completing.
```
WARN [STREAM-IN-/1.2.3.4] 2015-11-26 10:51:35,930 StreamSession.java:625 - [Stream #481a5680-9423-11e5-bd1f-75f3cea48b3b] Retrying for following error java.lang.RuntimeException: Last written key DecoratedKey(XXX, YYY) >= current key DecoratedKey(-ZZZ, QQQ) writing into /var/lib/cassandra/data/ks/cf/ks-cf-tmp-ka-4-Data.db
```

4.8.3 Apache Cassandra™ changes

DataStax Enterprise 4.8.3 certifies Apache Cassandra™ 2.1.11 with additional production-certified Cassandra changes.

DSE 4.8.2

Warning: DataStax does not recommend 4.8.1 or 4.8.2 versions for production, see warning. Use 4.8.3 instead. Various streaming errors with out-of-order keys. (DSP-7424)

If you upgraded to 4.8.1 or 4.8.2 and started receiving streaming errors similar to the following errors, you must scrub the affected table after upgrading. These errors are triggered when range tombstones are streamed incorrectly, and can prevent critical operations like repair and bootstrapping from completing.

WARN [STREAM-IN-/1.2.3.4] 2015-11-26 10:51:35,930 StreamSession.java:625 - [Stream #481a5680-9423-11e5-bd1f-75f3cea48b3b] Retrying for following error java.lang.RuntimeException: Last written key DecoratedKey(XXX, YYY) >= current key DecoratedKey(-ZZZ, QQQ) writing into /var/lib/cassandra/data/ks/cf/ks-cf-tmp-ka-4-Data.db

10 November 2015

4.8.2 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.11.908 with production-certified Cassandra changes
Apache Thrift 0.9.3
Hive Connector 0.2.9

4.8.2 Resolved issues

Error loading class 'solr.FastLRUCache' in search. (DSP-4702)
Dynamic field marker does not have per document ttl applied to it in Thrift tables. (DSP-6886)
Reload core using dse tool does not update the solr_config file with coreOptions. (DSP-6899)
Better dsetool core name error handling. (DSP-6953)
Possible out of memory error from malformed Thrift protocol messages. (DSP-7019)
CQLSH COPY command could fail while reading timestamp options. (DSP-7045)

4.8.2 Apache Cassandra™ changes

DataStax Enterprise 4.8.2 certifies Apache Cassandra™ 2.1.11 with additional production-certified Cassandra changes.

DSE 4.8.1

Warning: DataStax does not recommend 4.8.1 or 4.8.2 versions for production, see warning. Use 4.8.3 instead. Various streaming errors with out-of-order keys. (DSP-7424)

If you upgraded to 4.8.1 or 4.8.2 and started receiving streaming errors similar to the following errors, you must scrub the affected table after upgrading. These errors are triggered when range tombstones are streamed incorrectly, and can prevent critical operations like repair and bootstrapping from completing.

WARN [STREAM-IN-/1.2.3.4] 2015-11-26 10:51:35,930 StreamSession.java:625 - [Stream #481a5680-9423-11e5-bd1f-75f3cea48b3b] Retrying for following error java.lang.RuntimeException: Last written key DecoratedKey(XXX, YYY) >= current key DecoratedKey(-ZZZ, QQQ) writing into /var/lib/cassandra/data/ks/cf/ks-cf-tmp-ka-4-Data.db

23 October 2015

4.8.1 Components

The follow components are changed from the previous version of DataStax Enterprise:

Apache Cassandra™ 2.1.11.872 with production-certified Cassandra changes
Apache Solr 4.10.3.1.204
Apache Spark 1.4.1.3
Apache Tomcat 6.0.44

4.8.1 Changes and enhancements

Sqoop import and export jobs support alternate connection managers. (DSP-5974)
Support interval faceting via CQL. (DSP-6804)

4.8.1 Resolved issues

SOLR created fields like _docBoost do not honor row TTLs - blocking TTL of row. (DSP-4872)
dse script hard codes ulimits during startup. (DSP-5713)
NullPointerException resulting from standard CQL query on Solr field. (DSP-6396)
Add package dependencies for ‘which’. (DSP-6459)
Sqoop cql-import fails with StackOverflowError when too many columns requested. (DSP-6528)
Portfolio demo: HistLoss table created by Hive is wrong (Spark okay). (DSP-6621)
FuzzySet integer min value causes assertion bug. (DSP-6658)
ConcurrentModificationException when using live indexing. (DSP-6717)
Misleading error message when attempting to unload non-existing core. (DSP-6722)
Solr slow sub-query log doesn't report prepare and process times for CQL Solr queries. (DSP-6724)
Ensure hard commit is executed immediately after re-indexing. (DSP-6735)
Indentation error in the Pyspark Streaming Distribution. (DSP-6739)
Disabling CqlSlowQueryLog service is not properly logged. (DSP-6760)
Unable to use multiple fq in a CQL Solr query. (DSP-6763)
Spark Streaming checkpointing doesn't work with internal authentication. (DSP-6776)
Exceptions in leader manager are swallowed and silently disable leader manager. (DSP-6786)
Support interval faceting via CQL. (DSP-6804)
Tuple docs are deleted twice. (DSP-6806)
Solr CQL queries fail with non-compound timestamp keys. (DSP-6810)
Installer needs to symlink /usr/bin/dse-client-tool. (DSP-6851)
-Xmn should not be set when using G1GC in cassandra-env.sh. (DSP-6860)
Disallow setting Solr concurrency level at 1 via JMX. (DSP-6868)
SSL + Kerberos qop=auth-int or auth-conf not reported as warning. (DSP-6939)

4.8.1 Apache Cassandra™ changes

DataStax Enterprise 4.8.1 certifies Apache Cassandra™ 2.1.11 with additional production-certified Cassandra changes.

DSE 4.8

23 September 2015

4.8 Components

Apache Cassandra™ 2.1.9.791
Apache Hadoop 1.0.4.18
Apache Hive 0.12.0.11
Apache Mahout 0.8
Apache Pig 0.10.1
Apache Solr 4.10.3.1.172
Apache Spark 1.4.1.1
Spark Jobserver 0.5.2
Apache Sqoop 1.4.5.15.1
Apache Tomcat 6.0.39
Java Driver for Apache Cassandra 2.1.7.1
Hive Connector 0.2.8
Netty 4.0.23.Final
Spark Connector 1.4.0

4.8 Changes and enhancements

DataStax Enterprise 4.8 introduces the following changes in addition to the new features.

Delete (unload_core) a Solr index without deleting/dropping the underlying keyspace and table. (DSP-1533)
DSE Search supports indexing and querying of Cassandra tuples and user defined types (UDTs). (DSP-3748)
Support SELECT count() in CQL Solr queries. (DSP-4843)
Solr Data Import Handler is deprecated. Use parquet or JSON instead. For Spark earlier than 1.4, use the DataReader API. (DSP-4887)
Support dynamic field Solr LatLonType with spatial subfield prefix naming conventions. (DSP-5021)
Enable slow query log collection by default, change dse.yaml options and default values. (DSP-5211)
DSE Search supports advanced spatial queries for polygon shapes. Use installation script for JTS. (DSP-5320)
Audit filter in dse.yaml can filter on keyspaces using a regular expression. (DSP-5407)
Restrict HTTP and Solr Admin access to DSE Search. Create a Tomcat connector or use rpc_address in cassandra.yaml. (DSP-5440)
Audit log tables use DateTieredCompactionStrategy (DTCS). (DSP-5462)
DataStax recommends changing tables that were created in earlier releases to use DTCS:
```
ALTER TABLE dse_audit.audit_log WITH COMPACTION={'class':'DateTieredCompactionStrategy'};
```
Implement updateDocuments for block join in live indexing. (DSP-5506)
Node health change and new commands to inspect node health and indexing status. (DSP-5508)
- Node health options are always enabled. You can customize the refresh rate for node_health_options in dse.yaml.
- Use the dsetool node_health command to retrieve a dynamic score between 0 and 1 that describes the health of a DataStax Enterprise node.
- Use the dsetool core_indexing_status command to retrieve the dynamic indexing status (INDEXING, FINISHED, or FAILED) of the specified core in a DSE Search node.
The CassandraStorage() handler is deprecated. Use CqlNativeStorage() instead. Thrift-based Hadoop classes that are not Java-driver based are deprecated. (DSP-5750)
Support for Shark is removed. Use Spark SQL instead. (DSP-5824)
Automatically insert CQLSearchHandler into solrconfig.xml. (DSP-5831)
Integrate Spark Jobserver. (DSP-5827)
The configuration of credentials for the Spark SQL Thrift Server is controlled by the hive-site.xml file in the Spark directory. (DSP-5927)
DseSparkConfHelper, DseSparkContext, DseStreamingContext are deprecated. DataStax Enterprise configuration is applied with scripts at startup. Using Spark requires no modifications. (DSP-5987)
Reduce search index size by providing a way to disable DSE from adding _partitionKey field to Solr documents. (DSP-6021)
The dse client-tool supports basic operations that are required to run client applications. (DSP-6025)
- In the authenticated cluster, provide Cassandra credentials instead of JMX username and password.
- The dsetool sparkmaster command is deprecated, and is replaced by the dse client-tool spark master-address command.
- Replaces the dsetool jobtracker command.
The multi-threaded indexing back pressure algorithm provides improved performance, reliability, and visibility. (DSP-6103)
DSE PySpark Scala wrappers are deprecated. PySpark and DSE PySpark are still supported using the more efficient DataFrames API. (DSP-6203).
BYOH Hive is deprecated and will be removed in a future release. (DSP-6337)
Integrate Spark 1.4. (DSP-6356)
You can use an ODBC or JDBC client to interact with the Spark SQL Thrift server. When reading and writing large amounts of data, DataStax recommends using the Cassandra-backed DataFrames. (DSP-6385)
Parameter name change for the Spark Connector property: spark.cassandra.input.page.row.size is renamed to spark.cassandra.input.fetch.size_in_rows.
G1GC Java Garbage Collector is enabled by default when DataStax Enterprise runs under Java 8. (DSP-6392)
In DSE Search, Solr deep paging is off by default and is configurable. (DSP-6400)
Set the cql_solr_query_paging option in the dse.yaml file as appropriate for the workload.
- For development clusters in mixed workloads, DataStax recommends using the default cql_solr_query_paging: off setting to turn paging off. This cql_solr_query_paging: off setting is helpful for a mixed workload cluster where normal search queries are run.
- For SearchAnalytics nodes, DataStax recommends using the cql_solr_query_paging: driver setting. This cql_solr_query_paging: driver setting is helpful when analytics nodes leverage search.
Verify all unique key elements are indexed in Solr schema. (DSP-6460)
HiveServer2 authentication changes in a secured cluster. (DSP-6508)
Deleted doc might show in a DSE Search query when using facet tagging and excluding filters or query result cache is on. (DSP-6529)
Improve indexing performance by using multi-thread indexing for wide rows. (DSP-6592)
The default log directory is changed to $HOME/spark-thrift-server for starting the spark-thrift-server. (DSP-6720)

4.8 Resolved issues

CqlBulkOutputFormat is broken by upgrade to Cassandra 2.1. (DSP-4875)
Spark executors use cassandra.yaml and dse.yaml to get information about SSL and Kerberos. (DSP-5576)
Bootstrapping TDE nodes try to read from themselves. (DSP-5701)
The dse startup script hard codes ulimits for the cassandra user. (DSP-5713)
Permissions set by the DSE/Cassandra installation process on the /var/lib/cassandra directory are 750 by default. Permissions are now 751. (DSP-5970)
Remove query docFreq lookup for filters. (DSP-5982)
WAIT_FOR_START=10 is too low on an m1.large instance type. (DSP-6077)
Hive connector uses yaml files and property names that are not DSE-compatible. (DSP-6042)
DSE Spark --jars flag does not work. (DSP-6087)
The value of load_max_time_per_core is too low by default. (DSP-6260)
Range tombstones might cause unwanted deletes from the index. (DSP-6301)
NullPointerException thrown in Solr when enabling live indexing. (DSP-6336)
Remove unnecessary contention and improve live indexing reader latency by removing usage of RAMFile for deletes. (DSP-6362)
Failure to close key iterator could lead to reference leaks. (DSP-6377)
Cannot set max_solr_concurrency_per_core at 1. (DSP-6408)
Single pass queries do not work with UUID, TimeUUID, Inet, Decimal, and Varint types. (DSP-6443)
"_docBoost" prevents docs from being indexed. (DSP-6486)
enable_back_pressure_adaptive_nrt_commit is not set to true by default. (DSP-6453)
Changes in spark-env.sh do not get picked up properly after restart. (DSP-6502)
Portfolio manager web interface broken. (DSP-6593)
Solr re-index progress does not show in OpsCenter Activity. (DSP-6613)
Allow for configuring the NIO connector via Tomcat server.xml. (DSP-6645)
The BIO connector is more performant. However, you can revert back to NIO by setting the protocol to org.apache.coyote.http11.Http11NioProtocol in the Tomcat server.xml file under ~/dse/resources/tomcat/conf:
```
<Connector port="${http.port}" protocol="org.apache.coyote.http11.Http11NioProtocol"
connectionTimeout="20000" 
redirectPort="8443" URIEncoding="UTF-8"/>
```
After you start DataStax Enterprise, you can verify the NIO connector with these lines in the Tomcat Catalina log file:
```
Sep 14, 2015 11:22:39 PM org.apache.coyote.http11.Http11NioProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8983
Sep 14, 2015 11:22:41 PM org.apache.coyote.http11.Http11NioProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8983
```
Top deletion might cause unwanted deletes from the index. (DSP-6654)
2GB SSTable limitation for In-Memory table per node. (DSP-6685)

4.8 Known issues

DSE Search: Setting Solr concurrency by JMX causes deadlock with index flushing. (DSP-6772)
If you use JMX to set IndexPool Concurrency, a deadlock occurs for index workpool flushing. Indexing stops until the time that is set for flush_max_time_per_core is reached.

Workaround: Do not use JMX to set IndexPool Concurrency. If you need to change IndexPool Concurrency, edit the dse.yaml file to change max_solr_concurrency_per_core and restart the node.
To improve index performance, set production appropriate mergeScheduler values for DSE Search with near real time (NRT) indexing. See Configuring and tuning indexing performance. (DSP-9325)

4.8 Apache Cassandra™ changes

DataStax Enterprise 4.8 certifies Apache Cassandra™ 2.1.9 with additional production-certified Cassandra changes.