DataStax Enterprise 4.7 release notes

DataStax Enterprise release notes cover cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues for DataStax Enterprise 4.7 releases.

DataStax Enterprise release notes cover cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues for DataStax Enterprise 4.7 releases:

Requirement for Uniform Clusters 

All Nodes in each Cluster must be uniformly licensed to use the same Subscription. For example, if a Cluster contains 5 Nodes, all 5 Nodes within that Cluster must be either DSE Standard, or all 5 Nodes must be DSE Max. Mixing different Subscriptions within a Cluster is not permitted. “Cluster” means a collection of Nodes running the Software which communicate with one another via Gossip, and “Gossip” means the mechanism within the Software enabling related Nodes to communicate with one another. For more information, see Enterprise Terms.

Before you upgrade 

The latest version of DataStax Enterprise 4.7 is 4.7.9.

4.7.9 Release notes for DataStax Enterprise 

1 August 2016

4.7.9 Component changes
  • Apache Cassandra™ 2.1.15.1416
  • Apache Solr 4.10.3.0.1222
  • Apache Spark 1.2.2.5
  • Hive Connector 0.1.25
4.7.9 Changes and enhancements
  • Clarify resource generation when static resources are specified in dsetool. If generateResources=true and resources already exist, then fail gracefully. (DSP-7799)
  • DSE Analytics: Add dsetool cleanup_leases to cleanup old LeaderManager entries. (DSP-9647)
  • Improve multi-threaded processing of long running queries for search. (DSP-9685)
  • Better default threshold for Solr slow sub-query log. (DSP-9969)
  • Add a dse.yaml configuration option to avoid waiting for post-bootstrap reindexing so that the node is not marked down. (DSP-10067)
  • Maximum size of an uploaded Solr resource is configurable in dse.yaml. (DSP-10253)
4.7.9 Resolved issues
  • "select null" query does not work with SparkSQL Thriftserver, but does with spark-sql, shark, and hive. (DSP-6620)
  • Error out on unrecognized facet parameters. (DSP-7212)
  • Node reaches NORMAL status before internode transport is ready. (DSP-7331)
  • Empty options file for resource generation causes NPE and better error handling. (DSP-7606)
  • dsetool create_core generateResources=false should not allow coreOptions. (DSP-7918)
  • Multiple CQL Solr query fq clauses should work. (DSP-7975)
  • Sqoop SSL encryption is broken for CQL imports. (DSP-8061)
  • Co-located Spark Master and cluster mode driver cannot fail over reliably. (DSP-9158)
  • CFS keyspace warning about phantom datacenter DC1 or UNKNOWN-DC. (DSP-9246)
  • Cannot geodist() sort with RPT fields on multi-node clusters. (DSP-9392)
  • Improve the dynamic field warning for incorrectly stored map data in Cql3CassandraRowReader. (DSP-9394)
  • Delegation token renewal does not work for DSE Spark applications. (DSP-9445)
  • TTL collector shouldn't read whole rows from Cassandra. (DSP-9446)
  • Overly granular node health scores isolate request submission to too few replicas. (DSP-9512)
  • Error out on multiple 'q' for solr_queries. (DSP-9517)
  • dsetool create_core should accept generateResources=false. (DSP-9555)
  • Report configuration errors properly. (DSP-9605)
  • CVE-2014-0114 - Apache Commons BeanUtils. (DSP-9628)
  • Properly quote secondary index names for Solr cores. (DSP-9705)
  • Fix the error when DC name has apostrophe/single quote. (DSP-9754)
  • Upgrade unsafe Apache Tika version that Solr pulls into DSE. (DSP-9811)
  • logback-spark-server.xml has wrong fileNamePattern. (DSP-9890)
  • Make recover throws a SolrException if index doesn't exist. (DSP-9891)
  • CVE 2015-3250 - Transitive dependency on Apache directory LDAP API Model 1.0.0-M24. (DSP-9909)
  • CVE-2014-4715 and CVE 2014-4611 - Driver vulnerability due to LZ4 and xxHash 1.2.0. (DSP-9910)
  • Reduce logging on opening a new IndexSearcher. (DSP-9955)
  • Fixed CVE-2016-2175 - Apache PDFBox before 1.8.12 and 2.x before 2.0.1 does not properly initialize the XML parsers. (DSP-9975)
  • Spark options in dse.yaml are not ignored even though SPARK_ENABLED=0. (DSP-10023)
  • DSE tarball installs do not use $CASSANDRA_HOME when data directory locations are not set in cassandra.yaml. (DSP-10034)
  • CVE 2015-5262 - Update http-client that is used in Spark streaming demo to 4.5.2. (DSP-10041)
  • SparkWorker error logs during shutdown. (DSP-10128)
  • Properly check Hive connector version number. (DSP-10409)
  • When using HiveContext with SSL in Spark, the keystore password gets copied from truststore password. (DSP-10286)
  • NPE in ThriftPluginServer.java during DSE shutdown. (DSP-10331)
  • Reintroduce the Solr validation log configuration. (DSP-10334)
  • Spark SSL doesn't work in Spark. (DSP-10350)
  • Solr query results in NPE because of sort in requestHandler. (DSP-10362)

4.7.9 Apache Cassandra™ changes

DataStax Enterprise 4.7.9 certifies Cassandra™ 2.1.15 with additional production-certified Cassandra changes.

4.7.8 Release notes for DataStax Enterprise 

April 2016

4.7.8 Component changes
  • Apache Cassandra™ 2.1.13.1218 with additional production-certified Cassandra changes
  • Apache Solr 4.10.3.0.418
  • Apache Tomcat 6.0.45
  • Hive Connector 0.1.23
  • Netty 4.0.34.Final
4.7.8 Changes and enhancements
  • Configure the uptime required for a node to reach full health. (DSP-6883)
  • Throw error if Solr schema unique key has a tokenized field. (DSP-5431)
  • Add a warning after dsetool create_core if reindex=true is not specified. (DSP-7631)
  • Fix live indexing sorted terms seek and iteration. (DSP-8009)
  • Provide user credentials in a configuration file for use with dse and dsetool. (DSP-8128)
  • TomcatSolrRunner should get the cipher suites for SSL connections from cassandra.yaml. (DSP-8370)
  • Support for the Solr time zone (TZ) parameter for search queries with JSON. (DSP-8377)
  • Make EverywhereStrategy return correct primary endpoint from calculateNaturalEndpoints. (DSP-8507)
4.7.8 Resolved issues
  • dse.in.sh should set CASSANDRA_LOG_DIR only if it isn't already set. (DSP-5514)
  • Solr slow sub-query request modification consumes POST content. (DSP-6609)
  • Solr and demo apps should be served from CATALINA_HOME. (DSP-7414)
  • Tarball installs: resources/cassandra/bin/nodetool fails if DSE_HOME is set. (DSP-7445)
  • TDseClientTransportFactory is hard coded for cassandra.thrift.framed.size_mb. Respect the thrift_framed_transport_size_in_mb value in cassandra.yaml. (DSP-7719)
  • Bootstrapping a new data center reverts dse_perf keyspace to SimpleStrategy. (DSP-7845)
  • The dsetool utility does not have argument for non-default Cassandra port. (DSP-7856)
  • EverywhereStrategy should calculate natural endpoints in sorted order. (DSP-8023)
  • Netty EventExecutorGroup shutdown can occur before channel shutdown. (DSP-8074)
  • Fix example JMX remote password file location in cassandra-env.sh. (DSP-8245)
  • Support all inter-node (node-to-node) encryption options in DSE Search legacy netty transport. Use server_encryption_options properties in cassandra.yaml. (DSP-8374)
  • Slow start of spark-sql-thriftserver due to reconnection in SchemaManagerService. (DSP-8470)
  • FuzzySet assertion for large segments. (DSP-8479)
  • Named parameters in simple statements don't work. (DSP-8495)
  • Missing index updates after core failure. (DSP-8502)
  • LeaderManager should ignore decommissioned nodes. (DSP-8532)
  • Stop optimizing in the commit that occurs at the end of reindexing a core. (DSP-8613)
  • Node should exit when plugin fails to start (deadlock in PluginManager.activate/deactivate). (DSP-8648)
  • Avoid reading from Cassandra during first phase of CQL Solr query without lazy field loading. (DSP-8833)
  • Solr rounds the timestamp key to second if used as primary key when distributed. (DSP-8879)
  • Document deletion fails due to incorrect escaping of Solr special characters in unique key. (DSP-8884)
  • Make dsetool search commands work with SSL + Kerberos. (DSP-8896)
  • Fix duplicate results from Solr query. (DSP-8936)

4.7.8 Apache Cassandra™ changes

DataStax Enterprise 4.7.8 certifies Apache Cassandra™ 2.1.13 with additional production-certified Cassandra changes.

4.7.7 Release notes for DataStax Enterprise 

13 January 2016

4.7.7 Component changes
  • Apache Cassandra 2.1.12.1046
  • Netty 4.0.33.dse
4.7.7 Resolved issues
  • Reindex doesn't correctly page through wide partitions. (DSP-7591)
  • Incremental repair fails on large LCS column families. (DSP-7124)

4.7.7 Certified Cassandra changes

DataStax Enterprise 4.7.7 certifies Cassandra with additional production-certified Cassandra changes.

4.7.6 Release notes for DataStax Enterprise 

9 December 2015

4.7.6 Component changes
  • Apache Cassandra 2.1.11.969
  • Apache Solr 4.10.3.0.262
4.7.6 Resolved issues
  • Search Solr NPE in Lucene TermHandler. (DSP-6908)
  • Removed byoh-env.sh: PIG_PARTITIONER='dsetool partitioner' call to speed up BYOH start. (DSP-6962)
  • After creating Thrift table for data import handler, unable to create valid Solr core. (DSP-6982)
  • Conversion of unordered docids into bitsets for a Solr join query. (DSP-7021)
  • Gossip race condition causing unavailable ranges during queries. (DSP-7230)
  • User latency tracking no longer performs a reverse DNS lookup. (DSP-7252)
  • Exclude hadoop-client lib from Spark. (DSP-7253)
  • Remove unnecessary resources/shark/lib/httpclient-4.3.1.jar. (DSP-7254)
  • Various streaming errors with out-of-order keys. (DSP-7424)
    If you upgraded to 4.7.4 or 4.7.5 and started receiving streaming errors similar to the following errors, you must scrub the affected table after upgrading. These errors are triggered when range tombstones are streamed incorrectly, and can prevent critical operations like repair and bootstrapping from completing.
    WARN  [STREAM-IN-/1.2.3.4] 2015-11-26 10:51:35,930  StreamSession.java:625 - [Stream #481a5680-9423-11e5-bd1f-75f3cea48b3b] Retrying for following error
java.lang.RuntimeException: Last written key DecoratedKey(XXX, YYY) >= current key DecoratedKey(-ZZZ, QQQ) writing into /var/lib/cassandra/data/ks/cf/ks-cf-tmp-ka-4-Data.db

4.7.6 Certified Cassandra changes

DataStax Enterprise 4.7.6 certifies Cassandra with additional production-certified Cassandra changes.

4.7.5 Release notes for DataStax Enterprise 

Warning: DataStax does not recommend this version for production. Use DataStax Enterprise 4.7.6 or later instead. Various streaming errors with out-of-order keys. (DSP-7424)
If you upgraded to 4.7.4 or 4.7.5 and started receiving streaming errors similar to the following errors, you must scrub the affected table after upgrading. These errors are triggered when range tombstones are streamed incorrectly, and can prevent critical operations like repair and bootstrapping from completing.
WARN  [STREAM-IN-/1.2.3.4] 2015-11-26 10:51:35,930  StreamSession.java:625 - [Stream #481a5680-9423-11e5-bd1f-75f3cea48b3b] Retrying for following error
java.lang.RuntimeException: Last written key DecoratedKey(XXX, YYY) >= current key DecoratedKey(-ZZZ, QQQ) writing into /var/lib/cassandra/data/ks/cf/ks-cf-tmp-ka-4-Data.db
9 November 2015
4.7.5 Component changes
  • Apache Cassandra 2.1.11.908 with production-certified Cassandra changes
  • Apache Thrift 0.9.3
4.7.5 Resolved issues
  • Error loading class 'solr.FastLRUCache' in search. (DSP-4702)
  • Dynamic field marker does not have per document ttl applied to it in thrift tables. (DSP-6886)
  • Better dsetool core name error handling. (DSP-6953)
  • Possible out of memory error from malformed Thrift protocol messages. (DSP-7019)
  • CQLSH COPY command could fail while reading timestamp options. (DSP-7045)

4.7.5 Certified Cassandra changes

DataStax Enterprise 4.7.5 certifies Cassandra with additional production-certified Cassandra changes.

4.7.4 Release notes for DataStax Enterprise 

Warning: DataStax does not recommend this version for production. Use DataStax Enterprise 4.7.6 or later instead. Various streaming errors with out-of-order keys. (DSP-7424)
If you upgraded to 4.7.4 or 4.7.5 and started receiving streaming errors similar to the following errors, you must scrub the affected table after upgrading. These errors are triggered when range tombstones are streamed incorrectly, and can prevent critical operations like repair and bootstrapping from completing.
WARN  [STREAM-IN-/1.2.3.4] 2015-11-26 10:51:35,930  StreamSession.java:625 - [Stream #481a5680-9423-11e5-bd1f-75f3cea48b3b] Retrying for following error
java.lang.RuntimeException: Last written key DecoratedKey(XXX, YYY) >= current key DecoratedKey(-ZZZ, QQQ) writing into /var/lib/cassandra/data/ks/cf/ks-cf-tmp-ka-4-Data.db

22 October 2015

4.7.4 Component changes
  • Apache Cassandra 2.1.11.872 with production-certified Cassandra changes
  • Apache Solr 4.10.3.0.203
  • Hive Connector 0.1.22
  • Spark Cassandra Connector 1.2.5
4.7.4 Changes and enhancements
  • Use BIO Tomcat connector to improve DSE Search throughput. (DSP-6302)
  • Allow for configuring the NIO connector via Tomcat server.xml. (DSP-6645)
    The BIO connector is more performant. However, you can revert back to NIO by setting the protocol to org.apache.coyote.http11.Http11NioProtocol in the Tomcat server.xml file under ~/dse/resources/tomcat/conf:
    <Connector port="${http.port}" protocol="org.apache.coyote.http11.Http11NioProtocol"
connectionTimeout="20000" 
redirectPort="8443" URIEncoding="UTF-8"/>
    After you start DataStax Enterprise, you can verify the NIO connector with these lines in the Tomcat Catalina log file:
    Sep 14, 2015 11:22:39 PM org.apache.coyote.http11.Http11NioProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8983
Sep 14, 2015 11:22:41 PM org.apache.coyote.http11.Http11NioProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8983
  • Sqoop import and export jobs support alternate connection managers. (DSP-5974)
4.7.4 Resolved issues
  • SOLR created fields like _docBoost do not honor row TTLs - blocking TTL of row. (DSP-4872)
  • CqlBulkOutputFormat is broken by upgrade to Cassandra 2.1. (DSP-4875)
  • --verbose option doesn't enable verbose debugging in Sqoop. Use logback-tools.xml to configure logging. (DSP-6162)
  • load_max_time_per_core is changed to 5 minutes by default. (DSP-6260)
  • Range tombstones might cause unwanted deletes from the index. (DSP-6301)
  • NullPointerException thrown in Solr when enabling live indexing. (DSP-6336)
  • Remove unnecessary contention and improve live indexing reader latency by removing usage of RAMFile for deletes. (DSP-6362)
  • Failure to close key iterator could lead to reference leaks. (DSP-6377)
  • NullPointerException resulting from standard CQL query on Solr field. (DSP-6396)
  • Cannot set max_solr_concurrency_per_core at 1. (DSP-6408)
  • Single pass queries do not work with UUID, TimeUUID, Inet, Decimal, and Varint types. (DSP-6443)
  • Add package dependencies for ‘which’. (DSP-6459)
  • Sqoop cql-import fails with StackOverflowError when too many columns requested. (DSP-6528)
  • Deleted doc might show in a DSE Search query when using facet tagging and excluding filters or query result cache is on. (DSP-6529)
  • Portfolio manager web interface broken. (DSP-6593)
  • TermsComponent returning no results for multi-node cluster. (DSP-6597)
  • 'dse beeline' failed with password auth. (DSP-6598)
  • Solr reindex progress no longer shows in OpsCenter Activity. (DSP-6613)
  • Portfolio demo: HistLoss table created by Hive is wrong (Spark okay). (DSP-6621)
  • Spark SQL CLI fails to instantiate HiveMetaStoreClient with secured cluster. (DSP-6640)
  • Top deletion may cause unwanted deletes from the index. (DSP-6654)
  • FuzzySet integer min value causes assertion bug. (DSP-6658)
  • 2GB SSTable limitation for In-Memory table per node. (DSP-6685)
  • ConcurrentModificationException when using live indexing. (DSP-6717)
  • Solr slow sub-query log doesn't report prepare and process times for CQL Solr queries. (DSP-6724)
  • start-spark-sql-thriftserver doesn't work with internal authentication. (DSP-6727)
  • SparkSQL Thriftserver errors regarding log file (package install). (DSP-6733)
  • Ensure hard commit is executed immediately after reindexing. (DSP-6735)
  • Disabling CqlSlowQueryLog service is not properly logged. (DSP-6760)
  • Unable to use multiple fq in a CQL Solr query. (DSP-6763)
  • Spark Streaming checkpointing doesn't work with internal authentication. (DSP-6776)
  • Exceptions in leader manager are swallowed and silently disable leader manager. (DSP-6786)
  • Solr CQL queries fail with non-compound timestamp keys. (DSP-6810)
  • -Xmn should not be set when using G1GC in cassandra-env.sh. (DSP-6860)
  • Disallow setting Solr concurrency level at 1 via JMX. (DSP-6868)
  • SSL + Kerberos qop=auth-int or auth-conf not reported as warning. (DSP-6939)

4.7.4 Cassandra changes

Production-certified Cassandra fixes are included.

4.7.3 Release notes for DataStax Enterprise  

6 August 2015

4.7.3 Component changes
  • Apache Cassandra 2.1.8.689 with production-certified Cassandra changes
  • Java Driver for Apache Cassandra 2.1.7.1
  • Apache Solr 4.10.3.0.101
  • Apache Spark 1.2.2.2
4.7.3 Changes and enhancements
4.7.3 Resolved issues
  • Leader manager allows multiple leaders at the same time during leader change. (DSP-5424)
  • DSE Spark --jars flag does not work. (DSP-6087)
  • When a node is removed from the ring, it's cached data should also be removed from the state persister. (DSP-6145)
  • Documents indexed multiple times if ColumnFamily contains tombstones over the same row. (DSP-6300)

4.7.2 Release notes for DataStax Enterprise  

15 July 2015

4.7.2 Resolved issues
  • DSE unique key bloom filter is not being used when the unique key is composite. (DSP-6090)
  • DSE Search should shutdown the related threads after dropping cores. (DSP-6110)
  • Empty strings in primary key components cause index deletes. (DSP-6128)

4.7.1 Release notes for DataStax Enterprise  

9 July 2015

4.7.1 Component changes
  • Apache Cassandra 2.1.8.621
  • Apache Solr 4.10.3.0.8
  • Apache Spark 1.2.2.1
  • Apache Tomcat 6.0.44
  • Hive Cassandra Connector 0.1.17.1
  • Spark Cassandra Connector 1.2.3
  • Shark 1.2.1

4.7.1 Changes and enhancements

DataStax Enterprise 4.7.1 introduces the following changes:
  • For all releases and all docs: awesome doc search with relevant, release-specific results. For example, using the Google site search in DataStax Enterprise 4.7 online documents provides search results that include only DSE 4.7, Cassandra 2.1, & CQL for Cassandra 2.x.
  • Realtime Lucene search version 2 provides improved performance for live indexing. (DSP-4302)
    • Applying deletes is parallelized to improve reader open times.
    • The default autoSoftCommit maxTime is 1000ms when live indexing is enabled.
    • Improved performance for live indexing range queries.
  • Python supports loading cassandraTables from a Spark streaming context and saving a DStream to Cassandra. (DSP-4685)
  • Do not store spatial types in Lucene index by default. (DSP-4694)
  • Add Kafka Spark Streaming demo in ./dse/demos. (DSP-4804)
  • Extend request handling and processing to support custom searchers. (DSP-5075)
  • Allow passing JMX username and password to dsetool commands and dse commands. (DSP-5217)
  • Support CQL partition key restrictions together with Solr queries. (DSP-5123)
  • Change the default max_solr_concurrency_per_core to the number of available CPU cores. (DSP-5239)
  • Securely pass nodetool, dsetool, and other passwords when -u is passed. (DSP-5308)
  • New dsetool command to automatically generate schema at stdout. (DSP-5373)
  • Support dynamic fields/maps with FieldInputTransformer. (DSP-5428)
  • Support CQL LIMIT when Solr deep paging is active. (DSP-5657)
  • CompositeKeyField, unique key, is now a text field DocValue. (DSP-5756)
  • Include cassandra-stress example .yaml files in package installs. (DSP-5770)
  • DSE Search node bootstrapping is more efficient. (DSP-5948)

4.7.1 Resolved issues

  • Improvements of vnodes in Solr. (DSP-3759)
  • Compaction of files stored on the cfs-archive layer is not disabled, and files are compacted automatically. (DSP-4081)
  • Disable SSLv3 in the TomcatSolrRunner. (DSP-5070)
  • CQL Solr queries do not error out when additional WHERE clauses are present. (DSP-5109)
  • Hive queries fail with Thrift size error. (DSP-5247)
  • Leader manager notifies its listeners even when no change happened. (DSP-5422)
  • CSV copy command doesn't work inside cqlsh command file. (DSP-5469)
  • Incorrect Shark/Hive warning messages about RF when using NetworkTopologyStrategy. (DSP-5498)
  • dsetool get_core_schema: requires appropriate host to be set. (DSP-5516)
  • Logback logging broken in some places. (DSP-5577)
  • Change consistency level from QUORUM to LOCAL_QUORUM for CassandraStorageEngine. (DSP-5606)
  • DSE hangs on restart with empty XML config files after unclean shutdown. (DSP-5645)
  • Enabling userlatencytracking from dsetool causes NPE. (DSP-5654)
  • Autogenerated cores allow invalid map types. Validation refactoring. (DSP-5655)
  • byoh-env.sh problem locating dsetool. (DSP-5664)
  • Spark Worker's web_ui port is not consistent with documentation. (DSP-5665)
  • JDBC issue with Spark SQL thriftserver. (DSP-5686)
  • Demos: Test for dse in path not correct. (DSP-5671)
  • Bootstrapping TDE nodes try to read from themselves. (DSP-5701)
  • ODBC issue with Spark SQL thriftserver. (DSP-5730)
  • Query from SparkSQL fails when using user defined frozen type in Cassandra. (DSP-5800)
  • Running sstable2json if Solr enabled results in NPE. (DSP-5893)
  • Solr CQL index may handle RangeTombstones incorrectly. (DSP-5940)
  • Spark jobs hang during compaction. (DSP-5984)
  • HIVE - INNER JOIN, replacing nulls with other rows data. (DSP-5994)
  • Netty client starvation with more nodes than the configured max connections. (DSP-6061)

Release notes for DataStax Enterprise 4.7 

18 May 2015

4.7 Component changes
  • Apache Cassandra 2.1.5.469 with production-certified Cassandra fixes
  • Apache Hadoop 1.0.4.15
  • Apache Hive 0.12.0.7
  • Apache Mahout 0.8
  • Apache Pig 0.10.1
  • Apache Shark 1.1.1
  • Apache Solr 4.10.3.0.6
  • Apache Spark 1.2.1.2
  • Apache Sqoop 1.4.5.15.1
  • Apache Thrift 0.9.1
  • Apache Tomcat 6.0.39
  • Faster XML 2.1.2
  • Guava 16.0.1
  • HdrHistogram 1.2.1.1
  • Java Driver for Apache Cassandra 2.1.5
  • JBCrypt 0.3m
  • Netty 4.0.13.Final
  • Py4J 0.8.1
  • SLF4J 1.7.2
  • Spark Cassandra Connector 1.2.1
  • Snappy 1.0.5.2

4.7 Changes and enhancements

DataStax Enterprise 4.7 introduces the following changes in addition to the new major features.

  • For all releases and all docs: awesome doc search with relevant, release-specific results. For example, using the Google site search in DataStax Enterprise 4.7 online documents provides search results that include only DSE 4.7, Cassandra 2.1, and CQL for Cassandra 2.1.
  • Support for Oracle Java SE Runtime Environment (JRE) 8 and OpenJDK 7. (DSP-3338 and DSP-2619)
  • Support off-server encryption key management. (DSP-1758)
  • Generalize leader (JobTracker, Spark Master) management and automatic high availability for Spark analytics. (DSP-2760)
  • Save any RDD to Cassandra with automatic table creation. (DSP-3310)
  • DSE In-Memory improvements expand the amount of data that can be maintained in memory and allow faster read operations on larger volumes of transactional data. (DSP-3341)
  • Support for bulk loading data from Spark RDD to Cassandra. (DSP-3378)
  • Support for live indexing increases indexing throughput and reduces Lucene index reader latency. (DSP-3740)
  • Additions of Spark to CQL performance objects. (DSP-3751)
  • Support for the token() syntax for Solr CQL queries. (DSP-3809)
  • Switch from log4j to logback. (DSP-3858)
  • Manage the location of Solr data. (DSP-3930)
  • Implement global, configurable filter cache size. (DSP-3931)
  • Improve shard routing by taking into account node health factors. (DSP-3958)
  • Encrypt saved cache files when TDE is enabled. (DSP-4017)
  • Implement fault-tolerant distributed queries. (DSP-4072)
  • Internal queries are logged as system, not unknown. (DSP-4079)
  • Integrate Spark SQL with Thrift JDBC server. (DSP-4203)
  • Java Native Access (JNA) is upgraded to 4.0. (DSP-4600)
  • Support for Solr stored copy fields has been removed. Data for existing stored copy fields in the database will still be returned. (DSP-4091)
  • Improved error handling for core create and core reload failures. (DSP-4110)
  • Added SSL security to Spark. (DSP-4170)
  • Support for Spark MLlib. (DSP-4238)
  • Production certification of Spark 1.2. (DSP-4311)
  • Merge lucene-solr 4.10.2. (DSP-4473)
  • Support for single-pass distributed queries in CQL Solr queries. (DSP-4475)
  • Support for Solr cursors (pagination). (DSP-4476)
    Note: DataStax Enterprise 4.7 integrates native driver paging with Solr cursor-based paging. To retain the same behavior for your queries in 4.7 as in 4.6, ensure that your driver has protocol-level paging disabled for CQL-based Solr queries.
  • Implement distributed pivot faceting with HTTP and CQL Solr queries. (DSP-4477)
  • Implement sparse offheap filter. (DSP-4518)
  • Allow for more than one worker per node. (DSP-4558)
  • Production certification of Solr 4.10.
  • Upgrade to lucene-solr 4.10.3. (DSP-4582)
  • Hive connector supports schema generation for decimal types. (DSP-4664)
  • Implement MemoryOnlyStrategy using mmap and mlock. (DSP-4650)
  • Support for conditional log coloring. (DSP-4677)
  • Allow custom filter cache implementations. (DSP-4686)
  • Spark plus Search integration with workload on a single SearchAnalytics node. Limitations apply. (DSP-4703)
  • Core resource options file for resource generation are read by dsetool on the client side. (DSP-4766)
  • In Cassandra 2.1, the cqlsh utility uses the native protocol. With the DataStax python driver, the default cqlsh listen port is 9042. (DSP-4886)
  • Add kmip management functions to dsetool. (DSP-4959)
  • Add dsetool autojt command to enable switching to automatic leaders. (DSP-4965)
  • Make updates replayed by the Solr commit log on startup immediately searchable. (DSP-5095)
  • DSE Search does not support Solr Managed Resources. (DSP-4735)

4.7 Resolved issues

Resolved issues from DataStax Enterprise 4.6.7 and earlier are included.

  • JMX defaults to binding only localhost to mitigate "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code."

    After you upgrade to DataStax Enterprise 4.7, get the latest default values from cassandra-env.sh and ensure that your local cassandra-env.sh file has the latest default values. If you require remote access of JMX, you can safely enable remote JMX access by turning on JMX security. The latest default values from cassandra-env.sh are:

    LOCAL_JMX=yes
if [ "$LOCAL_JMX" = "yes" ]; then  
    JVM_OPTS="$JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT -XX:+DisableExplicitGC"
else  
    JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT"  
    JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"  
    JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.ssl=false"  
    JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=true"  
    JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.password.file=/etc/dse/cassandra/jmxremote.password"
fi
JVM_OPTS="$JVM_OPTS$JVM_EXTRA_OPTS"
  • Analytics jobs use the broadcast address to connect to nodes even when ec2snitch or another snitch that supports public vs private is used. (DSP-3694)
  • Improve error handling in case of core creation failures. (DSP-4110)
  • Speed up dsetool. (DSP-4396)
  • Integrate Spark SQL command line. (DSP-4399)
  • Transitional authenticators do not work with the native protocol. (DSP-4620)
  • Upgrade PySpark support to Spark 1.2 (DSP-4660)
  • The Cassandra context is removed. Use the Spark context instead. (DSP-4672)
  • Delete by query failing on mixed workload clusters. (DSP-4675)
  • LDAP authenticator allows empty password to be authenticated. (DSP-4687)
  • Race condition while loading secondary indexes upon upgrade. (DSP-4689)
  • CompoundFileDirectory exceptions. (DSP-4848)
  • DseDaemon throw NPE in safeStop if failure occurs in startup. (DSP-4865)
  • Fix Solr document expiration when row has already been physically deleted from Cassandra. (DSP-4869)
  • NPE when using Solr data import handler. (DSP-4888)
  • Upgrade to Java driver 2.1.5. (DSP-4947)
  • Plugin/stats not showing in the Solr Admin UI. (DSP-5007)
  • Installed JNA is no longer used. (DSP-5033)
  • Creating solr_query column before using dsetool causes solr queries in CQL to fail. (DSP-5105)
  • Can't run describe keyspace in cqlsh on Solr core. (DSP-5117)
  • Block incremental repair operations on remote anti-compaction. (DSP-5331)
  • Removed CQLStorage handler, migrate the tables to CqlNativeStorage format that uses Cql Record Reader. (DSP-5538)
4.7 Issues
  • Dynamic fields of the Solr type LatLongTypeSolr are not supported. (DSP-5002)
  • Creating indexes with DEFERRED REBUILD is not supported in Spark SQL; a new column defaults to empty instead of none. (DSP-4713)
  • Solr_query fails in mixed upgrade state because of Solr pagination. (DSP-5334)

    On a cluster with mixed versions of DSE, for example 4.6 and 4.7, where only the 4.7 version supports pagination, issuing queries from the 4.7 nodes returns only FetchSize results because 'FetchSize' is the driver iused when executing the query. To workaround this limitation, disable driver pagination so that Solr pagination is also disabled for the queries on 4.7 nodes.

  • When starting DSE in Hadoop mode, an exception might occur that is related to the Task Tracker initially being unable to connect to the JobTracker. Automatic and immediate retry successfully connects and and does not impact Hadoop performance. (DSP-5621)

4.7 Cassandra changes

Production-certified Cassandra fixes are included in DataStax Enterprise 4.7.