DataStax Enterprise 5.0 release notes

DataStax Enterprise release notes cover cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues for DataStax Enterprise 5.0.

DataStax Enterprise release notes cover required planning, components, changes and enhancements, known issues, and resolved issues. The latest version of DataStax Enterprise 5.0.x is 5.0.15.
Warning:

DSP 5.0.0 to 5.0.11: TTL expiration timestamps are susceptible to the year 2038 problem. If the TTL value is long and an expiration date that is greater than the maximum threshold of 2038-01-19T03:14:06+00:00, the data is immediately expired and purged on the next compaction. When using a long TTL, DataStax strongly recommends upgrading to DSE 5.0.15 or later and taking required action to protect against silent data loss. (DSP-15412)

DSE 5.0.0 to 5.0.9: Data is marked as repaired (by anti-compaction) for both full and incremental repairs. See (DSP-14464).

DSE 5.0.6 and later: If you use partitioned vertex tables (PVTs) in DSE Graph, do not upgrade to 5.0.6 and later. See DSP-12540.

DSE 5.0.0 to 5.0.2: Lifecycle Manager (LCM) incorrectly creates DSE Graph clusters in 5.0.0 to 5.0.2. See OPSC-10004 and OPSC-10052.

DSE 5.0.1: If you use Super Columns, do not upgrade to 5.0.1. See DSP-10602.

DSE 5.0.0: CFS is corrupted during upgrade to 5.0.0. See DSP-10299.

Requirement for Uniform Clusters 

All Nodes in each Cluster must be uniformly licensed to use the same Subscription. For example, if a Cluster contains 5 Nodes, all 5 Nodes within that Cluster must be either DataStax Basic, or all 5 Nodes must be DataStax Enterprise. Mixing different Subscriptions within a Cluster is not permitted. “Cluster” means a collection of Nodes running the Software which communicate with one another via Gossip, and “Gossip” means the mechanism within the Software enabling related Nodes to communicate with one another. For more information, see Enterprise Terms.

Before you upgrade 

Upgrade advice Compatibility
Before you upgrade to a later major version, upgrade to the latest patch release (5.0.15) on your current version. Be sure to read the relevant upgrade documentation. Upgrades to DSE 5.0 are supported from:
Check the compatibility page for your products. DSE 5.0 product compatibility:
See Upgrading DataStax drivers. DataStax Drivers: You may need to recompile your client application code.
Use DataStax Bulk Loader for loading and unloading data. Loads data into DSE 5.0 or later and unloads data from any Apache Cassandra™ 2.1 or later data source.

Release notes

DSE 5.0.15 

7 November 2018

5.0.15 Components 

All components from DSE 5.0.15 are listed. Components that are updated for DSE 5.0.15 are indicated.
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.2353 (updated)
  • Apache Spark™ 1.6.3.9
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.53
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.13 (updated)
  • Hive Connector 0.4.50 (updated)
  • Java Driver for Apache Cassandra 3.0.8 (updated)
  • Jetty 8.1.14.v20131031
  • Netty 4.0.54.Final (updated)
  • Spark Jobserver 0.6.2.154 (updated)
  • TinkerPop 3.2.9-20180507-c6f6e7f0 (updated)

DataStax Enterprise 5.0.15 is compatible with Apache Cassandra™ 3.0.18 and adds additional production-certified changes.

5.0.15 Highlights 

The executive summary highlights are just a top-level view. Be sure to review all release notes.

DSE Analytics and DSEFS 5.0.15 highlights 

  • CRITICAL Fixed a race condition that could corrupt data retrieved from DSEFS or cause spurious FileNotFoundExceptions and other problems under very heavy load. (DSP-17178)
  • CRITICAL Security improvement: DSEFS uses an isolated native memory pool for file data and metadata sent between nodes. This isolation makes it harder to exploit potential memory management bugs. (DSP-16492)
  • Improved reliability of DSEFS startup when some nodes in the cluster cannot be reached. (DSP-16875)
  • Fixed DSEFS startup crash caused by using a duration type in non-dsefs keyspaces. (DSP-16825)

Spark bug fixes:

  • DataStax Spark Cassandra Connector: Upgraded Java driver to version 3.0.8. Fixes duplicate UUID generation problem that can happen when running multiple Spark executors on the same node. (DSP-15643)
  • Fixed reading from external filesystem (S3, HDFS) based external tables. (DSP-16561)

Improvements:

  • DataStax Spark Cassandra Connector: Avoid a freeze on exit of Spark applications (including Spark shell), if the application program does not close the Spark context. (DSP-16051)
  • Reduced the number of internode connections kept open by DSEFS. (DSP-14284)
  • Improved DSEFS diagnostics and logging. (DSP-16101)

DSE Graph 5.0.15 highlights 

Minor Graph Search operations fixes:
  • Changing search index schema using a gremlin script is supported. (DSP-15831)
  • Deleting a search index that was defined inside a graph no longer fails. (DSP-16765)

DSE Search 5.0.15 highlights 

  • Minor performance and stability improvements. (DSP-17220, DSP-15664)

DataStax Enterprise 5.0.15 Changes and enhancements 

DSE Core 5.0.15 changes and enhancements 

  • New sstablepartitions tool for identifying large partitions. (DB-803)
  • New nodetool listendpointspendinghints command prints hint information about the endpoints this node has hints for. (DB-1674)
  • Improved error handling and logging for TDE encryption key management. (DP-15314)
  • DataStax does more extensive testing on OpenJDK 8 due to the end of public updates for Oracle JRE/JDK 8. (DSP-16179)
  • sstableloader supports custom config file locations. (DSP-16092)
  • Support creating system keys before the output directory is configured in dse.yaml. (DSP-15380)

DSE Analytics 5.0.15 changes and enhancements

  • DSE client applications, like Spark, hard stop if user home is not defined, does not exist, or the current user does not have write permissions. (DSP-15476)

DSEFS 5.0.15 changes and enhancements 

  • Improved handling of DSEFS client idle connection. (DSP-14284)
  • Security improvement: DSEFS uses an isolated native memory pool for file data and metadata sent between nodes. This isolation makes it harder to exploit potential memory management bugs. (DSP-16492)

DSE Graph 5.0.15 changes and enhancements 

  • Improved Gremlin console authentication configuration. (DSP-9905)
  • Maximum evaluation timeout is 1094 days. (DSP-16709)
  • Added convenience methods for reading graph configuration: getEffectiveAllowScan and getEffectiveSchemaMode. (DSP-16650)
  • The hardcoded default schema_mode is changed from Development to Production. (DSP-16650)

DSE Search 5.0.15 changes and enhancements 

  • If a client executes a query that results in a shard attempting to send an internode frame larger than the size specified in frame_length_in_mb, the client receive an error message with a message like this:
    Attempted to write a frame of <n> bytes with a maximum frame size of <n> bytes
    

    In earlier versions, the query timed out with no message. Information was provided only as error in the logs.

  • The node health option uptime_ramp_up_period_seconds default value in dse.yaml is reduced to 3 hours (10800 seconds). (DSP-15752)
  • In earlier releases, CQL search queries failed with UTFDataFormatException on very large SELECT clauses and when tables have a very large number of columns. (DSP-17220)

    With this fix, CQL search queries fail with UTFDataFormatException only when SELECT clauses constitute a string larger than 64k UTF-8 encode bytes.

  • Critical memory leak and corruption fixes for encrypted indexes. (DSP-17111)

5.0.15 Resolved issues 

5.0.15 DataStax Enterprise core resolved issues 

  • Race condition occurs on bootstrap completion. (DB-1383)
  • Fix handling of start bound in legacy paged queries. (DB-1984)
  • Move TWCS message "No compaction necessary for bucket size" to Trace level or NoSpam. (DB-2022)
  • nodetool upgradesstables fails with 20-year TTL. After upgrade to 5.0.15, take required action. (DB-2109)
  • Non-portable syntax (MX4J bash-isms) in cassandra-env.sh broke service scripts. (DB-2123)
  • sstableloader options assume the RPC/native (client) interface is the same as the internode (node-to-node) interface. (DB-2184)
  • Connections on non-serialization errors are not dropped. (DB-2233)
  • Audit events for CREATE ROLE and ALTER ROLE with incorrect spacing exposes PASSWORD in plain text. (DB-2285)
  • Rebuild should not fail when a keyspace is not replicated to other DCs. (DB-2301)
  • Client warnings are not always propagated via LocalSessionWrapper. (DB-2304)
  • Timestamps inserted with ISO 8601 format are saved with wrong millisecond value. (DB-2312)
  • Compaction fails with IllegalArgumentException: null. (DB-2329)
  • Create a log message when DDL statements are executed. (DB-2383)
  • Compaction strategy instantiation errors don't generate meaningful error messages, instead return only InvocationTargetException. (DB-2404)
  • repair may skip some ranges due to received range cache. (DB-2432)
  • NullPointerException during compaction on table with TimeWindowCompactionStrategy (TWCS). (DB-2472)
  • Prevent potential SSTable corruption with nodetool refresh. (DB-2594)
  • The nodetool gcstats command output incorrectly reports the GC reclaimed metric in bytes, instead of the expected MB. (DB-2598)
  • TypeParser is not thread safe. (DB-2602)
  • STCS, DTCS, TWCS, TMD aren't thread-safe. (DB-2609)
  • sstableloader does not decrypt passwords using config encryption in DSE. (DSP-13492)
  • Fix legacy complex range tombstone serialization+deserialization for static and regular columns. (DSP-15878)
  • Fix error in MVs referencing a function with uppercase letters on its name. (DSP-15878)
  • Improve replace fail messages when a replace is retried before QUARANTINE_DELAY. (DSP-15878)
  • Delay closing connection when nodes are removed to allow in-flight commands to complete. (DSP-15878)
  • Harden txn log files against exceptions when adding records and improve log messages. (DSP-15878)
  • Fix JVM startup check. (DSP-15878)
  • Upgraded Java Driver for Apache Cassandra 3.0.8 fixes possible duplicate UUID generation. (DSP-15643)
  • nodetool describecluster incorrectly shows DseDelegateSnitch instead of the snitch configured in cassandra.yaml. (DSP-16158)
  • Improved compatibility with external tables stored in the DSE Metastore in remote systems. (DSP-16561)
  • Heap-size calculation is incorrect for RpcCallStatement + SearchIndexStatement. (DSP-16731)
  • Non-internal users are unable to use permissions granted on CREATE. (DSP-16824)
  • EverywhereStrategy picks non-token-owning nodes as endpoints. (DSP-16951)
  • Fix sstableloader error when internode encryption, client_encryption, and config encryption are enabled. (DSP-17536)

5.0.15 DSE Analytics resolved issues 

  • Java driver in Spark Connector uses daemon threads to prevent shutdown hooks from being blocked by driver thread pools. (DSP-16051)
  • DSEFS silently fails when TCP port 5599 is not open between nodes. (DSP-16101)
  • Unable to get available memory before Spark Workers are registered. (DSP-16790)

5.0.15 DSEFS resolved issues 

  • DSEFS throws exceptions and cannot initialize when listen_address is left blank. (DSP-16296)
  • Timeout issues in DSEFS startup. (DSP-16875)
    Initialization would fail with error messages similar to:
    com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
  • DSEFS fails to start when there is a table with duration type or other type DSEFS that can't understand. (DSP-16825)
  • Under high loads, DSEFS reports temporary incorrect state for various files/directories. (DSP-17178)
  • DSEFS exit code not set in some cases (DSP-17266)

5.0.15 DSE Graph resolved issues 

  • Changing search index schema using a gremlin script might fail with Search index may not be modified while it is being reindexed. Please wait until reindexing has finished. message. (DSP-15831)
  • Setting graph.traversal_sources.g.evaluation_timeout breaks graph. (DSP-16709)
  • Deleting a search index that was defined inside a graph fails. (DSP-16765)
  • DSEFS Hadoop layer doesn't properly translate DSEFS exceptions to Hadoop exceptions in some methods. (DSP-16933)

5.0.15 DSE Search resolved issues 

  • Servlet container shutdown (Tomcat) prematurely stops logback context. (DSP-15807)
  • Wikipedia demo does not work with package installation for 5.0.11 and later. (DSP-16398)

5.0.15 Known issues 

  • The Spark Jobserver demo has an incorrect version for the Spark Jobserver API. (DSP-15832)

    Workaround: In the demo's gradle.properties file, change the version from 0.6.2 to 0.6.2.238.

  • Spark shutdown stops executors but does not wait for everything else to close, causing CoarseGrainedScheduler errors on app termination: org.apache.spark.SparkException: Could not find CoarseGrainedScheduler or it has been stopped. (DSP-16751)

5.0.15 Cassandra changes 

DataStax Enterprise 5.0.15 is compatible with Apache Cassandra™ 3.0.18 and adds additional production-certified changes.

5.0.15 TinkerPop changes 

DataStax Enterprise (DSE) 5.0.15 includes all changes from previous releases plus these production-certified changes that are in addition to TinkerPop 3.2.9:

DSE 5.0.14 

11 May 2018

5.0.14 Components 

  • Apache Cassandra™ 3.0.15.2269 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.2283 (updated)
  • Apache Spark™ 1.6.3.9 (updated)
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.53
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.11
  • Hive Connector 0.4.49 (updated)
  • Java Driver for Apache Cassandra 3.0.6 (updated)
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.153 (updated)
  • TinkerPop 3.3.3-20180507-ea594461 (updated)

5.0.14 Highlights 

This release consists of bug fixes, an improvement in DSE Graph anonymous traversals and bytecode-based traversals, and support for specifying different Solr field types for each CQL map key.

DataStax Enterprise 5.0.14 Changes and enhancements 

Changes and enhancements include:
DSE Graph 5.0.14 changes and enhancements
  • Improved performance of anonymous traversals and bytecode-based traversals that made use of withStrategy() configurations. (DSP-15673)

DSE Search 5.0.14 changes and enhancements 

  • Support for specifying different Solr field types for each CQL map key. (DSP-15622)

5.0.14 Resolved issues 

5.0.14 DataStax Enterprise core resolved issues 

  • DSE demos use Jetty Runner 9.4.8. (DSP-14772)
  • CVE-2017-7525: FasterXML Jackson-databind is prone to a remote-code execution vulnerability. (DSP-14784)
  • Materialized view schema file for snapshots is created as tables. (DSP-15486)
  • Init timestamp with Long.MIN_VALUE instead of -1. (DSP-15486)
  • AssertionError in ThrottledUnfilteredIterator due to empty UnfilteredRowIterator. (DSP-15486)
  • Make sstableloader use cassandra.config.loader instead of hard-coded YamlConfigurationLoader. (DSP-15486)
  • Backport CASSANDRA-9241, fix nodetool toppartitions. (DSP-15486)
  • Ignore lost+found directory on startup checks. (DSP-15486)
  • Auto-Bootstrap fails while a rebuild succeeds when adding a node, fix NPE via QueryProcessor.resultify. (DSP-15486)
  • Handle output stream flush fails before closing OutboundTcpConnection. (DSP-15486)
  • Make PFS compatibility mode for GPFS opt-in. (DSP-15486)
  • Fix how duplicated tombstones are handled in legacy layout within the same clustering. (DSP-15486)
  • Avoid NPE when providing invalid number of arguments for token function. (DSP-15486)
  • Fetch/query no columns in priming connections to avoid errors if system.local columns are changed. (DSP-15486)
  • Protect against BigDecimals with large scale. (DSP-15486)
  • DSE client-tool help doesn't work if ~/.dserc file exists. (DSP-15869)
  • CVE-2018-1308: XML External Entity (XXE) processing attack through Apache Solr DataImportHandler dataConfig request parameter. (DSP-15902)
  • CVE-2016-1000031: Security vulnerability in Apache Commons FileUpload. (DSP-15908)
  • LDAP library issue. (DSP-15927)
  • Allow all connection pool options to be set for LDAP. (DSP-15948)

5.0.14 DSE Analytics resolved issues 

  • JSch is susceptible to a path traversal vulnerability. (DSP-13961)
  • A shard request timeout caused an assertion error from Lucene getNumericDocValues in the log that could be causing a live indexing bug. (DSP-14216)
  • DSE Analytics: Additional configuration is required when enabling context-per-jvm in the Spark Jobserver. (DSP-15163)
  • dse spark-sql does not work with two authentication schemes. (DSP-15343)
  • Worker UI does not display the actual class name of driver application running in cluster mode. (DSP-15479)
  • Spark Master is not elected on mixed version clusters. (DSP-15374)

    Upgrades from DSE 4.8.x to 5.0.14 and later 5.0.x require planning and specific action before starting DSE 5.0 analytics nodes. See

  • DSEFS transactions not always replayed at startup. (DSP-15462)
  • Spark Master REST API is deprecated. (DSP-15491)
  • Prevents situation where multiple Spark masters may be started on the same machine. (DSP-15636)
  • In Portfolio demo, pricer is said to be run with sudo. This is no longer needed. (DSP-15970)
  • Fixed a bug which prevented the CassandraHiveMetastore from adding multiple partitions for File based datasources. Fixes MSCK REPAIR TABLE command. (DSP-16067)

5.0.14 DSEFS resolved issues 

  • Timeout when trying to umount a dsefs location. (DSP-15453)
  • Exception is thrown by DseFsPlugin during shutdown and is not reported. (DSP-15474)
  • DSE might not shutdown properly when DSEFS encounters a problem, and exceptions are not logged. (DSP-15482)
  • DSEFS fetching a file from an offset returns empty content. (DSP-15907)

5.0.14 DSE Graph resolved issues 

  • GraphSON parsing error prevents proper type detection under certain conditions. (DSP-14066)
  • Synchronization hurts graph OLAP on multi-core executors. Improve scalability of OLAP queries with remote traverses. (DSP-15068)
  • graph solr phrase() predicate shows IndexOutofBound error. (DSP-15408)
    • Single-character tokens used in search index queries, for example with predicate token("a") are erroneously dropped.
    • Search index queries using phrase(...) predicates fail exceptionally when processing values that end in a prefix of the search phrase.
  • toJSON() does not always work with geo types. (DSP-15650)
  • ObjectMapper contention for fluent API requests. (DSP-15732)

5.0.14 DSE Search resolved issues 

  • NPE during loading data with RT geonames. (DSP-12361)
  • deleteById and deleteByQuery overflow prepared statement cache. (DSP-15620)
  • Specialized fields on map<text,text> are not indexed. (DSP-15622)
  • Offline sstable tools may fail if DSE Search index is present on a table. (DSP-15628)
  • HTTP read on solr_stress doesn't inject random data into placeholders. (DSP-15727)

5.0.14 Known issues 

No known new issues.

5.0.14 Cassandra changes 

DataStax Enterprise 5.0.14 certifies Apache Cassandra™ 3.0.15 with additional production-certified Cassandra changes.

5.0.14 TinkerPop changes 

DataStax Enterprise (DSE) 5.0.14 includes all changes from previous releases plus these production-certified changes that are in addition to TinkerPop 3.2.9:
  • Performance enhancement to Bytecode deserialization. (TINKERPOP-1936)
  • Path history isn't perserved for keys in mutations. (TINKERPOP-1947)
  • Traversal construction performance enhancements. (TINKERPOP-1950)
  • Bump to Groovy 2.4.15 - resolves a Groovy bug preventing Lambda creation in GLVs in some cases. (TINKERPOP-1953)

DSE 5.0.13 

24 April 2018

5.0.13 Resolved issue

  • Fix LDAP library issue. (DSP-15927)

DSE 5.0.12 

13 February 2018

5.0.12 Components 

  • Apache Cassandra™ 3.0.13.2128 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.2138 (updated)
  • Apache Spark™ 1.6.3.3
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.53
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.11 (updated)
  • Hive Connector 0.4.47
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.8-20180125-184680fd (updated)

5.0.12 Highlights 

Executive summary highlights for DSE 5.0.12: The executive summary highlights are just a top-level view. Be sure to review all release notes.

DSE Enterprise 5.0.12 highlights 

  • Fix for possible data loss scenario caused by the TTL expiration timestamps susceptible to the year 2038 problem. (DSP-15412)

    See Recovering expired data caused by TTL year 2038 problem.

  • Fix for local DC connections are compressed despite internode_compression: dc. (DSP-15087)
  • Improve gossip dissemination time. (DSP-15087)
  • More quickly detect down nodes for batchlogs using the incoming connections. (DSP-15087)
  • New metrics for batchlog-replays. (DSP-15087)
  • Commands to support migrating from compact storage. These commands are required to upgrade to DSE 6.0. (DSP-14966)
  • Add nodetool abortrebuild. (DSP-14827)
  • Add metrics on coordination of read commands. ((DSP-14775)
  • Add anticompaction metrics and warn users when incremental repair is inefficient. (DSP-14494)

DSE Advanced Replication 5.0.12 highlights 

  • Improved handling and bug fixes in scenarios where the source cluster has multiple logical data centers. (DSP-14767, DSP-14515, DSP-15121)

DSE Analytics and DSEFS 5.0.12 highlights 

  • Fixed a DSEFS issue that could prevent upgrades from 5.0.x to 5.1.5. (DSP-15237)

DSE Graph 5.0.12 highlights 

  • Security improvements in Graph Loader. (DGL-281, DGL-275) and Server (DSP-14540, DSP-14482, DSP-14678, DSP-13372)

DSE Search 5.0.12 highlights 

  • Performance and corruption issues with encrypted indexes addressed with a full reindex after upgrade. (DSP-14943, DSP-14485, DSP-15265)
  • Better protection against abusing the Solr filter cache with too many entries. (DSP-14534)
  • Performance improvements with RF=(# nodes) DCs. (DSP-12962)

DataStax Enterprise 5.0.12 Changes and enhancements 

DataStax Enterprise core 5.0.12 changes and enhancements 

  • Generate Kerberos debug output. (DSP-12430)
  • Default number of threads used by performance objects is increased from 1 to 4; configure threads with new dse.yaml performance_core_threads parameter. (DSP-14643)
  • Add metrics on coordination of read commands. (DSP-14775)
  • DSE start-up parameter -Dnative_transport_startup_delay_second to delay startup of native transport. (DSP-14839)
  • Configurable round-trip time (RTT) latency between datacenters with option cross_dc_rtt_in_ms. (DSP-14775)
  • Throttle base partitions during MV repair streaming to prevent OOM. (DSP-14775)
  • Add cross_dc_rtt_in_ms to cross dc requests, default 0. (DSP-14775)
  • Warning to run full repair when increasing replication factor. (DSP-14494)
  • Add anticompaction metrics and warn users when incremental repair is inefficient. (DSP-14494)
  • New metrics for batchlog-replays. (DSP-14839)
  • Failed bootstrap streaming leaves authentication uninitialized. (DSP-14839)
  • Eliminate thread roundtrip for version handshake. (DSP-14827)
  • Do not leak body buffer in case of protocol exceptions and upgrade Netty to 4.0.52. (DSP-14827)
  • Add nodetool rebuild mode reset-no-snapshot option. (DSP-14827)
  • Add nodetool abortrebuild command. (DSP-14827)
  • Make nodetool assassinate more resilient to missing tokens. (DSP-14827)
  • New CQL ALTER TABLE DROP COMPACT STORAGE option to remove Thrift-compatibility from tables. (DSP-14839)
  • DSE Multi-Instance not functional when Spark enabled. (DSP-15015)
  • Register SizeEstimatesRecorder earlier and enable cleanup of invalid entries. (DSP-15024)
  • Improved schema migration logging. (DSP-15087)
  • Make migration-delay configurable. (DSP-15087)
  • Improved streams logging. (DSP-15087)
  • Support frozen collection list and set in stress. (DSP-15087)
  • Print heap histogram on OOM errors by default. (DSP-15087)
  • Fixes for waitForGossiper. (DSP-15087)
  • More quickly detect down nodes for batchlogs using the incoming connections. (DSP-15087)
  • Keep SSTable level for decommission, remove, and move operations. (DSP-15087)
  • LCS levels are not respected for nodetool refresh and replacing a node. (DSP-15087)
  • Use more intelligent level picking when we have a non-l0 file. (DSP-15087)
  • Improve gossip dissemination time. (DSP-15087)
  • Expanded hinted handoff instrumentation. (DSP-15087)
  • Fix for Local DC connections are compressed despite internode_compression: dc. (DSP-15087)
  • Separate commit log replay and commit throwable inspection and policy handling. (DSP-15087)
  • Fix infinite loop on index query paging in tables with clustering. (DSP-15087)
  • Generate LDAP debug output. (DSP-15176)
  • Allow ALTER of system_distributed keyspace tables. (DSP-15294)
  • Do not fetch columns that are not in the filter fetched set. (DSP-15294)
  • New nodetool getseeds and reloadseeds commands. (DSP-15412)

DSE Analytics 5.0.12 changes and enhancements 

  • Improved failure detection for Spark applications. (DSP-14559)

DSEFS 5.0.12 changes and enhancements 

  • Improved error message when Spark fails to connect to DSEFS server. (DSP-14388)
  • DSEFS schema improvements to simplify upgrades. (DSP-14369)
  • Improved error message for DSEFS shell commands. (DSP-14157)
  • Improved error messages are passed to the DSEFS clients, including DSEFS shell, if error occurs while reading a file. (DSP-14371)
  • Improve DSEFS ls performance. (DSP-14408)
  • HTTP communication logging level changed from DEBUG to TRACE to improve filtering. (DSP-14400)
  • Changed lightweight transaction query consistency level from SERIAL to LOCAL_SERIAL. (DSP-14905)
  • The default is changed to LOCAL_SERIAL consistency level, instead of SERIAL. (DSP-14973)
  • DSEFS does not use ssl_native_port for internal connections between DSEFS node and Cassandra when client encryption is enabled. (DSP-15029)

DSE Graph 5.0.12 changes and enhancements 

  • Enable and configure the graph sandbox by default to improve security. (DSP-11679)
  • Gremlin Console command line options for connecting to host. (DSP-12726)
  • Gremlin console schema outputs some syntax with double periods which will error on re-import. (DSP-12904)
  • graph.tx().commit(); call is not allowed on graph.tx().commit(); graph.tx().config().option("allow_scan", true).open(); g.V().count(). Instead, use graph.tx().config().option("allow_scan", true).open(); g.V().count(). (DSP-14482)
  • Gremlin console plugins.txt is read-only by default. (DSP-13372)

DSE Search 5.0.12 changes and enhancements 

  • NPE when dropping the Solr core while indexing is in progress. (DSP-13252)
  • Maximum number of entries in SolrFilterCache is limited to 32K. (DSP-14534)
  • After compact storage is dropped from a table that also has a search index, HTTP writes and deletes-by-ID on the search index are disabled. (DSP-14966)
  • Wikipedia demo path error. (DSP-11327)
  • Avoid token filtering on single-node CQL solr_query. (DSP-12962)

5.0.12 Resolved issues 

5.0.12 DataStax Enterprise core resolved issues 

  • Audit logging does not support UNSET values from prepared statements. (DSP-13043)
  • Token create, cancel, and renew security needs tightening. (DSP-14311)
  • Add anticompaction metrics and warn users when incremental repair is inefficient. (DSP-14494)
  • dbsummary does not work with default performance_core_threads. (DSP-14643)
  • Add cross_dc_rtt_in_ms to cross dc requests, default 0. (DSP-14775)
  • Throttle base partitions during MV repair streaming to prevent OOM errors. (DSP-14775)
  • Warn user to run full repair when increasing replication factor. (DSP-14494)
  • Make nodetool assassinate more resilient to missing tokens. (DSP-14827)
  • Eliminate thread roundtrip for version handshake. (DSP-14827)
  • Do not leak body buffer in case of protocol exceptions and upgrade Netty to 4.0.52. (DSP-14827)
  • Add new rebuild mode reset-no-snapshot (DSP-14827)
  • Add nodetool abortrebuild command. (DSP-14827)
  • Failed bootstrap streaming leaves authentication uninitialized. (DSP-14839)
  • Add -Dcassandra.native_transport_startup_delay_seconds to delay startup of native transport, default 0. (DSP-14839)
  • New metrics for batchlog-replays. (DSP-14839)
  • Memory leak causes executor descriptions to accumulate in DSE process. (DSP-14868)
  • Only serialize failed batchlog replay mutations to hints. (DSP-14959)
  • Change token allocation to use RF=1 method when RF equals rack count. (DSP-14959)
  • nodetool arguments with spaces print script errors. (DSP-14959)
  • Skip building views during base table streams on range movements. (DSP-14959)
  • Register SizeEstimatesRecorder earlier and enable cleanup of invalid entries. (DSP-15024)

5.0.12 DSE Analytics resolved issues 

  • Fix for possible scenario where newly-added nodes can have a schema mismatch for system keyspaces. (DSP-11787)
  • Default logging level for org.apache.hadoop.hive.metastore.ObjectStore set to ERROR to avoid spurious warnings. (DSP-12308)
  • Port setting not respected in DseCassandraConnectionFactory. (DSP-14442)
  • Spark Master/Worker Web UI should bind to RPC listen address and advertise RPC broadcast address by default. (DSP-14433)
  • Default logging level for org.apache.spark.rpc has been changed to ERROR. (DSP-14651)
  • Message is not consistently displayed when SparkContext is created with different configuration. (DSP-14758)
  • CVE-2017-15095 jackson-databind is vulnerable to remote code execution (RCE) attacks. (DSP-15096)

5.0.12 DSEFS resolved issues 

  • "ERROR: Request body rejected, ConnectionClosedException" message is not logged in system.log if the client disconnects in the middle of the request. (DSP-14597)
  • Reads incorrectly show Response body rejected errors. (DSP-14615)
  • DSEFS does not retry queries. (DSP-14649)
  • Incorrect return of 0 exit code for failed command execution. (DSP-14652)
  • User name/password was not provided warning is in the DSEFS shell log when security is not enabled. (DSP-14708)
  • Added getScheme, getDefaultPort, concat, and truncate method implementations to DseFileSystem Hadoop API. (DSP-14605)
  • Native memory leak in DSEFS occurs when the server encounters I/O problem while writing a data block. (DSP-15020)
  • DSEFS doesn't use native_transport_port_ssl if set. (DSP-15029)
  • Block not found when reading a file. (DSP-15030)
  • DSEFS fsck command does not fix File not found: / problem which can occur in rare cases after new cluster nodes are started in parallel. (DSP-15048)
  • No check if parent element of a given target path is a directory for mkdir, put, move operations. (DSP-15100)
  • "Promise already completed" error in DSEFS connection pool. (DSP-15122)
  • DSEFS files created through Hadoop API do not properly inherit RF and block size from the parent directory. (DSP-15139)
  • Rare NullPointerException during DSEFS startup. (DSP-15289)
  • Occasional NoHostAvailable exceptions when shutting down DSE with DSEFS enabled. (DSP-15404)

5.0.12 DSE Graph resolved issues 

  • 0 (zero) is not treated as unlimited abort of max num errors. (DGL-307)
  • Unnecessary INSERT and DELETE to dse_security.digest_tokens for every graph statement executed over native protocol. (DSP-13670)
  • Graph sandbox should have org.apache.tinkerpop.gremlin.structure.io whitelisted by default. (DSP-14540)
  • Whitelist org.apache.tinkerpop.gremlin.spark.structure.Spark in sandbox so that Apache TinkerPop Spark-Gremlin application can be stopped programmatically. (DSP-14678)
  • Setting a TraversalSource option from the DSE Driver isn't effective. (DSP-14713)
  • Incorrect behavior with empty id lists for has(). (DSP-15009)
  • Retryable failures have severity DEBUG. Only terminal failures have severity ERROR or WARN. (DSP-15045)
  • User should not be allowed to call graph.tx() when using managed transactions. (DSP-11543)

5.0.12 DSE Search resolved issues 

  • dsetool search commands should return non-zero if operation was not successful. (DSP-9631)
  • Add warnings to DSE Search reload and reindex that reloads impact entire datacenter and reindex is asynchronous. (DSP-9820)
  • CQL solr queries with JSON clause miss singlePass optimizations. (DSP-11407)
  • solr_stress: license is wrong and terms component is disabled in solrconfig-rt.xml. (DSP-11992)
  • Text field does not work for group by operations; unexpected docvalues type SORTED_SET error message for text fields. (DSP-14106)
  • dsetool upgrade_index_files does not work with authentication enabled. (DSP-14114)
  • Incorrect connection limiter scheduler shutdown order for internode transport clients. (DSP-14256)
  • Do not log or send back full Groovy script when the script is too large. (DSP-14410)
  • Cannot turn tracing off after running queries with tracing on. (DSP-14439)
  • When executing CQL search queries with a keyspace RF=(number of nodes), then the token filter is no longer created resulting in faster queries. (DSP-14468)
  • EncryptedFSDirectory#outputLengthCache corruption makes encrypted index files unreadable. (DSP-14485)
  • Indexing wiki demo fails when solrslowlog is enabled. (DSP-14521)
  • Solr filter cache fails after restart. (DSP-14608)
  • Expose StallMetrics in the MetricRegistry. (DSP-14638)
  • UpdateMetrics::Latency::Mean is "unavailable" when writes are in progress. (DSP-14950)
  • Excessive time spent reading unencrypted segment sizes during search index (Solr core) loading. Slow startup on nodes with large encrypted indexes is resolved after upgrade to DSE 5.0.12 is completed with a full reindex for all search indexes using encryption. (DSP-14943, DSP-14485, DSP-15265)
  • Shutdown order in SolrCore causes RejectedExecutionExceptions around CommitTracker. (DSP-15040)
  • Cannot create core using HTTP due to missing "create" permission. (DSP-15046)
  • Delete by query fails with exception on vnodes. (DSP-15051)
  • Inconsistent behavior from dsetool when SSL is enabled. (DSP-15171)

5.0.12 Cassandra changes 

DataStax Enterprise 5.0.12 certifies Apache Cassandra™ 3.0.13 with additional production-certified Cassandra changes.

5.0.12 TinkerPop changes 

DataStax Enterprise (DSE) 5.0.12 includes all changes from previous releases plus these production-certified changes that are in addition to TinkerPop 3.2.8:
  • Improve type-safety in Gremlin.Net methods. (TINKERPOP-1752)
  • Fix for problems with hasId() fails for empty collections. (TINKERPOP-1802)
  • Python supports GraphSON types g:Date, g:Timestamp and g:UUID. (TINKERPOP-1807)
  • Improve error messaging on failed bytecode translation. (TINKERPOP-1811)
  • Graph API removed from usage in the process test suite. (TINKERPOP-1813/TINKERPOP-1814)
  • Consistent behavior of self-referencing edges. (TINKERPOP-1821)
  • Improve flexibility of detachment for EventStrategy. (TINKERPOP-1829)
  • Race condition in TinkerGraph index creation. (TINKERPOP-1830)
  • Bug fix in TraversalHelper.replaceStep. (TINKERPOP-1832)
  • API fix for DetachedEdge.Builder#setInV and setOutV doesn't return the builder. (TINKERPOP-1833)
  • Long forms of e and -i are now working. (TINKERPOP-1851)
  • union() can produce extra traversers. (TINKERPOP-1867)
  • Performance enhancement for OLAP: n^2 synchronious operation in OLAP WorkerExecutor.execute() method. (TINKERPOP-1870)

DSE 5.0.11 

19 October 2017

5.0.11 Components 

  • Apache Cassandra™ 3.0.13.1862 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1985 (updated)
  • Apache Spark™ 1.6.3.3
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.53
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.8
  • Hive Connector 0.4.47
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.7-20170905-ac15c412

5.0.11 Highlight 

A single change for DSE Search:
  • Due to CVE-2017-12629, removed Solr RunExecutableListener to harden security of non-secured clusters. (DSP-14818)

DSE 5.0.10 

Important: DataStax recommends installing the latest patch release. The latest version of DataStax Enterprise is 5.0.15.

21 September 2017

5.0.10 Components 

  • Apache Cassandra™ 3.0.13.1862 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1887 (updated)
  • Apache Spark™ 1.6.3.3
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.53
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.8 (updated)
  • Hive Connector 0.4.47 (updated)
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.7-20170905-ac15c412 (updated)

5.0.10 Highlights 

Executive summary highlights for DSE 5.0.10: The executive summary highlights are just a top-level view. Be sure to review all release notes.

DSE Enterprise 5.0.10 highlights 

DSE Analytics and DSEFS 5.0.10 highlights 

  • DSE Analytics safety feature prevents DSE from starting up if the Spark data directories are not present when DSEFS is enabled and the data directories are missing. This safety check resolves problems in earlier releases where DSE would start, but analytics nodes could experience hard-to-detect problems. (DSP-13238, DSP-13245)
  • DSEFS includes important stability fixes and performance improvements. To use DSEFS in production, DataStax strongly recommends upgrading DSE 5.0.9 and earlier installations to DSE 5.0.10 to leverage these improvements.

5.0.10 highlights 

  • improved DSE Graph Loader custom id support for multiple keyed vertices. (DGL-258)
  • Improved stability by limiting the number of supported vertex labels, disallowing the multi-properties for the partition key, removing provisioning during graph creation, and deprecating PVTs. (DSP-14300, DSP-11078, DSP-13501)
  • Upgraded TinkerPop to improve the performance and support DSL in Gremlin. (DSP-13545, DSP-13828, DSP-13467)

DSE Search 5.0.10 highlights 

DSE Search highlights include:
  • Improved stability and performance when dealing with non-indexed fields. (DSP-6501)
  • Fixed the memory leak issue when encrypting the index. (DSP-13826)

DataStax Enterprise 5.0.10 Changes and enhancements 

DataStax Enterprise core changes and enhancements 

Changes to nodetool repair. (DSP-14464)
  • When run without options on new tables, the default behavior is nodetool repair -full. (Earlier versions were incremental when no options were specified.)
  • When run without options on a keyspace or set of tables, nodetool repair runs incremental repair on tables previously repaired and full repair on new tables.
  • Anti-compaction is no longer run after full repairs. Use nodetool repair --run-anticompaction to restore the previous behavior.
  • Incremental repair is no longer supported on tables with MVs and CDC. An incremental repair executed on table with MVs or CDC will run full repair instead.

After upgrades from DSE 5.0.0-5.0.9 to DSE 5.0.10 or later, you must follow instructions in the upgrade guide to migrate off of incremental repairs. To continue running incremental repairs, use nodetool repair -inc.

DSE Analytics 5.0.10 changes and enhancements 

  • When ALLOW_SPARK_HOME=true, support to specify a user-specific Spark home directory with the SPARK_HOME environment variable. (DSP-8100)
  • Documentation for removing a node running DSEFS. (DSP-13701)
  • Improved troubleshooting capabilities by including keyspace name in Hive Connector CL log warning message. (DSP-14184)

DSEFS 5.0.10 changes and enhancements 

  • Performance and stability of listing DSEFS directories with thousands of entries is improved. (DSP-13812)
  • DSEFS read performance is improved. (DSP-13309)
  • Connection reuse is improved. Closing idle connections is disabled by default. New idle_connection_timeout_ms option in dse.yaml defines how long to wait before an idle client-server connection is closed. (DSP-14010)
  • For initial DSEFS connections only, the new DSEFS shell option --prefer-contact-points uses defined contact point to force connecting to a remote node, even if local node is available. (DSP-14108)

DSE Graph 5.0.10 changes and enhancements 

  • Improved and simplified data batch loading of preformatted data. (DGL-235)

    Supporting changes:

    • Schema discovery and schema generation are deprecated. (DGL-246)
    • Standard IDs are deprecated. (DGL-247)
    • Transformations are deprecated. (DGL-248)
    • Standard vertex IDs are deprecated. Use custom vertex IDs instead. (DSP-13485)
  • improved DSE Graph Loader custom id support for multiple keyed vertices. (DGL-258)
  • Don't instantiate DseQueryHandler for each statement in graph. (DSP-13287)
  • Graph profile() results should display CQL by default even in console. (DSP-13292)
  • GraphSON 2.0 serialization performance enhancements. (DSP-13467)
  • Partitioned vertex tables (PVTs) are deprecated. (DSP-13501)
  • Do partition deletes for the property/edge table entries, if possible. (DSP-13671)
  • Remove provisioning state during graph creation. Graph is either live or non-existing. (DSP-13686)
  • Improved support for domain-specific languages (DSL) in Gremlin enables the DataStax driver to specify TraversalSource. (DSP-13545)
  • cache=false at the transaction level now includes disabling AdjacencyListStoreImpl and IndexStoreImpl. (DSP-13560)
  • Timeouts for graph traversals now start from the time the request is received. Earlier releases started timeouts for graph traversals at processing start time. Timeouts will appear more readily on an overloaded server. (DSP-13828)
  • Numeric sack values no longer need to be explicitly typed (for example, 3.0D). You can still provide for greater specificity in the expected return type. (DSP-14026)
  • Lambdas provided to the sack() step are now recognized by the LambdaRestrictionStrategy. You must disable the restrict_lambda setting to call this method. (DSP-14118)
  • Gremlin console uses the default plugins.txt in the DSE distribution. If a user home is specified with bin/dse gremlin-console ~/gremlin-console then extra checks are performed to ensure that plugins.txt is populated. (DSP-14286)
  • Prevent multi-properties for the partition/clustering key. (DSP-14300)

DSE Search 5.0.10 changes and enhancements 

  • Full validation on all schema fields might result in validation failures after upgrade. (DSP-6501)
    • All field definitions in the schema are validated and must be DSE Search compatible, even if the fields are not indexed, have docValues applied, or used for copy-field source.
    • Tune the schema before you upgrade. All field definitions in the schema are validated and must be DSE Search compatible, even if the fields are not indexed, have docValues applied, or used for copy-field source. With the tuned index, performance gains are especially recognized for unused large blobs.
  • rtOffheapPostings is present by default in demo and auto-generated solrconfig.xml files. (DSP-10088)
  • The default filter cache settings are changed. (DSP-13153)
  • DSE Search indexing optimizes for SSDs by default. Spinning disk detection logic is removed. (DSP-13924)
  • Improved error messages on invalid solr_query are more descriptive for invalid queries and syntax errors. (DSP-14003)
  • The table system_auth.resource_role_permissons_index is no longer used. Drop this table after all nodes are upgraded to DSE 5.0.10. upgrades from DSE 5.0.10+ to DSE versions earlier than 5.1.3 are not recommended. (DSP-14159)

5.0.10 Resolved issues 

5.0.10 DataStax Enterprise core resolved issues 

  • Update python driver in cqlsh to fix CASSANDRA-12847. (DSP-13509)
  • dsetool ring prints ERROR when data_file_directories is removed from cassandra.yaml. (DSP-13547)
  • Adjust and check directory ownership when starting DSE. (DSP-13245)
  • CVE-2017-7957 xstream-core is vulnerable to Denial of Service (DoS) attacks. (DSP-13419)
  • Apache HttpClient directory traversal through malformed URI. (DSP-13580)
  • MemoryOnlyStrategy regions not immediately loaded into physical memory with new kernels. (DSP-14169)
  • DSE installer fails on upgrade from DSE 4.8.x to DSE 5.0.x. (DSP-14170)
  • Make scrub validate the partition key, add validation to schema mutation creation. (DSP-14366)
  • SSTable index files can become corrupted due to StreamingHistogram bug. (DSP-14279)
  • New nodetool mark_unrepaired command unifies repaired and unrepaired compaction buckets. (DSP-14255)
  • Make full repair default and disallow incremental repair on MV tables. (DSP-14255)
  • Revert CASSANDRA-11223 behavior in AbstractReadCommandBuilder. (DSP-14135)
  • Simplify role-permissions handling (DSP-14159)
  • Prevent marking remote SSTables shadowing compacted data as repaired. (DSP-14141)
  • CPU layout assertions on startup should show in log file instead of stopping startup. (DSP-14281)
  • Completed rebuild logging always says 0 bytes. (DSP-13870)
  • nodetool rebuild and bootstrap improvements. (DSP-13870)
  • Incremental repairs are the default for nodetool repair, even with nodetool repair -full or nodetool repair -pr are incremental. (DSP-14464)

    After upgrades from DSE 5.0.0-5.0.9 to DSE 5.0.10, you must follow instructions in the upgrade guide to migrate off of incremental repairs. To continue running incremental repairs, use nodetool repair -inc.

5.0.10 DSE Analytics resolved issues 

  • MultipleRetry policy may retry with an incorrect consistency level. (DSP-13542)
  • Spark-sql throws error when using an existing but hidden keyspace. (DSP-13556)
  • When Spark Worker cannot connect to the master for a long time, it is automatically restarted. (DSP-13724)
  • DSEFS does not support symlink for data directories. (DSP-14110)

5.0.10 DSEFS resolved issues 

  • NullPointerException: Unexpected null value of column valid_from in <dse keyspace>.inodes while running fsck. (DSP-12615)
  • Memory leak occurs with incorrect use of WebHDFS API. (DSP-13813)
  • DSE will not start if DSEFS is enabled and fails to start due to a configuration problem. (DSP-13238)
  • Handling unclosed DSEFS streams causes applications to hang before exiting. (DSP-14010)
  • The service dse stop command does not wait for the process to be completely stopped. (DSP-14014)

5.0.10 DSE Graph resolved issues 

  • -help prints help twice. (DGL-257)
  • DGL prints warning excessively. (DGL-262)
  • The number of vertex labels is limited to 200 per graph. (DSP-11078)
  • Gremlin server log directory setting doesn't work if default log location is moved. Use dse-env.sh to change log locations. (DSP-13508)
  • Backward compatibility issue with .select() .by() or local(). (DSP-13607)
  • Remove unnecessary sharedData.refresh() calls when creating/dropping a graph. (DSP-13779)
  • Rare client-side ParsingException. (DSP-14000)
  • Reintroduce provisioning/dropping states for backward compatibility. Issue a warning when a graph is found. (DSP-14111)

5.0.10 DSE Search resolved issues 

  • Shard request exceptions are not logged at the replica level. (DSP-12691)
  • Unnecessary double segment flushing on hard commit. (DSP-13971)
  • Single-pass CQL Solr queries cannot select some data types. (DSP-14022)
  • Non-indexed field prevents data from being indexed. (DSP-14001)
  • Parsing error on cleanup of Solr secondary index with empty string in partition ID. (DSP-14234)
  • Allow dynamic multi-valued fields without a corresponding CQL column. (DSP-13277)
  • Memory leak during index encryption. (DSP-13826)
  • Non-indexed frozen map column produces unexpected results without error message. (DSP-13997)

5.0.10 Cassandra changes 

DataStax Enterprise 5.0.10 certifies Apache Cassandra™ 3.0.13 with additional production-certified Cassandra changes.

DSE 5.0.9 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

29 June 2017

5.0.9 Components 

  • Apache Cassandra™ 3.0.13.1735 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1715 (updated)
  • Apache Spark™ 1.6.3.3
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.53 (updated)
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.7 (updated)
  • DSEFS 5.0.9, starting with DSE 5.0.9 DSEFS matches the DSE release
  • Hive Connector 0.4.45
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.5-20170531-e1d94660 (updated)

5.0.9 Highlights 

Executive summary highlights for DSE 5.0.9: The executive summary highlights are just a top-level view. Be sure to review all release notes.

5.0.9 DSE Enterprise highlights 

DataStax Enterprise 5.0.9 includes CASSANDRA-13004 that fixes possible corruption while adding a column to a table or removing a column from a table. (DSP-13684)

This fix requires a messaging protocol version change to VERSION_3014. DataStax strongly recommends additional steps for upgrades from 5.0.0 through 5.0.8 to DSE 5.0.9 and later.

5.0.9 DSE Analytics and DSEFS highlights 

  • Improved DSEFS performance when authorization is enabled. (DSP-13107).
  • The SparkBuildExample project was updated with tags to cover several recent DSE releases.

5.0.9 DSE Search highlights 

DSE Search highlights include:
  • Security and load-balancing issues in the Solr HTTP interface issues introduced in DSE 5.0.0 are resolved. (DSP-13317, DSP-13270)
  • Auto-tuning issue on misconfigured hardware introduced in 5.0.8 is resolved. (DSP-13241)
  • A potentially severe memory leak in index encryption introduced in 5.0.0 is resolved. (DSP-13155)

5.0.9 Changes and enhancements 

Changes and enhancements include:

5.0.9 DataStax Enterprise core changes and enhancements 

  • New nodetool sjk command for troubleshooting and monitoring that runs Swiss Java Knife (SJK) on the local node. (DSP-13544)
  • Skip complex column range tombstones on simple columns. (DSP-13644)
  • Fail remaining ongoing tasks when aborting a repair job. (DSP-13544)
  • On node failure, fail only repair tasks of that node. (DSP-13544)
  • Coerce integer JSON values to timestamp as well as long values. (DSP-13644)
  • Integrate top threads command in nodetool. (DSP-13544)

5.0.9 DSE Analytics changes and enhancements 

  • DSEFS: Improve error message when DSEFS is low on storage space. (DSP-13324)
  • DSEFS: Add local cache for name and inode retrieval to improve authorization performance. (DSP-13107)

5.0.9 DSE Graph changes and enhancements 

  • Make explicit parameter for setting tmp dir for mapdb and netty. (DGL-167)
  • Support recursive loading of directories. (DGL-172)
  • Specify file matching pattern for directory load. (DGL-177)
  • Remove double cluster client in ClusterBuilder. Instead, use a single client and configure the CL in a SimpleGraphStatement for creating the graph. (DGL-183)
  • Support for geospatial data types. (DGL-225)
  • Change in schema definition for geo types. For each geo type, you must declare type as geo bound or cartesian bound. (DGL-243)
  • Include additional remote error information for Gremlin processing failures with TinkerPop protocol. (DSP-13112)
  • A change is required if more than 256 parameters are passed on a graph query request for TinkerPop drivers and drivers using Cassandra native protocol. Passing very large numbers of parameters on requests is an anti-pattern, because the script evaluation time increases proportionally. DataStax recommends reducing the number of parameters to speed up script compilation times. Consider alternate methods for parameterizing scripts, like passing a single map. If the graph query request requires many arguments, pass a list. If you pass more than 256 parameters, increase the max_query_params option in dse.yaml. (DSP-12789)

5.0.9 DSE Search changes and enhancements 

  • A new CQL type tinyint. (DSP-11940)
  • Runtime node blacklisting for distributed search queries; the EndpointStateTracker MBean now has Blacklisted boolean attribute. (DSP-12965)
  • Display reindexing progress with dsetool core_indexing_status --progress option. (DSP-12617)
  • Support for indexing frozen sets and lists of native and user-defined (tuple/UDT) element types. (DSP-12983)
  • Changes in default behavior for auto-generated solr schemas to enable DocValues. (DSP-13345)

5.0.9 Resolved issues 

5.0.9 DataStax Enterprise core resolved issues 

  • Snappy libraries have wide open permissions. (DSP-11668)
  • CqlSlowLogPlugin can fail to determine the table name of a DropIndexStatement if the index was dropped already. (DSP-11811)
  • SafeNet/KMIP authentication failure via LDAP. (DSP-12739)
  • dsetool logs clear credentials on logs. (DSP-12985)
  • Possible corruption while adding a column to a table or removing a column from a table, see CASSANDRA-13004. (DSP-13684)

5.0.9 DSE Analytics resolved issues 

  • When DSE node with Spark Master gracefully shuts down at the same time that an application is submitted or stopped, Spark Master fails to save the recovery storage information. (DSP-12795)
  • Change lease manager log message to improve Spark Master troubleshooting. (DSP-12846)

5.0.9 DSE Graph resolved issues 

  • Don't create a graph in dryrun mode if it doesn't exist. (DGL-159)
  • Specify file matching pattern for directory load. (DGL-177)
  • DGL creates duplicate edges when rerunning when using custom ids. (DGL-205)
  • Graph loader loads entire grapshon and gryo files in to memory. (DGL-209)
  • Properties with empty strings are skipped. New graph loader -skip_blank_values option. (DGL-215)
  • RangeStep fails when used with negative values. (DSP-11671)
  • Logging level in DigestTokensManager lowered from INFO to DEBUG. (DSP-12234)
  • Return bulk information in responses to DSE drivers. (DSP-12391)
  • Race condition can cause Spark Executor creation loop during DSE node shutdown. (DSP-12589)
  • Read consistency on graphs can be set to nonsense. (DSP-12650)
  • Order propertyKeys correctly in schema.describe(). (DSP-12761)
  • Gremlin scripts taking a long time to compile. See required change if more than 256 parameters are passed on a graph query request. (DSP-12789)
  • gremlin-console isn't properly initialized when started in debug mode. (DSP-12900)
  • Incomplete error message for invalid gremlin query with addV(). (DSP-13108)
  • Exception thrown when attempting to read IDs of vertices retrieved through a full-graph scan. (DSP-13210)
  • Graph should start listening to schema updates only after DSE system keyspace is set up. (DSP-13251)
  • Cache empty result sets for queries that didn't return elements. (DSP-13342)
  • Change ranking of indices so that Search index < Secondary Index < MV index. (DSP-13212)
  • If a gremlin recursive drop() times out, database can be left in an inconsistent state. (DSP-13299)
  • Gremlin inside() function no longer uses search index. (DSP-13553)
  • Remove unnecessary sharedData.refresh() calls when creating/dropping a graph. (DSP-13779)

5.0.9 DSE Search resolved issues 

  • Remove <dataDir> option from solrConfig files in demo apps. (DSP-9402)
  • CQL Search queries time out when a column has a colon (:) in it. Solr field name policy applies to DSE Search field names. (DSP-11296)
  • Error out on tuple subfield sorting and faceting. (DSP-11907)
  • Poor performance when searching with UDT sub-fields. (DSP-12812)
  • Optimize solr .within() queries correctly. (DSP-12830)
  • Excessive StatefulEncryptorAdapter usage by evicting StatefulEncryptorAdapter cache when index output gets closed. (DSP-13155)
  • Search indexing defaults auto-tuning responds poorly to unreliable /proc/cpuinfo data. (DSP-13241)
  • Solr is accepting HTTP requests before all cores have loaded. (DSP-13270)
  • Reduce the severity of logging around failed auto-soft commit. (DSP-13672)

5.0.9 Cassandra changes 

DataStax Enterprise 5.0.9 certifies Apache Cassandra™ 3.0.13 with additional production-certified Cassandra changes.

DSE 5.0.8 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

7 April 2017

5.0.8 Components 

  • Apache Cassandra™ 3.0.12.1656 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1619 (updated)
  • Apache Spark™ 1.6.3.3 (updated)
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.5 (updated)
  • DSEFS 5.0.19
  • Hive Connector 0.4.45 (updated)
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.5 (updated)

5.0.8 Changes and enhancements 

Changes and enhancements include:

5.0.8 DSE Analytics changes and enhancements 

  • Improved performance of writes in DSEFS. (DSP-12107)

5.0.8 DSE Graph changes and enhancements 

  • The default value of schema_agreement_timeout_in_ms is reverted to 10 seconds (10000 ms). (DSP-12419)
  • Support for geo polygon search on graph with a geo index on a vertex property. (DSP-12638)

    JTS must be installed before the search index is created for polygon search to work. (DSP-12638)

  • VertexInputRDD.getOrCreateVertex method performance improvement; Graph OLAP query running time reduced by ~10%. (DSP-12782)

5.0.8 DSE Search changes and enhancements 

  • Improve default selection for dse.yaml and solrconfig.xml write path configuration. See Configuring and tuning indexing performance. (DSP-12491)
  • Default batch size for the search TTL Process is changed. (DSP-11493)

    When a value is not specified for ttl_index_rebuild_options.max_docs_per_batch in dse.yaml, the default is changed from 100 to 4096.

  • Improved error handling for authentication and authorization of Solr HTTP requests and Solr Admin UI. (DSP-12550)

    Requests that fail due to lack of permissions return a 403 error, not a 401 error that was returned in earlier versions.

  • When using the Solr Admin UI, users require SELECT permissions on any search core that they view. Specific permissions are required for all core operations. (DSP-11910)
  • Remove unnecessary database reads in the scheduled Solr document expiration process. (DSP-12861)

5.0.8 Known issue 

  • Search indexing defaults auto-tuning responds poorly to unreliable /proc/cpuinfo data. (DSP-13241)

5.0.8 Resolved issues 

5.0.8 DataStax Enterprise core resolved issues 

  • Synchronize StorageService and DseDaemon shutdown hooks. (DSP-8421)
  • cqlsh SOURCE command shouldn't assume PlainTextAuthenticator. (DSP-12773)
  • Passing credentials through DSE_USERNAME and DSE_PASSWORD is broken when dserc file exists. (DSP-12602)
  • Cannot reload core after thrift table upgrade from DSE 4.8 to 5.0. (DSP-13019)
  • Handle missing metadata when opening a keyspace. (DSP-13490)
  • Bounded range splitting token allocation for first RF nodes. (DSP-12784)
  • Implement the NoReplicationTokenAllocator. (DSP-12784)
  • Log when waiting remaining repair tasks to finish. (DSP-12784)
  • Avoid AssertionError when shutting down finished repair executor. (DSP-12784)
  • cqlsh SOURCE cmd shouldn't assume PlainTextAuthenticator. (DSP-12781)
  • Add allocate_tokens_for_local_replication_factor flag (DSP-12781)
  • Skip flush on repair streaming. (DSP-12781)
  • Minor optimization in GPFS, and related snitches. (DSP-12740)
  • Issue CQL native protocol warning that DTCS is deprecated. (DSP-12740)
  • Log message when rebuild has finished. (DSP-12740)
  • Improve repair overstreaming for multi-DCs and large RFs. (DSP-12740)

5.0.8 DSE Analytics resolved issues 

  • Memory leaks detected during Spark usage. (DSP-8752)
  • On start, Spark worker registers with master that is then changed, but doesn't reregister with new master. (DSP-11306)
  • Don't audit log graph internal queries. Only queries of type GRAPH_TINKERPOP_TRAVERSAL are written to the audit log. (DSP-11911)
  • Benign DSEFS broken pipe exceptions reported in system.log on reading from Spark. (DSP-12586)
  • Unnecessary SparkMaster restarts. (DSP-12585)
  • Retry few times retrieval of configuration from DSE node before failing. (DSP-12588)
  • Weather sensor demo website not graphing all data values. (DSP-13041)

5.0.8 DSE Graph resolved issues 

  • Provide directory input source. (DGL-208)
  • VertexCache should be bypassed with custom ids. (DGL-192)
  • GraphEventPluginImpl tries to use ThreadPoolPlugin before it is activated. (DSP-10597)
  • In graphs, multiple property key definitions existed with names that differed only by case ("id" and "ID"), OLAP queries return the value of only one key. (DSP-12137)
  • OLAP doesn't work with SSL. (DSP-12179)
  • Numeric overflow when calculating cache size in bytes prevents DSE from starting. (DSP-12537)
  • Improve handling of ASM "Method code too large" exception when processing large Gremlin script. (DSP-12802)
  • Many threads get stuck compiling the same script. (DSP-12814)
  • CassandraFSPlugin fails occasionally on startup, preventing the node from booting up. (DSP-12873)
  • Graph doesn't work with DseAuthenticator and DseAuthorizer and authentication and authorization options are false. (DSP-12937)
  • Comparing IDs of newly created elements with normal elements causes a class cast exception. (DSP-12738)
  • Check that a new ID given to a schema element has not already been used. (DSP-12826)
  • Vertex properties without meta-properties defined in schema create invalid RDD data. (DSP-13016)
  • OLAP case sensitivity for edges and meta-properties. (DSP-13085)

5.0.8 DSE Search resolved issues 

  • Remove unnecessary Solr resource repair on core creation and reloading. (DSP-10270)
  • Make TimeUUIDField epoch not platform-dependent. (DSP-11424)
  • CQL should show error on using unsupported column aliases with solr_query queries. (DSP-12500)
  • Heap is exhausted while search re-indexes very wide partitions. New IndexPool MBean attributes. (DSP-12547)
  • Solr queries don't work with partition restriction that has a space. (DSP-12610)
  • BlockCache corruption with high concurrency. (DSP-12620)
  • Combining normal column + 2 UDT subfields search requires 'fq' syntax. (DSP-12347)
  • Native driver connections in dsetool aren't isolated to specified host. (DSP-12438)
  • Core is not correctly unloaded on restarted nodes. (DSP-12434)
  • DistributedRequestException isn't created with a detail message. (DSP-12493)
  • Improved error handling for authentication and authorization of Solr HTTP requests and Solr Admin UI. (DSP-12550)

    Requests that fail due to lack of permissions return a 403 error, not a 401 error that was returned in earlier versions.

  • Concurrent sorting issue with RT. (DSP-12600)
  • Better TTL logging. (DSP-12885)
  • After providing wrong schema next reload fails with mappingVersion. (DSP-12908)
  • The TTL task is never de-scheduled. (DSP-12960)

5.0.8 Cassandra changes 

DataStax Enterprise 5.0.8 certifies Apache Cassandra™ 3.0.12 with additional production-certified Cassandra changes.

DSE 5.0.7 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

2 March 2017

5.0.7 Components 

  • Apache Cassandra™ 3.0.11.1586 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1531
  • Apache Spark™ 1.6.3.1
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.4
  • DSEFS 5.0.19
  • Hive Connector 0.4.44
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.4

5.0.7 Resolved issue 

  • DSE failed unexpectedly during schema migration when creating 100 tables. (DSP-12698)
  • Handle missing metadata when opening a keyspace. (DSP-13490)
  • Bounded range splitting token allocation for first RF nodes. (DSP-12784
  • Log when waiting remaining repair tasks to finish. (DSP-12784)
  • Avoid AssertionError when shutting down finished repair executor. (DSP-12784)

5.0.7 Cassandra changes 

DataStax Enterprise 5.0.7 certifies Apache Cassandra™ 3.0.11 with additional production-certified Cassandra changes.

DSE 5.0.6 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

14 February 2017

5.0.6 Components 

  • Apache Cassandra™ 3.0.11.1564 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1531 (updated)
  • Apache Spark™ 1.6.3.1 (updated)
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.4 (updated)
  • DSEFS 5.0.19
  • Hive Connector 0.4.44
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.4

5.0.6 Known issues 

  • Cannot restart when PVT graph is present. (DSP-12540)
    Warning: If you use PVTs, do not upgrade to 5.0.6.

5.0.6 Changes and enhancements 

  • DSE graphloader has additional option, -create_graph (DGL-171)
  • DSE graphloader has additional options, -consistency_level, -allow_remote_hosts_in_quorum, and -remote_hosts_per_dc (DGL-175)

5.0.6 Resolved issues 

5.0.6 DataStax Enterprise core resolved issues 

  • Cassandra logback.xml shipped with maxIndex = 50. Throws error: Large window sizes are not allowed. (DSP-12061)
  • Perform repair sync sequentially to avoid overloading coordinator. (DSP-12096)
  • Filter out duplicate sstables when performing snapshots to avoid duplicate hardlink errors. (DSP-12416)
  • Prevent duplicate SyncUtil.force() in SeqentialWriter on close. (DSP-12416)
  • Disable preemptive sstable opening if sstable_preemptive_open_interval_in_mb <= 0, and warn about high GC pressure for values below 4. (DSP-12525)

5.0.6 DSE Analytics resolved issues 

  • When Spark Master changes and an OLAP query is run on the new Spark Master, gremlin creates new Spark application. (DSP-9333)
  • Verbose errors/warnings in Spark shells (dse spark, dse spark-sql). (DSP-10261)
  • spark-jobserver accessing dsefs file fails when cluster is secured with dseauth/internal. (DSP-11580)
  • Issue with Spark collection types with UDTs - SPARKC-363. (DSP-11991)
  • DSEFS shouldn't create a file if "get" fails. (DSP-12082)
  • DSEFS Make rm -r synchronous and report progress with rm -v. (DSP-12175)
  • DSEFS MatchError in the appendBlock portion of DsefsBlockService. (DSP-12354)
  • Wrong behavior of DSEFS Hadoop API. (DSP-12457)

5.0.6 DSE Graph resolved issues 

  • Metrics should be updated only after a query has succeeded. (DGL-187)
  • Partitions with custom IDs don't work. (DSP-11021)
  • -e doesn't work in dse gremlin-console. (DSP-11156)
  • realtime_evaluation_timeout gives rise to 2 different errors. (DSP-11470)
  • Graph Solr regular expression (regex) queries unnecessarily escape non-ASCII characters. (DSP-12129)
  • graph.snapshot() fails when authentication is enabled. (DSP-12147)
  • Creating a graph snapshot with no vertex label specified will result in an empty snapshot. (DSP-12224)
  • Increase default timeout when waiting for Cassandra schema agreement. (DSP-12235)
  • QueryUtils#getStandardVertexIdComponents is not thread safe. (DSP-12254)
  • Pvt cannot serialize ColumnIdentifier. (DSP-12240)
  • PVTs should handle empty update rows. (DSP-12323)
  • Unmodified Geo index is detected as changed. (DSP-12359)
  • IllegalMonitorStateException when using PVTs. (DSP-12378)
  • Graph executors should shutdown if the JVM is shutdown. (DSP-12325)
  • Hardcoded QUORUM ConsistencyLevel in SharedDataImpl. New consistency_mode option. (DSP-12510)
  • Enable split-DC graph ID allocation. (DSP-12516)

5.0.6 DSE Search resolved issues 

  • Stored=true copy fields are not supported and result in warning in the logs. (DSP-11087)
  • DSE Search upgrade to 4.8.x with kerberos does not allow to reload core on node0. (DSP-11462)
  • Solr query fails with "Lucene document does not contain unique field" when the unique key is not composite and is not stored. (DSP-12106)
  • Make UnInvertedField disabled by default. (DSP-12184)
  • Incorrect "range.facet.other" results for distributed range facet queries. (DSP-12198)
  • Upgrade failed with secure password: SolrStress run-benchmark read-cql failed with password: java.lang.RuntimeException: javax.security.auth.login.LoginException: No LoginModules configured for DseClient. (DSP-12057)

5.0.6 Cassandra changes 

DataStax Enterprise 5.0.6 certifies Apache Cassandra™ 3.0.11 with additional production-certified Cassandra changes.

DSE 5.0.5 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

21 December 2016

5.0.5 Components 

  • Apache Cassandra™ 3.0.11.1485 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1481 (updated)
  • Apache Spark™ 1.6.2.3 (updated)
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.3 (updated)
  • DSEFS 5.0.15 (updated)
  • Java Driver for Apache Cassandra 3.0.3
  • Hive Connector 0.4.44 (updated)
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150
  • TinkerPop 3.2.4

5.0.5 Changes and enhancements 

  • DSE Analytics: Support submitting Spark jobs from jars stored in DSEFS. (DSP-10862)
  • DSEFS: Improved performance of dsefs shell cat command. (DSP-11875)
  • DSE: Log the max allowed key length to identify whether the JCE extensions are installed. (DSP-11177)

5.0.5 Resolved issues 

5.0.5 DataStax Enterprise core resolved issues 

  • DseConnectionUtil.createCluster does not set cipher suite from configuration. (DSP-10624)
  • Rebuild of node/dc some DSE made keyspaces have their replication strategy reset to default replication. (DSP-11381)
  • On package install, dse start has error but starts fine. (DSP-11750)
  • Unresponsive nodes with high CPU utilization when using LDAP. (DSP-11980)
  • Perform repair sync sequentially to avoid overloading coordinator (DSP-12096)
  • Filter out duplicate sstables when performing snapshots to avoid duplicate hardlink errors (DSP-12416)
  • Always die on AssertionError and log flush failure errors. (DSP-12416)
  • Prevent duplicate SyncUtil.force() in SeqentialWriter on close (DSP-12416)

5.0.5 DSE Analytics resolved issues 

  • Thrift server fails to start on multiDC environment if a DC is down. (DSP-11617)
  • Spark has dependency on specific metrics JAR files. (DSP-11388)
  • MV not registered in Spark Metastore. (DSP-11642)
  • DSEFS throws timeout exceptions during heavy write load. (DSP-11678)
  • DSEFS: cfs-stress miscalculates data footprint when attempting to write 10GB+. (DSP-11738)

5.0.5 DSE Graph resolved issues 

  • Configurable environment variables in graph loader script make default heap more stable for users. (DGL-123)
  • Search index that changes indexing option will not be applied. (DSP-11116)
  • Traversal interruption is broken. (DSP-11321)
  • Don't log query errors from GraphStatement.java. (DSP-11448)
  • ClassCastException: HasContainer cannot be cast to DseHasContainer. (DSP-11574)
  • Graph queries without a vertex label will use an index if available. (DSP-11651)
  • Dropping edge property also drops edge. (DSP-11667)
  • Graph search indices should not be reindexed upon modification if properties are new. (DSP-11797)
  • PVT indexes are not being created. (DSP-11998)
  • Don't write PVT mutations to commitlog. (DSP-12062)

5.0.5 DSE Search resolved issues 

  • Selecting binary fields and tuples doesn't work with single pass queries. (DSP-11511)
  • dsetool create_core fails with Lock on opening new Searcher after unsuccessful dsetool reload_core. (DSP-11573)
  • solr_stress and CQL should work with Kerberos. (DSP-11650)
  • Tombstones are skipped and not indexed during repair. (DSP-11798)
  • Live docs query is not removed when reader is closed, causing excessive heap usage. (DSP-11819)

5.0.5 Cassandra changes 

DataStax Enterprise 5.0.5 certifies Apache Cassandra™ 3.0.11 with additional production-certified Cassandra changes.

DSE 5.0.4 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

16 November 2016

5.0.4 Components 

  • Apache Cassandra™ 3.0.7.1443 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1434 (updated)
  • Apache Spark™ 1.6.2.2
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • DataStax Spark Cassandra Connector 1.6.2 (updated)
  • Hive Connector 0.4.41
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Jobserver 0.6.2.150 (updated)
  • TinkerPop 3.2.4

5.0.4 Changes and enhancements 

  • DSE Search: Make shard token range selections optionally visible to the client. See Tracing distributed queries. (DSP-10506)
  • Make the maximum performance object background tasks different from maximum threads. (DSP-11463)

5.0.4 Resolved issues 

5.0.4 DataStax Enterprise core resolved issues 

  • Preflight check is incorrect about vm.max_map_count. (DSP-10706)
  • nodetool uses cassandra-env.sh MAX_HEAP_SIZE value if set. (DSP-11285)
  • nodetool startup script assumes JMX_PORT. (DSP-11590)
  • Wait for remaining tasks to finish on RepairJob after task failure (DSP-11640)
  • Allow the prepared statement cache size to be changed. (DSP-11420)

5.0.4 DSE Analytics resolved issues 

  • Deleting a directory or file holds locks after giving the control back to the client. (DSP-11282)
  • Allow for oversubscribing Spark Worker cores. (DSP-11460)
  • Touching global execution context may cause classloader issues. (DSP-11471)
  • Spark Jobserver demo broken. (DSP-11630)
  • Spark Jobserver logging broken. (DSP-11638)
  • Improved error handling and reporting in DSEFS. Fixed potential lockup if an I/O error occurs while writing a data block. (DSP-11349)

5.0.4 DSE Graph resolved issues 

  • Enable printing of DGL version. (DGL-133)
  • Preparing SDM queries. (DSP-10217)
  • "Could not find a suitable index" message should be more descriptive. (DSP-10744)
  • Date-typed property values don't work with bytecode-json graph language over native protocol. (DSP-11243)
  • Support for searching indexed LineStringType fields from graph. (DSP-11275)
  • Gremlin Time Machine examples are broken. (DSP-11249)
  • Graph cache metrics erroneously sharing hit/miss state across graphs. (DSP-11300)
  • Edge queries using hasLabel don't work. (DSP-11382)
  • Improve the caching efficiency of graph-generated Solr queries. (DSP-11479)

5.0.4 DSE Search resolved issues 

  • Lucene should do parallel delete processing for NRT and parallel creation of FrozenBufferedUpdates for RT. New indexConfig values. (DSP-10416)
  • Allow the prepared statement cache size to be changed. (DSP-11420)
  • Solr unnecessarily reads stored fields in HTTP queries with an explicit field list. (DSP-11469)
  • Add missing Tika 1.13 dependencies and replace unsafe version of Jackson. (DSP-11311)
  • Excessive live docs filters cause slow queries. (DSP-11512)
  • Search startup/reload performance issue resolved by turning off the reader and merge warming. (DSP-11498)

    To keep segment warming, add mergedSegmentWarmer to solrConfig.xml.

5.0.4 Cassandra changes 

DataStax Enterprise 5.0.4 certifies Apache Cassandra™ 3.0.7 with no additional production-certified Cassandra changes.

DSE 5.0.3 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

3 October 2016

5.0.3 Components 

  • Apache Cassandra™ 3.0.7.1346 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1347 (updated)
  • Apache Spark™ 1.6.2.2 (updated)
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DSEFS 0.8.9 (updated)
  • DataStax Java driver 1.0.0 for DSE 5.0.x
  • Hive Connector 0.4.41
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Connector 1.6.0
  • Spark Jobserver 0.6.2.100 (updated)
  • TinkerPop 3.2.3 (updated)

5.0.3 Changes and enhancements 

5.0.3 Resolved issues 

5.0.3 DataStax Enterprise core resolved issues 

  • dse cassandra-stop can return before cassandra exits. (DSP-8257)
  • cqlshrc.sample.kerberos & cqlshrc.sample.kerberos_ssl are incorrect. (DSP-10251)
  • service dse start timing out before JVM starts. (DSP-10478)
  • Add the ability to override use of SSL or Kerberos in dse client-tool. (DSP-10585)
  • Fixed indentation in dse.yaml. (DSP-10944)
  • The internode router doesn't work with reconnectable snitches, preventing query execution when using the GossipingPropertyFileSnitch with preferLocal set to true, and the Ec2MultiRegionSnitch. (DSP-11019)
  • otc_coalescing_strategy should be DISABLED by default in cassandra.yaml. (DSP-11140)

5.0.3 DSE Advanced Replication resolved issues 

  • Fix authentication and encryption for SSL remote cluster connections. New edge configuration parameters and dse.yaml setting. (DSP-9470)
  • Insert statement setting edge id (PK) results in an invalid query. (DSP-10714)
  • gc_grace_seconds value for replication log table is set too long. (DSP-10730)
    For existing clusters with versions 5.0.0, 5.0.1, or 5.0.2, you must manually adjust the setting in cqlsh:
    alter table dse_advrep.advrep_replication_log with gc_grace_seconds = 5;
  • Trigger creation/dropping fails when Cassandra authentication is on. (DSP-11142)

5.0.3 DSE Analytics resolved issues 

  • BYOS jar was fixed to support write to CFS. (DSP-10066)
  • Logging to stdout for DSEFS commands. (DSP-10897)
  • DSEFS client freezes when closed during read operation. For example, "dse hadoop fs -cat dsefs:///....." hangs when CTRL-C. (DSP-10907)
  • DSE spark failed to access HDFS in Kerborized Hadoop. (DSP-10224)
  • Spark app should not run when driver metadata persistence fails. (DSP-10324)
  • Spark encryption doesn't work for class sharing in REPL. Backport SPARK-11563. (DSP-10477)
  • Fixed mis-configured logging for spark-beeline. (DSP-10594)
  • Version mismatch between py4j python src and jar. (DSP-10895)
  • Spark Master in EverywhereStrategy keyspace has availability issues. (DSP-10941)

5.0.3 DSE Graph resolved issues 

  • Checking for edge duplication on load. (DGL-62)
  • Graphloader now supports DSE authentication, Kerberos use and SSL (DGL-109)
  • Nested map not being used correctly. (DGL-110)
  • Handle multi-cardinality property for schema generation. (DGL-128)
  • Graph data is stored across the multiple datacenters rather than localizing the data in the graph datacenter. (DSP-10267)
  • Property cache with search doesn't work. (DSP-10698)
  • Index on Text() property asText() doesn't work but asString() does. (DSP-10729)
  • DDLQueryBuilder overwrites the existing solrconfig.xml resource on index update. (DSP-11080)
  • VERIFY_UNIQUE should be a transaction level configuration. (DSP-11089)

5.0.3 DSE Search resolved issues 

  • Delayed failure when creating core with RAMDirectoryFactory, because RAMDirectoryFactory and other non-persistent directory factories are unsupported features. (DSP-10379)
  • Search failover handles multiple failure responses from a node incorrectly. (DSP-10454)
  • CQL 'date' and 'time' types not supported in DSE Search. (DSP-10654)
  • Make the error on tokenized unique key optional. (DSP-10657)
  • Add stack trace to validation errors. (DSP-10764)
  • Throw error for frozen collection in search. Frozen collections are not supported on the base Cassandra table. (DSP-10772)
  • dsetool ring error with RandomPartitioner. (DSP-10773)
  • Make it more difficult for auto soft commit to run after core shutdown. (DSP-10780)
  • TimeUUIDField distributed sort, schema generation, and docValues propagation. (DSP-10827)
  • Support for negative dates in search. (DSP-10913)

    To insert negative dates for the CQL timestamp type, insert an epoch time in milliseconds. The TimestampType does not accept a textual representation of negative dates.

  • Slow query log NPEs instead of logging slow statement. (DSP-10930)
  • Cannot remove a non-indexed column from table w/Solr secondary index. See Deleting Solr data. (DSP-11085)
  • SchemaTool does not handle empty bytebuffer values for timestamp/datetype. (DSP-11091)
  • DseAuth internal not working with DSEFS. (DSP-11494)
  • Add TermsComponent Back to Stress solrconfig.xml. (DSP-11495)

5.0.3 Known issues 

5.0.3 Cassandra changes 

DataStax Enterprise 5.0.3 certifies Apache Cassandra™ 3.0.7 with no additional production-certified Cassandra changes.

DSE 5.0.2 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

15 August 2016

5.0.2 Components 

  • Apache Cassandra™ 3.0.7.1293 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2.1241 (updated)
  • Apache Spark™ 1.6.2.1
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DSEFS 0.8.7 (updated)
  • Hive Connector 0.4.41 (updated)
  • DataStax Java driver 1.0.0 for DSE 5.0.x (updated)
  • Java Driver for Apache Cassandra 3.0.3
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Connector 1.6.0
  • Spark Jobserver 0.5.2
  • TinkerPop 3.2.1

5.0.2 Known issues 

  • CQL 'date' and 'time' types not supported in DSE Search. (DSP-10654)
  • If you used the DataStax Installer to upgrade to DSE 5.0.2, you might see yaml parse errors which cause the DSE server to not start properly. (DSP-10845)
    To resolve, use at least 4 spaces to indent these dse.yaml options:
    • histogram_data_options
    • user_level_latency_tracking_options
  • Lifecycle Manager (LCM) does not specify DSE Graph serializers appropriately in DSE 5.0.0 to DSE 5.0.2. (OPSC-10004, OPSC-10052)
  • Graph data is stored across the multiple datacenters rather than localizing the data in the graph datacenter. (DSP-10267)

    Before upgrading to DSE 5.0.2, make sure the replication factors for the graph_name and graph_name_system keyspaces are high before starting upgrades if you have graphs created in an earlier version.

    For graphs created in multi-datacenter clusters, the Cassandra and graph settings must use NetworkTopologyStrategy and a replication factor greater than one. If the graph is created with a replication setting of SimpleStrategy and a replication factor of 1, the graph data is stored across the multiple datacenters rather than localizing the data in the graph datacenter. See Creating a graph in the Gremlin console and Specifying Cassandra and graph settings.

5.0.2 Changes and enhancements 

5.0.2 DataStax Enterprise core changes and enhancements 

  • Doc URL is wrong in service start error message. (DSP-10420)
  • TieredStorageStrategy doesn't accept subproperties of the underlying compaction strategy. (DSP-10461)
  • DseVersionNotifier in Gossiper causes problem of rolling upgrade of a large cluster. (DSP-10520)

5.0.2 DSE Advanced Replication changes and enhancements 

  • DSE Advanced Replication metrics CLI not working, doesn't have proper default JMX port. (DSP-10491)
  • DSE Advanced Replication support for SSL and Kerberos authentication with client connections command options. (DSP-10560)

5.0.2 DSE Analytics changes and enhancements 

  • Add/enable support in Spark for S3 server-side encryption (SSE). (DSP-6548)
  • Allow for enabling Spark shared secret encryption and authentication separately. (DSP-9686)
  • Log stacktrace to a file on error in DSEFS shell. (DSP-10559)

5.0.2 DSE Graph changes and enhancements 

  • New dse.yaml settings to provide default values for gremlinPool. (DSP-10068)

5.0.2 DSE Search changes and enhancements 

  • Standardize Solr metrics. (DSP-8566)
  • Allow setting perf solrslowlog threshold in dsetool. (DSP-9970)
  • Add time-based execution to solr_stress. (DSP-10059)
  • Maximum size of an uploaded Solr resource is configurable in dse.yaml. (DSP-10253)
  • Give custom field types control over the type of their serialized value. (DSP-10418)

5.0.2 Resolved issues 

5.0.2 DataStax Enterprise core resolved issues 

  • Report configuration errors properly. (DSP-9605)
  • CVE-2014-4715 and CVE 2014-4611 - Driver vulnerability due to LZ4 and xxHash 1.2.0. (DSP-9910)
  • Add --help to dse remove-node command. (DSP-9988)
  • DSE tarball installs do not use $CASSANDRA_HOME when data directory locations are not set in cassandra.yaml. (DSP-10034)
  • CVE 2015-5262 - Update http-client used in Spark streaming demo to 4.5.2. (DSP-10041)
  • Throw MarshalException when ESRI library can't convert invalid geometry to well-known text (WKT). (DSP-10070)
  • Import cqltypes to get the geo functionality in cqlsh. (DSP-10170)
  • dse add-node --max-heap-size should accept 24G, 800M, or similar values, or fail on invalid values. (DSP-10131)
  • Normalize comments in dse.yaml. (DSP-10171)
  • Portfolio demo website start script should be marked as executable. (DSP-10247)
  • init script does not set ulimits correctly. (DSP-10358)
  • Fixed data corruption when upgrading thrift tables with Super Columns. (DSP-10602)
  • Delete statement is generated from an insert statement with null values. (DSP-10634)
  • Replication fails on insert row when all the primary and partition key values are specified and set ALL of the field values to be non-values (null values, tombstones). (DSP-10675)

5.0.2 DSE Analytics resolved issues 

  • Delegation token renewal does not work for DSE Spark applications. (DSP-9445)
  • logback-spark-server.xml has wrong fileNamePattern. (DSP-9890)
  • DSEFS memory leak if the client disconnects from the server during data transfer. (DSP-9889)
  • Weather demo queries don't work with SparkSQL. (DSP-9961)
  • Fix lease upgrade initialization race condition. (DSP-10073)
  • Spark options in dse.yaml are not ignored even though SPARK_ENABLED=0. (DSP-10023)
  • Fix accessing CFS in BYOS when DSE is secured with Kerberos and non-Yarn Spark is used. (DSP-10038)
  • Security issues with Hadoop delegation tokens. (DSP-10082)
  • Spark Worker error logs during shutdown. (DSP-10128)
  • Delegation token is exposed in processes list. (DSP-10227)
  • Resource leaks and data corruption might happen when reading multi-block files from DSEFS. (DSP-10245)
  • Portfolio demo web app broken. (DSP-10246)
  • When using HiveContext with SSL in Spark, the keystore password gets copied from truststore password. (DSP-10286)
  • Delegation token is not generated when Kerberos is not the default scheme. (DSP-10264)
  • Properly check Hive connector version number. (DSP-10409)

5.0.2 DSE Graph resolved issues 

  • Prevent EventType serialization. (DSP-10103)
  • Remove Northwind demo from DSE distribution, since it is in the graph loader. (DSP-10074)
  • Profiling shows query loses 3ms somewhere. (DSP-10400)
  • Use standard time units for graph configuration in dse.yaml. Option names and ISO-8601 format used in earlier releases are still valid, although deprecated. (DSP-9800)
  • Multi-node graph becomes zombie if dropped while a node is down. (DSP-9568)
  • Move graph post-install steps to dse-libgraph package (Debian). (DSP-10392)

5.0.2 DSE Search resolved issues 

  • Reduce logging on opening a new IndexSearcher. (DSP-9955)
  • Add support for specifying consistency levels in solr_stress (DSP-10174)
  • Provide a way to manually renew delegation tokens for BYOS. (DSP-10290)
  • Reintroduce the Solr validation log configuration. (DSP-10334)
  • Provide a way to disable Solr_Query custom payload for backwards compatibility. (DSP-10354)
  • Solr query results in NPE because of sort in requestHandler. (DSP-10362)
  • Make sure Solr field validation errors are logged only to the validation log. (DSP-10388)
  • Single-threaded Lucene parallel delete causes Out of Memory error when live indexing is enabled. (DSP-10415)
  • Number of server query worker threads is hard coded. (DSP-10519)

5.0.2 Cassandra changes 

DataStax Enterprise 5.0.2 certifies Apache Cassandra™ 3.0.7 with additional production-certified Cassandra changes.

DSE 5.0.1 

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.
Important: DataStax recommends the latest patch release. The latest version of DataStax Enterprise 5.0 is 5.0.15.

7 July 2016

5.0.1 Components 

  • Apache Cassandra™ 3.0.7.1159 (updated)
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2
  • Apache Spark™ 1.6.1.2
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DSEFS 0.8.3
  • Hive Connector 0.4.39
  • Java Driver for Apache Cassandra 3.0.2
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Connector 1.6.0
  • Spark Jobserver 0.5.2
  • TinkerPop 3.2.1

5.0.1 Known issues 

  • DSE Search CQL queries can fail for drivers which do not support Cassandra Native Protocol V4. (DSP-10354)
  • CQL 'date' and 'time' types not supported in DSE Search. (DSP-10654)
  • Data corruption when upgrading thrift tables with Super Columns. (DSP-10602)
    Warning: If you use Super Columns, do not upgrade to 5.0.1. Wait for DSE 5.0.2.

5.0.1 Resolved issue 

  • CFS is corrupted during upgrade from 4.8 to 5.0.0. (DSP-10299)

5.0.1 Cassandra changes 

DataStax Enterprise 5.0.1 certifies Apache Cassandra Cassandra™ 3.0.7 with additional production-certified Cassandra changes.

DSE 5.0.0 

28 June 2016

Important: DataStax recommends installing the latest patch release. Due to DSP-14464, DataStax does not recommend DSE 5.0.0-5.0.9 for production.

5.0 Components 

  • Apache Cassandra™ 3.0.7.1158
  • Apache Hadoop® 1.0.4.18
  • Apache Hive™ 0.12.0.13
  • Apache Mahout™ 0.8
  • Apache Pig™ 0.10.1
  • Apache Solr™ 4.10.3.2
  • Apache Spark™ 1.6.1.2
  • Apache Sqoop™ 1.4.5.15.1
  • Apache Tomcat® 6.0.45
  • DSEFS 0.8.3
  • Hive Connector 0.4.39
  • Java Driver for Apache Cassandra 3.0.0
  • Jetty 8.1.14.v20131031
  • Netty 4.0.34.Final
  • Spark Connector 1.6.0
  • Spark Jobserver 0.5.2
  • TinkerPop 3.2.1

5.0 New features 

DSE Graph 

DSE Advanced Replication 

DSE Multi-Instance 

DSE Tiered Storage 

5.0 Changes and enhancements 

Changes and enhancements for:

5.0 DataStax Enterprise changes and enhancements 

  • Support encryption of the Cassandra commit log. (DSP-2768)
  • DSE Unified Authentication enables multiple authentication schemes on DataStax Enterprise. (DSP-4191)
  • Deprecation of cassandra.yaml property values. (DSP-4838)
  • Set the slow query threshold as a percentile. (DSP-5089)
  • Full auditing of authentication using native protocol authenticators. (DSP-5963)
  • DataStax Enterprise has two metastores: one for Hive and one for Spark, Spark CQL, and Spark Thrift server. Internally, the tables in the Spark metastore use the Spark Cassandra source tables that are generated for the corresponding Cassandra database tables. (DSP-6204)
  • New connection options for dsetool to specify a truststore/certificate and password for client-to-node SSL. (DSP-6263)
  • New dsetool command to reload truststore certificates without rebooting. (DSP-6180)
  • Updated encryption hints to work with Cassandra 3.0 hints. (DSP-6421)
  • RPCs over Cassandra native protocol leverage Cassandra authentication and role-based access. Set RPC permissions. (DSP-6438)
  • In cassandra.yaml, new default is trickle_fsync: true, resulting in performance gain for SSDs. For platters, change to trickle_fsync: false. (DSP-6564)
  • dsetool search commands work with Cassandra password authentication. (DSP-6583)
  • Configure the uptime required for a node to reach full health. (DSP-6883)
  • CFSCompactionStrategy is deprecated. Use SizeTieredCompactionStrategy instead. (DSP-7872)
  • Package installation commands for Yum and APT support installation with only Cassandra core or with analytics, search, and DSE Graph. (DSP-8195)
  • Installation options for enabling different node types and hints directory. (DSP-8357)
  • Improved QueryMetrics accuracy. (DSP-8402)
  • cqlshrc file requires only the Kerberos service entry, qop defaults to auth, service defaults to dse. (DSP-8667)
  • Tomcat web server configuration file location is changed to /etc/dse/tomcat/conf. (DSP-8740)
  • Output of dsetool status is improved to show mixed workloads, node health, and graph enabled. (DSP-9399)
  • The dse demos are removed from package installations. The DataStax Installer has options to install DataStax Enterprise demos. (DSP-9596)

5.0 DSE Analytics changes and enhancements 

  • When you create multiple DSE Analytics datacenters, you must set an appropriate replication factor for the dse_leases keyspace or your Job Tracker is not elected. (DSP-4328)
  • Job Tracker elections are managed automatically. The ability to set the Job Tracker manually is removed. (DSP-5944)
  • DSE PySpark Scala wrappers are removed. PySpark and DSE PySpark are still supported using the more efficient DataFrames API. (DSP-6317)
  • RPCs over Cassandra native protocol leverage Cassandra authentication and role-based access. RPC access to DseClientTool is required to launch Spark. Set RPC permissions. (DSP-6438)
  • Spark Worker works inside DataStax Enterprise process and does not spawn a separate process. (DSP-6755)
  • By default, the Hadoop 2.7.1 client libraries are used, except for Hive and Pig, which use Hadoop 1.0.4 libraries with built-in Hadoop trackers. (DSP-7848)
  • Configurable number of slots and available memory for integrated Hadoop. (DSP-8134)
  • DSEFS is a new distributed file system within DataStax Enterprise that is intended primarily for Spark streaming use cases. CFS is deprecated and will be replaced by DSEFS. (DSP-8353)
  • Hadoop and Sqoop are deprecated. Use Spark instead. (DSP-7848)
  • Hadoop is deprecated for use with DataStax Enterprise. DSE Hadoop and BYOH (Bring Your Own Hadoop) are also deprecated. DataStax recommends using Spark for DSE Analytics. (DSP-9081)

5.0 DSE Search changes and enhancements 

  • Encrypt Solr indexes. (DSP-3368)
  • Support date queries on timeuuids with new custom type (solr.TimeUUIDField) that extends solr.UUIDField. (DSP-3905)
  • Improve the Performance objects performance by using common thread pool and scheduler; async_writers options in dse.yaml are removed. (DSP-4443)
  • Show numFound in CQL query results. (DSP-4586)
  • Ability to configure the slow query threshold as a percentage of the request times. (DSP-5089)
  • Optionally configure live indexing (RT) postings to be allocated offheap. (DSP-5491)
  • Auto inject CQLQueryHandler. (DSP-5831)
  • Encrypt the Solr commit log. (DSP-5955)
  • netty_server_port is deprecated. After all nodes in cluster are running DataStax Enterprise 5.0, the Inter-node messaging options coordinate requests from the local node. (DSP-5928)
  • Space saving profiles for automatic resource generation. (DSP-6375)
  • Index time boost support is removed. Use query time boosting instead. If you have any _docBoost columns in backing CQL tables, delete them after upgrading to DSE 5.0. (DSP-6507)
  • Support for geospatial point type in table schemas and queries. (DSP-6520)
  • New dsetool command to stop reindexing on a Solr core. (DSP-6384)
  • Indexing and querying support for JSON documents, CQL extension makes it easier to work with JSON documents. (DSP-6787)
  • Resource generation supports specifying indexed columns, and new coreOptionsInline option for dsetool create_core and infer_schema commands. (DSP-6900)
  • Clustering key range deletes should be reflected in Solr index. (DSP-7015)
  • dsetool encrypts Solr core without reindexing. (DSP-7083)
  • Lucene field cache is deprecated. Instead, set docValues="true" on the field. (DSP-7090)
  • Support indexing and searching for the point type. (DSP-7220)
  • Support for ShortType CQL columns. The CQL type smallint maps to the Solr type TrieIntField. (DSP-7273)
  • Lenient mode for generateResources=true skips unsupported columns, but still generates a schema and a core. (DSP-7555)
  • Add message to reload index after dsetool create_core when reindex=true is not specified. (DSP-7631)
  • Support all internode encryption options in DSE Search legacy netty transport. (DSP-8374)
  • Driver paging settings are enabled automatically for DSE SearchAnalytics workloads. (DSP-8618)
  • Make search auto-generation profile names camel case. (DSP-9773)
  • HTTP-based Solr shard transport option is deprecated. Use Inter-node messaging options instead. (DSP-9856)
  • Improve index tuning default selections for dse.yaml and solrconfig.xml write path configuration. (DSP-12491)

5.0 Known issues 

Known issues for 5.0:
  • TTL expiration timestamps are susceptible to the year 2038 problem. If the TTL value is long and an expiration date is greater than the maximum threshold of 2038-01-19T03:14:06+00:00, the data is immediately expired and purged on the next compaction. (DSP-15412)
    Warning: When using a long TTL, DataStax strongly recommends upgrading to DSE 5.0.12 or later and taking required action to protect against silent data loss.
  • Resource leaks and data corruption that may happen when reading multi-block files from DSEFS. (DSP-10245)

    Do not create files larger than one block. Set the block size of the file to a large value when creating a file or directory. Use -b block size option when creating directories and files.

  • CFS is corrupted during upgrade from 4.8 to 5.0.0. (DSP-10299)
    Warning: CFS is unusable after upgrading to DataStax Enterprise (DSE) 5.0.0 from any earlier DSE release. If you attempt to insert new data into CFS after upgrading to DSE 5.0.0, the data will not be inserted into CFS and an error message is not displayed.
    • If you are using CFS, do not upgrade to DataStax Enterprise 5.0.0. Instead, install DataStax Enterprise 5.0.1 or later.
    • If you are not using CFS, or using CFS in a new DataStax Enterprise 5.0.0 installation, you will not encounter this issue.
  • To improve index performance, set production-appropriate mergeScheduler values for DSE Search with near real time (NRT) indexing. (DSP-9325)
  • DSE Search: Select X as Y results in error on CQL Solr queries. (DSP-7945)
  • No out-of-the-box support for extracting text from encrypted PDFs with Tika that is included with DSE 5.0. (DSP-10072)
  • Even with nodetool repair -full or nodetool repair -pr, DSE 5.0.0-5.0.9 are run as incremental and mark sstables as repaired causing anti-compaction. (DSP-14464)
  • Potential data loss for INSERTs with very large TTLs, where "very large" is close to the maximum allowed value of 630720000 seconds (20 years), starting from 2018-01-19T03:14:06+00:00. As time progresses, the maximum supported TTL is gradually reduced as the maximum expiration date approaches. For instance, on 2028-01-19T03:14:06 with a TTL of 10 years is impacted. If you use very large TTLs, DataStax strongly recommends upgrading to 5.0.12 or later. (DSP-15412)

  • Upgrades from DataStax Enterprise 4.8.x to DataStax Enterprise 5.0 on RHEL-based systems incorrectly install DSE 5.1.8 when demos are installed. (DSP-15937)
    Workaround: For upgrades on RHEL-based systems that have demos installed, you must specify the package installation in a single line, and specify the version for dse-full and dse-demos. For example:
    sudo yum install dse-full-5.0.15-1  dse-demos-5.0.15-1
  • If DSE 5.1.8 was incorrectly installed:
    1. Uninstall DSE 5.1.8:
      sudo yum remove "dse-*" "datastax-*"
    2. Install the DSE 5.0.x version again:
      sudo yum install dse-full-5.0.15-1  dse-demos-5.0.15-1

5.0 Resolved issues 

5.0 DataStax Enterprise core resolved issues 

  • SSL + Kerberos qop=auth-int or auth-conf not reported as warning. (DSP-6939)
  • Forced joins should check for the correct syntax. (DSP-7625)
  • Update logback settings because of CASSANDRA-10241. (DSP-8762)
    Debug logging is on by default. The following actions are in debug.log:
    • Memtable flushing actions
    • Commit log replayed files
    • Compacted sstables
    • SStable opening (SSTableReader)
    To disable debug logging, comment out the ASYNCDEBUGLOG appender in the root level section:
    <root level="INFO">
        <appender-ref ref="SYSTEMLOG" />
        <appender-ref ref="STDOUT" />
        <appender-ref ref="ASYNCDEBUGLOG" /> <!-- Comment this line to disable debug.log -->
    ...
      </root>

5.0 DSE Analytics resolved issues 

  • The Spark Streaming Kafka demo is removed. (DSP-7725)
  • Spark shell variable hc is renamed to sqlContext to match OSS Spark naming conventions. CassandraSparkContext is removed. (DSP-7958)

5.0 DSE Search resolved issues 

  • Removed byoh-env.sh: PIG_PARTITIONER='dsetool partitioner' call to speed up BYOH start. (DSP-6962)
  • After creating Thrift table for data import handler, unable to create valid Solr core. (DSP-6982)
  • Wikipedia cql create_table does not work with LDAP. (DSP-7236)
  • The dsetool utility does not have argument for non-default Cassandra port. (DSP-7856)
  • Repair admin table so that creating/reloading cores doesn't fail when any nodes are down. (DSP-7948)
  • Fix live indexing sorted terms seek and iteration. (DSP-8009)
  • Optimize live indexing sorted terms seekExact with TermState. (DSP-8080)
  • create_core recovery doesn't work. (DSP-9891)
  • Dropping a table can force Cassandra shutdown if Solr core is still closing. (DSP-9907)

5.0 Cassandra changes 

DataStax Enterprise 5.0 certifies Apache Cassandra™ 3.0.7 with additional production-certified Cassandra changes.

5.0 Spark, Hive, and Hadoop changes 

DataStax Enterprise 5.0 certifies Spark™ 1.6.1.2, Hive™ 0.12.0.13, and Hadoop® 1.0.4.18 with these additional production-certified changes:
  • Remove httpclient dependency (HADOOP-10105)
  • Bump up commons-collections version to 3.2.2 to address a security flaw (HADOOP-12577)
  • Update apache httpclient version to 4.5.2; httpcore to 4.4.4 (HADOOP-12767)
  • HiveSessionImpl#fetchResults should not try to fetch operation log when hive.server2.logging.operation.enabled is false. (HIVE-8529)
  • Verify sem.getInputs() and sem.getOutputs() (HIVE-12875)
  • Spark Hive Thrift Server big decimal data issue (SPARK-12985)
The location of the dse.yaml file depends on the type of installation:
Installer-Services /etc/dse/dse.yaml
Package installations /etc/dse/dse.yaml
Installer-No Services install_location/resources/dse/conf/dse.yaml
Tarball installations install_location/resources/dse/conf/dse.yaml