DataStax Enterprise 6.0 release notes

DataStax Enterprise release notes include cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues for DataStax Enterprise 6.0.x.

DataStax Enterprise release notes cover cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues for DataStax Enterprise (DSE) 6.0.x.
Note: Each point release contains a highlights and executive summary section to provide guidance and add visibility to important improvements.

Requirement for uniform clusters

All nodes in each cluster must be uniformly licensed to use the same subscription. For example, if a cluster contains 5 nodes, all 5 nodes within that cluster must be either DataStax Basic, or all 5 nodes must be DataStax Enterprise. Mixing different subscriptions within a cluster is not permitted. “Cluster” means a collection of nodes running the software which communicate with one another via Gossip, and “Gossip” means the mechanism within the software enabling related nodes to communicate with one another. For more information, see Enterprise Terms.

Before you upgrade

The latest version of DataStax Enterprise is 6.0.2.

Be sure to read all of the relevant upgrade documentation, including Planning your DataStax Enterprise upgrade. Upgrades are supported from DSE 5.1 to 6.0 and from DSE 5.0 to 6.0.

  • Product compatibility
    • DataStax Enterprise 6.0 is compatible with:
      • DSE OpsCenter 6.5
      • DataStax Studio 6.0
    • DataStax Drivers

      Depending on the driver version, you might need to recompile your client application code. See Upgrading DataStax drivers.

    • DataStax Bulk Loader compatibility
      • Can load data into DSE 5.0 or later
      • Can unload data from any Apache CassandraTM 2.1 or later data source

DSE 6.0.2 release notes

dse.yaml

The location of the dse.yaml file depends on the type of installation:
Package installations /etc/dse/dse.yaml
Tarball installations installation_location/resources/dse/conf/dse.yaml

19 July 2018

6.0.2 Components

All components from DSE 6.0.0 are listed. Components that are updated for DSE 6.0.2 are indicated.
  • Apache Solr™ 6.0.1.1.2321 (updated)
  • Apache Spark™ 2.2.1.2
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.5
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.45 DSE custom version
  • TinkerPop 3.3.3 with additional production-certified changes (updated)

DSE 6.0 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.0.2 Highlights

High-value benefits of upgrading to DSE 6.0.2 include these highlights:

DSE Analytics and DSEFS

  • Fixed issue where CassandraConnectionConf creates excessive database connections and reports too many HashedWheelTimer instances. (DSP-16365)

DSE Graph

  • Fixed several edge cases of using search indexes. (DSP-14802, DSP-16292)

DSE Search

  • Search index permissions can be applied at the keyspace level. (DSP-15385)
  • Schemas with stored=true work because stored=true is ignored. The workaround for 6.0.x upgrades with schema.xml fields with “indexed=false, stored=true, docValues=true” is no longer required. (DSP-16392)
  • Minor bug fixes and error handling improvements. (DSP-16435, DSP-16061, DSP-16078)

6.0.2 DSE core

Changes and enhancements:
  • sstableloader supports custom config file locations. (DSP-16092)
  • -d option to create local encryption keys without configuring the directory in dse.yaml. (DSP-15380)
Resolved issues:
  • Tool sstablepartitions for identifying large partitions. (DB-803)
  • Show delegated snitch in nodetool describecluster. (DB-2057)
  • Use more precise grep patterns to prevent accidental matches in cassandra-env.sh. (DB-2114)
  • Add missing equality sign to SASI schema snapshot. (DB-2129)
  • For tables using DSE Tiered Storage, nodetool cleanup places cleaned SSTables in the wrong tier. (DB-2173)
  • Support creating system keys before the output directory is configured in dse.yaml. (DSP-15380)
  • Client prepared statements are not populated in system.prepared_statements table. (DSP-15900)
  • Improved compatibility with external tables stored in the DSE Metastore in remote systems. (DSP-16561)

6.0.2 DSE Analytics

Changes and enhancements:

  • Apache Hadoop Azure libraries for Hadoop 2.7.1 have been added to the Spark classpath to simplify integration with Microsoft Azure and Microsoft Azure Blob Storage. (DSP-15943)
  • AlwaysOn SQL (AOSS) improvements:
    • AlwaysOn SQL (AOSS) support for enabling Kerberos and SSL at the same time. (DSP-16087)
    • Add 120 seconds wait time so that Spark Master recovery process completes before status check of AlwaysOn SQL (AOSS) app. (DSP-16249)
    • AlwaysOn SQL (AOSS) driver continually runs on a node even when DSE is down. (DSP-16297)
    • AlwaysOn SQL (AOSS) binds to native_transport_address. (DSP-16469)
    • Improved defaults and errors for AlwaysOn SQL (AOSS) workpool. (DSP-16343)
Resolved issues:
  • Problems with temporary and data directories for Spark applications. (DSP-15476, DSP-15880)
    • DSE client applications, like Spark, will not start if user HOME environment variable is not defined, user home directory does not exist, or the current user does not have write permissions.
    • Temporary data directory for AOSS is /var/log/spark/rdd, the same as the server-side temporary data location for Spark. Configurable with SPARK_EXECUTOR_DIRS environment variable in spark-env.sh.
    • If TMPDIR environment variable is missing, /tmp is set for all DSE apps. If /tmp directory does not exist, it is created with 1777 permissions. If directory creation fails, perform a hard stop.
  • CassandraConnectionConf creates excessive database connections and reports too many HashedWheelTimer instances. (DSP-16365)
  • Need to disable cluster object JMX metrics report to prevent count exceptions spam in Spark driver log. (DSP-16442)
  • Fixed Spark-Connector dependencies and published SparkBuildExamples. (DSP-16699)

6.0.2 DSEFS

Changes and enhancements:

  • DSEFS operations: chown, chgrp, and chmod support recursive (-R) and verbose (-v) flag. (DSP-14238)
  • Client and internode connection improvements. (DSP-14284, DSP-16065)
  • DSEFS clients close idle connections after 60 seconds, configurable in dse.yaml. (DSP-14284)
  • Improvements to DataSourceInputStream remove possible lockup. (DSP-16409)
    • If the second read is issued after a failed read, it is not blocked forever. The stream is automatically closed on errors, and subsequent reads will fail with IllegalStateException.
    • The timeout message includes information about the underlying DataSource object.
    • No more reads are issued to the underlying DataSource after it reports hasMoreData = false.
    • The read loop has been simplified to properly move to the next buffer if the requested number of bytes hasn't been delivered yet.
    • Empty buffer returned from the DataSource when hasMoreData = true is not treated as an EOF. The read method validates offset and length arguments.
  • Security improvement: DSEFS uses an isolated native memory pool for file data and metadata sent between nodes. This isolation makes it harder to exploit potential memory management bugs. (DSP-16492)

Resolved issues:

  • DSEFS silently fails when TCP port 5599 is not open between nodes. (DSP-16101)

6.0.2 DSE Graph

Changes and enhancements:
  • Vertices and vertex properties created or modified with graphframes respect TTL as defined in the schema. In earlier versions, vertices and vertex properties had no TTL. Edges created or modified with graphframes continue to have no TTL. (DSP-15555)
  • Improved Gremlin console authentication configuration. (DSP-9905)
Resolved issues:
  • 0 (zero) is not treated as unlimited abort of max num errors. (DGL-307)
  • Search indexes are broken for multi cardinality properties. (DSP-14802)
  • DGF interceptor does not take into account GraphStep parameters with g.V(id) queries. (DSP-16172)
  • The clause LIMIT does not work in a graph traversal with search predicate TOKEN, returning only a subset of expected results. (DSP-16292)

6.0.2 DSE Search

Changes and enhancements:

  • The node health option uptime_ramp_up_period_seconds default value in dse.yaml is reduced to 3 hours (10800 seconds). (DSP-15752)
  • CQL solr_query supports Solr facet heatmaps. (DSP-16404)
  • Improved handling of asynchronous I/O timeouts during search read-before-write. (DSP-16061)
  • Schemas with stored=true work because stored=true is ignored. (DSP-16392)
  • Use monotonically increasing time source for search query execution latency calculation. (DSP-16435)

Resolved issues:

  • Search index permissions can be applied at keyspace level. (DSP-15835)
  • The encryptors thread cache in ThreadLocalIndexEncryptionConfiguration leaves entries in the cache. (DSP-16078)
  • Classpath conflict between Lucene and SASI versions of Snowball. (DSP-16116)
  • Indexing fails if fields have 'indexed=false', 'stored=true', and `docValues=true'. (DSP-16392)

DataStax Bulk Loader 1.1.0

Changes and enhancements:

Cassandra enhancements for DSE 6.0.2

DataStax Enterprise 6.0.2 is compatible with Apache Cassandra™ 3.11 and adds no production-certified enhancements.

DataStax Enterprise 6.0.2 is compatible with Apache Cassandra™ 3.11 and adds no production-certified enhancements.

General upgrade advice for DSE 6.0.2

General upgrade advice for DataStax Enterprise 6.0.2.

General upgrade advice for DataStax Enterprise (DSE) 6.0.2:
PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

4.0
===

New features
------------
   - SSTableDump now supports the -l option to output each partition as it's own json object
     See CASSANDRA-13848 for more detail
   - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
     See CASSANDRA-13132
   - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
     See CASSANDRA-11936
   - Support for arithmetic operations on number has been added. See CASSANDRA-11935
   - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
     consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
   - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
   - The initial build of materialized views can be parallelized. The number of concurrent builder
     threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
     This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.

Upgrading
---------
    - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables
      have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11.
      Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL.
      Syntax for creating compact tables is also deprecated.
    - Support for legacy auth tables in the system_auth keyspace (users,
      permissions, credentials) and the migration code has been removed. Migration
      of these legacy auth tables must have been completed before the upgrade to
      4.0 and the legacy tables must have been removed. See the 'Upgrading' section
      for version 2.2 for migration instructions.
    - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst
      Tother things, this imply the removal of all yaml option related to thrift
      ('start_rpc', rpc_port, ...).
    - Cassandra 4.0 removed support for any pre-3.0 format. This means you
      cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to
      a 3.0.x/3.x version first (and run upgradesstable). In particular, this
      mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
      will need to upgrade those sstable in 3.0.x/3.x first.
    - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous
      versions will causes issues during rolling upgrades (CASSANDRA-13274).
    - Cassandra will no longer allow invalid keyspace replication options, such
      as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
      add new nodes to a datacenter before they can set set ALTER or CREATE
      keyspace replication policies using that datacenter. Existing keyspaces
      will continue to operate, but CREATE and ALTER will validate that all
      datacenters specified exist in the cluster.
    - Cassandra 4.0 fixes a problem with incremental repair which caused repaired
      data to be inconsistent between nodes. The fix changes the behavior of both
      full and incremental repairs. For full repairs, data is no longer marked
      repaired. For incremental repairs, anticompaction is run at the beginning
      of the repair, instead of at the end. If incremental repair was being used
      prior to upgrading, a full repair should be run after upgrading to resolve
      any inconsistencies.
    - Config option index_interval has been removed (it was deprecated since 2.0)
    - Deprecated repair JMX APIs are removed.
    - The version of snappy-java has been upgraded to 1.1.2.6
    - the miniumum value for internode message timeouts is 10ms. Previously, any
      positive value was allowed. See cassandra.yaml entries like
      read_request_timeout_in_ms for more details.
    - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's
      documentation has been incorrect and the setting itself near useless.
      Batch mode remains a valid commit log mode, however.
    - There is a new commit log mode, group, which is similar to batch mode
      but blocks for up to a configurable number of milliseconds between disk flushes.
    - Due to the parallelization of the initial build of materialized views,
      the per token range view building status is stored in the new table
      `system.view_builds_in_progress`. The old table `system.views_builds_in_progress`
      is no longer used and can be removed. See CASSANDRA-12245 for more details.
    - nodetool clearsnapshot now required the --all flag to remove all snapshots.
      Previous behavior would delete all snapshots by default.

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and
      read_repair_chance table options have been deprecated, and will be removed entirely in 4.0.
      See CASSANDRA-13910 for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details


3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).


Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

TinkerPop changes for DSE 6.0.2

A list of DataStax Enterprise 6.0.2 production-certified enhancements to TinkerPop 3.2.9.

DataStax Enterprise (DSE) 6.0.2 includes these production-certified enhancements to TinkerPop 3.2.9:

  • Implemented TraversalSelectStep which allows to select() runtime-generated keys.
  • Coerced BulkSet to g:List in GraphSON 3.0.
  • Deprecated CredentialsGraph DSL in favor of CredentialsTraversalDsl which uses the recommended method for Gremlin DSL development.
  • Allowed iterate() to be called after profile().
  • Fixed regression issue where the HTTPChannelizer doesn’t instantiate the specified AuthenticationHandler.
  • Defaulted GLV tests for gremlin-python to run for GraphSON 3.0.
  • Fixed a bug with Tree serialization in GraphSON 3.0.
  • In gremlin-python, the GraphSON 3.0 g:Set type is now deserialized to List.

DSE 6.0.1 release notes

5 June 2018

6.0.1 Components

All components from DSE 6.0.0 are listed. Components that are updated for DSE 6.0.1 are indicated.
  • Apache Solr™ 6.0.1.1.2295 (updated)
  • Apache Spark™ 2.2.1.2 (updated)
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.5
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.45 DSE custom version (updated)
  • TinkerPop 3.3.2 with additional production-certified changes (updated)

DSE 6.0.1 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.0.1 Highlights

High-value benefits of upgrading to DSE 6.0.1 include these highlights:

DataStax Enterprise core

  • Fix binding JMX to any address. (DB-2081)
  • DataStax Bulk Loader 1.0.2 is bundled with DSE 6.0.1. (DSP-16206)

DSE Analytics and DSEFS

  • Upgrade to Spark 2.2.1 for bug fixes.
  • Fixed issue where multiple Spark Masters can be started on the same machine. (DSP-15636)
  • Improved Spark Master discovery and reliability. (DSP-15801, DSP-14405)
  • Improved AlwaysOn SQL (AOSS) startup reliability. (DSP-15871, DSP-15468, DSP-15695, DSP-15839)
  • Resolved the missing /tmp directory in DSEFS after fresh cluster installation. (DSP-16058)
  • Fixed handling of Parquet files with partitions. (DSP-16067)
  • Fixed the HashedWheelTimer leak in Spark Connector that affected BYOS. (DSP-15569)

DSE Search

  • Fix for the known issue that prevented using TTL (time-to-live) with DSE Search live indexing (RT indexing). (DSP-16038, DSP-14216)
  • Addresses security vulnerabilities in libraries packaged with DSE. (DSP-15978)
  • Fix for using faceting with non-zero offsets. (DSP-15946)
  • Fix for ORDER BY clauses in native CQL syntax. (DSP-16064)

6.0.1 DSE core

Changes and enhancements:
  • Improved NodeSync usability with secure environments. (DB-2034)
  • sstableloader supports custom config file locations. (DSP-16092)
  • LDAP tuning parameters allow all LDAP connection pool options to be set. (DSP-15948)
Resolved issues:
  • Use the indexed item type as backing table key validator of 2i on collections. (DB-1121)
  • Add getConcurrentCompactors to JMX in order to avoid loading DatabaseDescriptor to check its value in nodetool. (DB-1730)
  • Send a final error message when a continuous paging session is cancelled. (DB-1798)
  • Ignore empty counter cells on digest calculation. (DB-1881)
  • Apply view batchlog mutation parallel with local view mutations. (DB-1900)
  • Use same IO queue depth as Linux scheduler and advise against overriding it. (DB-1909)
  • Fix startup error message rejecting COMPACT STORAGE after upgrade. (DB-1916)
  • Improve user warnings on startup when libaio package is not installed. (DB-1917)
  • Set MX4J_ADDRESS to 127.0.0.1 if not explicitly set. (DB-1950)
  • Prevent OOM due to OutboundTcpConnection backlog by dropping request messages after the queue becomes too large. (DB-2001)
  • Fix exception in trace log messages of non-frozen user types. (DB-2005)
  • Limit max cached direct buffer on NIO to 1 MB. (DB-2028)
  • Reusing table ID with CREATE TABLE causes failure on restart. (DB-2032)
  • BulkLoader class exits without printing the stack trace. (DB-2033)
  • Fix binding JMX to any address. (DB-2081)
  • sstableloader does not decrypt passwords using config encryption in DSE. (DSP-13492)
  • dse client-tool help doesn't work if ~/.dserc file exists. (DSP-15869)

6.0.1 DSE Analytics

Known issues:

  • The Spark Jobserver demo has an incorrect version for the Spark Jobserver API. (DSP-15832)

    Workaround: In the demo's gradle.properties file, change the version from 0.6.2 to 0.6.2.238.

Changes and enhancements:
  • Decreased the number of exceptions logged during master move from node to node. (DSP-14405)
  • When querying remote cluster from Spark job, connector does not route requests to data replicas. (DSP-15202)
  • Long CassandraRDD.where clauses throw StackOverflow exceptions. (DSP-15438)
  • AlwaysOn SQL dependency on JPS is removed. The jps_directory entry in dse.yaml is removed. (DSP-15468)
  • Improved AlwaysOn SQL configuration. (DSP-15734)
  • Improved security for Spark JobServer. All uploaded JARs, temporary files, and logs are created under the current user's home directory: ~/.spark-jobserver. (DSP-15832)
  • Improved process scanning for AlwaysOn SQL driver. (DSP-15839)
  • In Portfolio demo, pricer is no longer required to be run with sudo. (DSP-15970)
  • Scala 2.10 in BYOS is no longer supported. (DSP-15999)
  • Improved validation for authentication configuration for AlwaysOn SQL. (DSP-16018)
  • Optimize memoizing converters for UDTs. (DSP-16121)
  • During misconfigured cluster bootstrap, the AlwaysOn SqlServer does not start due to missing /tmp/hive directory in DSEFS. (DSP-16058)
Resolved issues:
  • A shard request timeout caused an assertion error from Lucene getNumericDocValues in the log. (DSP-14216)
  • Multiple Spark Masters can be started on the same machine. (DSP-15636)
  • Do not start AlwaysOn SQL until Spark Master is ready. (DSP-15695)
  • DSE client tool returns wrong Spark Master address. (DSP-15801)
  • In some situations, AlwaysOn SQL cannot start unless DSE node is restarted. (DSP-15871)
  • Portfolio demo does not work on package installs. (DSP-15970)
  • Java driver in Spark Connector uses daemon threads to prevent shutdown hooks from being blocked by driver thread pools. (DSP-16051)
  • dse client-tool spark sql-schema --all exports definitions for solr_admin keyspace. (DSP-16073).
  • HashedWheelTimer leak in Spark Connector, affecting BYOS. (DSP-15569)

6.0.1 DSEFS

Resolved issues:
  • Can't quote file patterns in DSEFS shell. (DSP-15550)

6.0.1 DSE Graph

Changes and enhancements:
  • DseGraphFrame performance improvement reduces number of joins for count() and other id only queries. (DSP-15554)
  • Performance improvements for traversal execution with Fluent API and script-based executions. (DSP-15686)
Resolved issues:
  • edge_threads and vertex_threads can end up being 0. (DGL-305)
  • When using graph frames, cannot upload edges when ids for vertices are complex non-text ids. (DSP-15614)
  • CassandraHiveMetastore is prevented from adding multiple partitions for file-based data sources. Fixes MSCK REPAIR TABLE command. (DSP-16067)

6.0.1 DSE Search

Changes and enhancements:
Resolved issues:
  • Offline sstable tools fail is DSE Search index is present on a table. (DSP-15628)
  • HTTP read on solr_stress doesn't inject random data into placeholders. (DSP-15727)
  • Servlet container shutdown (Tomcat) prematurely stops logback context. (DSP-15807)
  • ERROR 500 on distributed http json.facet with non-zero offset. (DSP-15946)
  • Search index TTL Expiration thread loops without effect with live indexing (RT indexing). (DSP-16038)
  • Search incorrectly assumes only single-row ORDER BY clauses on first clustering key. (DSP-16064)

DataStax Bulk Loader 1.0.2

  • DataStax Bulk Loader 1.0.2 is bundled with DSE 6.0.1. (DSP-16206)
DataStax recommends using the latest DataStax Bulk Loader 1.2.0 For details, see DataStax Bulk Loader.

Cassandra enhancements for DSE 6.0.1

DataStax Enterprise 6.0.1 is compatible with Apache Cassandra™ 3.11 and adds production-certified enhancements.

DataStax Enterprise 6.0.1 is compatible with Apache Cassandra™ 3.11 and adds these production-certified changes:

  • cassandra-stress throws NPE if insert section isn't specified in user profile (CASSSANDRA-14426)
  • nodetool listsnapshots is missing local system keyspace snapshots (CASSANDRA-14381)
  • Remove string formatting lines from BufferPool hot path (CASSANDRA-14416)
  • Detect OpenJDK jvm type and architecture (CASSANDRA-12793)
  • Don't use guava collections in the non-system keyspace jmx attributes (CASSANDRA-12271)
  • Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
  • Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
  • Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
  • CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  • Serialize empty buffer as empty string for json output format (CASSANDRA-14245)
  • Cassandra not starting when using enhanced startup scripts in windows (CASSANDRA-14418)
  • Fix progress stats and units in compactionstats (CASSANDRA-12244)
  • Better handle missing partition columns in system_schema.columns (CASSANDRA-14379)
  • Deprecate background repair and probablistic read_repair_chance table options (CASSANDRA-13910)
  • Delay hints store excise by write timeout to avoid race with decommission (CASSANDRA-13740)
  • Add missed CQL keywords to documentation (CASSANDRA-14359)
  • Avoid deadlock when running nodetool refresh before node is fully up (CASSANDRA-14310)
  • Handle all exceptions when opening sstables (CASSANDRA-14202)
  • Handle incompletely written hint descriptors during startup (CASSANDRA-14080)
  • Handle repeat open bound from SRP in read repair (CASSANDRA-14330)
  • CqlRecordReader no longer quotes the keyspace when connecting, as the java driver will (CASSANDRA-10751)
  • Fix compaction failure caused by reading un-flushed data (CASSANDRA-12743)
  • Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
  • CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  • Check checksum before decompressing data (CASSANDRA-14284)

General upgrade advice DSE 6.0.1

General upgrade advice for DataStax Enterprise 6.0.1.

General upgrade advice for DataStax Enterprise (DSE) 6.0.1:
PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

4.0
===

New features
------------
   - SSTableDump now supports the -l option to output each partition as it's own json object
     See CASSANDRA-13848 for more detail
   - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
     See CASSANDRA-13132
   - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
     See CASSANDRA-11936
   - Support for arithmetic operations on number has been added. See CASSANDRA-11935
   - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
     consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
   - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
   - The initial build of materialized views can be parallelized. The number of concurrent builder
     threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
     This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.

Upgrading
---------
    - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables
      have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11.
      Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL.
      Syntax for creating compact tables is also deprecated.
    - Support for legacy auth tables in the system_auth keyspace (users,
      permissions, credentials) and the migration code has been removed. Migration
      of these legacy auth tables must have been completed before the upgrade to
      4.0 and the legacy tables must have been removed. See the 'Upgrading' section
      for version 2.2 for migration instructions.
    - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst
      Tother things, this imply the removal of all yaml option related to thrift
      ('start_rpc', rpc_port, ...).
    - Cassandra 4.0 removed support for any pre-3.0 format. This means you
      cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to
      a 3.0.x/3.x version first (and run upgradesstable). In particular, this
      mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
      will need to upgrade those sstable in 3.0.x/3.x first.
    - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous
      versions will causes issues during rolling upgrades (CASSANDRA-13274).
    - Cassandra will no longer allow invalid keyspace replication options, such
      as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
      add new nodes to a datacenter before they can set set ALTER or CREATE
      keyspace replication policies using that datacenter. Existing keyspaces
      will continue to operate, but CREATE and ALTER will validate that all
      datacenters specified exist in the cluster.
    - Cassandra 4.0 fixes a problem with incremental repair which caused repaired
      data to be inconsistent between nodes. The fix changes the behavior of both
      full and incremental repairs. For full repairs, data is no longer marked
      repaired. For incremental repairs, anticompaction is run at the beginning
      of the repair, instead of at the end. If incremental repair was being used
      prior to upgrading, a full repair should be run after upgrading to resolve
      any inconsistencies.
    - Config option index_interval has been removed (it was deprecated since 2.0)
    - Deprecated repair JMX APIs are removed.
    - The version of snappy-java has been upgraded to 1.1.2.6
    - the miniumum value for internode message timeouts is 10ms. Previously, any
      positive value was allowed. See cassandra.yaml entries like
      read_request_timeout_in_ms for more details.
    - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's
      documentation has been incorrect and the setting itself near useless.
      Batch mode remains a valid commit log mode, however.
    - There is a new commit log mode, group, which is similar to batch mode
      but blocks for up to a configurable number of milliseconds between disk flushes.
    - Due to the parallelization of the initial build of materialized views,
      the per token range view building status is stored in the new table
      `system.view_builds_in_progress`. The old table `system.views_builds_in_progress`
      is no longer used and can be removed. See CASSANDRA-12245 for more details.
    - nodetool clearsnapshot now required the --all flag to remove all snapshots.
      Previous behavior would delete all snapshots by default.

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and
      read_repair_chance table options have been deprecated, and will be removed entirely in 4.0.
      See CASSANDRA-13910 for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details


3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).


Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

TinkerPop changes for 6.0.1

A list of DataStax Enterprise 6.0.1 production-certified enhancements to TinkerPop 3.2.8.

DataStax Enterprise (DSE) 6.0.1 includes these production-certified enhancements to TinkerPop 3.2.9:
  • Performance enhancement to Bytecode deserialization. (TINKERPOP-1936)
  • Path history isn't preserved for keys in mutations. (TINKERPOP-1947)
  • Traversal construction performance enhancements (TINKERPOP-1950)
  • Bump to Groovy 2.4.15 - resolves a Groovy bug preventing Lambda creation in GLVs in some cases. (TINKERPOP-1953)

DSE 6.0.0 release notes

dse.yaml

The location of the dse.yaml file depends on the type of installation:
Package installations /etc/dse/dse.yaml
Tarball installations installation_location/resources/dse/conf/dse.yaml

cassandra.yaml

The location of the cassandra.yaml file depends on the type of installation:
Package installations /etc/dse/cassandra/cassandra.yaml
Tarball installations installation_location/resources/cassandra/conf/cassandra.yaml

logback.xml

The location of the logback.xml file depends on the type of installation:
Package installations /etc/dse/cassandra/logback.xml
Tarball installations installation_location/resources/cassandra/conf/logback.xml

17 April 2018

Warning: DSE 6.0.0 Do not use TTL (time-to-live) with DSE Search live indexing (RT indexing). To use these features together, upgrade to DSE 6.0.1. (DSP-16038)

6.0.0 Components

  • Apache Solr™ 6.0.1.1.2234
  • Apache Spark™ 2.2.0.14
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.5
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.44 (DSE custom version)
  • TinkerPop 3.3.2 with additional production-certified changes

DSE 6.0 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

6.0 New features

See DataStax Enterprise 6.0 new features.

6.0.0 DSE core

Experimental features. These features are experimental and are not supported for production:
Known issues:

Changes and enhancements:

Important: Support for Thrift-compatible tables (COMPACT STORAGE) is dropped. Before upgrading to DSE 6.0, you must migrate all tables that have COMPACT STORAGE to CQL table format.

Upgrades from DSE 5.0.x or DSE 5.1.x with Thrift-compatible tables require DSE 5.1.6 or later or DSE 5.0.12 or later.

Resolved issues:
  • Warn when the cassandra superuser logs in. (DB-104)
  • Prevent multiple serializations of mutation. (DB-370)
  • Internal implementation of paging by bytes. (DB-414)
  • Connection refused should be logged less frequently. (DB-455)
  • Refactor messaging service code. (DB-497)
  • Change protocol to allow sending keyspace independent of query string. (DB-600)
  • Add result set metadata to prepared statement MD5 hash calculation. (DB-608)
  • Fix LWT asserts for immutable TableMetadat.a (DB-728)
  • MigrationManager should use toDebugString() when logging TableMetadata. (DB-739)
  • Create administrator roles who can carry out everyday administrative tasks without having unnecessary access to data. (DB-757)
  • When repairing Paxos commits, only block on nodes are being repaired. (DB-761)
  • Allow to grant/revoke multiple permissions in one statement (DB-792)
  • SystemKeyspace.snapshotOnVersionChange() never called in production code. (DB-797)
  • Error in counting iterated SSTables when choosing whether to defrag in timestamp ordered path. (DB-1018)
  • Check for mismatched versions when answering schema pulls. (DB-1026)
  • Expose ports (storage, native protocol, JMX) in system local and peers tables. (DB-1040)
  • Rename ITrigger interface method from augment to augmentNonBlocking. (DB-1046)
  • Load mapped buffer into physical memory after mlocking it for MemoryOnlyStrategy. (DB-1052)
  • New STARTUP message parameters identify clients. (DB-1054)
  • Emit client warning when a GRANT/REVOKE/RESTRICT/UNRESTRICT command has no effect. (DB-1083)
  • Update bundled Python driver to 2.2.0.post0-d075d57. (DB-1152)
  • Forbid advancing KeyScanningIterator before exhausting or closing the current iterator. (DB-1199)
  • Ensure that empty clusterings with kind==CLUSTERING are Clustering.EMPTY. (DB-1248)
  • New nodetool abortrebuild command stops a currently running rebuild operation. (DB-1234)
  • Batchlog replays do not leverage remote coordinators. (DB-1337)
  • Avoid copying EMPTY_STATIC_ROW to heap again with offheap memtable. (DB-1375)
  • Allow DiskBoundaryManager to cache different directories. (DB-1454)
  • Abort repair when there is only one node. (DB-1511)
  • OutOfMemory during view update. (DB-1493)
  • Drop response on view lock acquisition timeout and add ViewLockAcquisitionTimeouts metric. (DB-1522)
  • Handle race condition on dropping keyspace and opening keyspace. (DB-1570)
  • dsetool ring prints ERROR when data_file_directories is removed from cassandra.yaml. (DSP-13547)
  • Driver: Jackson-databind is vulnerable to remote code execution (RCE) attacks. (DSP-13498)
  • LDAP library issue. (DSP-15927)

6.0.0 DSE Advanced Replication

Changes and enhancements:
  • Support for DSE Advanced Replication V1 is removed. For V1 installations, you must first upgrade to DSE 5.1.x and migrate your DSE Advanced Replication to V2, and then upgrade to DSE 6.0. (DSP-13376)
  • Enhanced CLI security prevents injection attacks and sanitizes and validates the command line inputs. (DSP-13682)
Resolved issues:
  • Improve logging on unsupported operation failure and remove the failed mutation from replog. (DSP-15043)
  • Channel creation fails with NPE when using mixed case destination name. (DSP-15538)

6.0.0 DSE Analytics

Experimental features. These features are experimental and are not supported for production:
Known issues:
  • DSE Analytics: Additional configuration is required when enabling context-per-jvm in the Spark Jobserver. (DSP-15163)
Changes and enhancements:
  • Previously deprecated environment variables, including SPARK_CLASSPATH, are removed in Spark 2.2.0. (DSP-8379)
  • AlwaysOn SQL service, a HA (highly available) Spark SQL Thrift server. (DSP-10996)
    • JPS is an option required for nodes with AlwaysON SQL enabled.
    • The spark_config_settings and hive_config_settings are removed from dse.yaml. The configuration is provided in the spark-alwayson-sql.conf file in DSEHOME/resources/spark/conf with the same default contents as DSEHOME/resources/spark/conf/spark-defaults.conf. (DSP-15837)
  • Cassandra File System (CFS) is removed. Use DSEFS instead. Before upgrading to DSE 6.0, remove CFS keyspaces. See the From CFS to DSEFS dev blog post. (DSP-12470)
  • Optimization for SearchAnalytics with SELECT COUNT(*) and no predicates. (DSP-12669)
  • Authenticate JDBC users to Spark SQL Thrift Server. Queries that are executed during JDBC session are run as the user who authenticated through JDBC. (DSP-13395)
  • Solr optimization is automatic; spark.sql.dse.solr.enabled is deprecated, use spark.sql.dse.search.enabled instead. (DSP-13398)
  • Optimization for SearchAnalytics with SELECT COUNT(*) and no predicates. (DSP-13398)
  • dse spark-beeline command is removed, use dse beeline instead. (DSP-13468)
  • cfs-stress tool is replaced by fs-stress tool. (DSP-13549)
  • Encryption for data stored on the server and encryption of Spark spill files is supported. (DSP-13841)
  • Improved security with Spark. (DSP-13991)
  • Spark local applications no longer use /var/lib/spark/rdd, instead configure and use .sparkdirectory for processes started by the user. (DSP-14380)
  • Input metrics are not thread-safe and are not used properly in CassandraJoinRDD and CassandraLeftJoinRDD. (DSP-14569)
  • AlwaysOn SQL workpool option adds high availability (HA) for the JDBC or ODBC connections for analytics node. (DSP-14719)
  • CFS is removed. Before upgrade, move HiveMetaStore from CFS to DSEFS and update URL references. (DSP-14831)
  • Include SPARK-21494 to use correct app id when authenticating to external service. (DSP-14140)
  • Upgrade to DSE 6.0 must be complete on all nodes in the cluster before Spark Worker and Spark Master will start. (DSP-14735)
  • Spark Jobserver is the DSE custom version 0.8.0.44. (DSP-14970)
    • For applications to use the compatible Spark Jobserver API in DataStax repository, migrate jobs that extend from SparkHiveJob and SparkSqlJob to SparkSessionJob. See example in the DemoSparkSessionJob in the demos directory.
    • FileDao is deprecated. To store metadata, Spark Jobserver uses H2 SqlDao. Metadata such as job history, uploaded jars in FileDao format is not available after upgrade.
    • Spark Jobserver has a single metadata directory: /tmp/spark-jobserver/
  • Spark Cassandra Connector in DSE 6.0.0, has the following changes:
    • Changes to default values: spark.output.concurrent.writes: 100, spark.task.maxFailures: 10 (DSP-15164)
    • spark.cassandra.connection.connections_per_executor_max is removed; use new properties spark.cassandra.connection.local_connections_per_executor, spark.cassandra.connection.remote_connections_per_executor_min, and spark.cassandra.connection.remote_connections_per_executor_max. (DSP-15193
    • All Spark-related parameters are now camelCase. Parameters are case-sensitive. The snake_versions are automatically translated to the camelCaseVersions except when the parameters are used as table options. In SparkSQL and with spark.read.options(...), the parameters are case-insensitive because of internal SQL implementation.
    • The DSE artifact is com.datastax.dse : spark-connector: 6.0.0.
    • The DseSparkDependencies JAR is still required. (DSP-15694)
  • Use NodeSync (continuous repair) and LOCAL_QUORUM for reading from Spark recovery storage. (DSP-15219)
    Supporting changes:
    • Spark Master will not start until LOCAL_QUORUM is achieved for dse_analytics keyspace.
    • Spark Master recovery data is first attempted to be updated with LOCAL_QUORUM, and if that fails, then attempt with LOCAL_ONE. Recovery data are always queried with LOCAL_QUORUM (unlike previous versions of DSE where we used LOCAL_ONE)
    • DSE Analytics internal data moved from spark_system to dse_analytics keyspace.
    Note: DataStax strongly recommends enabling NodeSync for continuous repair on all tables in the dse_analytics keyspace. NodeSync is required on the rm_shared_data keyspace that stores Spark recovery information.
Resolved issues:
  • DSE does not work with Spark Crypto based encryption. (DSP-14140)

6.0.0 DSEFS

Changes and enhancements:
  • Wildcard characters are supported in DSEFS shell commands. (DSP-10583)
  • DSEFS should support all DSE authentication schemes. (DSP-12956)
  • Improved authorization security sets the default permission to 755 for directories and 644 for files. New DSEFS clusters create the root directory / with 755 permission to prevent non-super users from modifying root content; for example, by using mkdir or put commands. (DSP-13609)
  • Enable SSL for DSEFS encryption. (DSP-13771)
  • HTTP communication logging level changed from DEBUG to TRACE. (DSP-14400)
  • DSEFS shell history has been moved to ~/.dse/dsefs_history. (DSP-15070)
  • New tool to move hive metastore from CFS to DSEFS and update references.
  • Add echo command to DSEFS shell. (DSP-15446)
  • Changes in dse.yaml for advanced DSEFS settings.
  • Alternatives wildcards are Hadoop compatible. (DSP-15249)

6.0.0 DSE Graph

Known issues:
  • Dropping a property of vertex label with materialized view (MV) indices breaks graph. To drop a property key for a vertex label that has a materialized view index, additional steps are required to prevent data loss or cluster errors. See Dropping graph schema. (DSP-15532)
  • Secondary indexes used for DSE Graph queries have higher latency in DSE 6.0 than in the previous version. (DB-1928)
Changes and enhancements:
  • Improved and simplified data batch loading of pre-formatted data. (DGL-235)

    Supporting changes:

    • Schema discovery and schema generation are deprecated. (DGL-246)
    • Standard vertex IDs are deprecated. Use custom vertex IDs instead. (DSP-13485)
    • Standard IDs are deprecated. (DGL-247)
    • Transformations are deprecated. (DGL-248)
  • Schema API changes: all .remove() methods are renamed to .drop() and schema.clear() is renamed to schema.drop(). Schema API supports removing vertex/edge labels and property keys. Unify use of drop | remove | clear in the Schema API and use .drop() everywhere. (DSP-8385, DSP-14150)
  • Include materialized view (MV) indexes in query optimizer only if the MV was fully built. (DSP-10219)
  • DSE profiling of graph statements from the gremlin shell. (DSP-13484)
  • Improve Graph OLAP performance by smart routing query to DseGraphFrame engine with DseGraphFrameInterceptorStrategy. (DSP-13489)
  • OSS TinkerPop 3.3 supports Spark 2.2. (DSP-13632)
  • Partitioned vertex tables (PVT) are removed. (DSP-13676)
  • Graph online analytical processing (OLAP) supports drop() with DseGraphFrame interceptor. Simple queries can be used in drop operations. (DSP-13998)
  • DSE Graphs vertices and edges tables are accessible from SparkSQL and automated to dse_graph SparkSQL database. (DSP-12046)
  • More Gremlin APIs are supported in DSEGraphFrames: dedup, sort, limit, filter, as()/select(), or(). (DSP-13649)
  • Some graph and gremlin_server properties in earlier versions of DSE are no longer required for DSE 6.0. These settings were removed from dse.yaml. The default settings from the earlier versions of dse.yaml are preserved:
    • adjacency_cache_clean_rate
    • adjacency_cache_max_entry_size_in_mb
    • adjacency_cache_size_in_mb
    • gremlin_server_enabled
    • index_cache_clean_rate
    • index_cache_max_entry_size_in_mb
    • schema_mode - default schema_mode is production
    • window_size
    • ids (all vertex ID assignment and partitioning strategy options)
    • various gremlin_server settings
    If these properties exist in the dse.yaml file after upgrading to DSE 6.0, logs display warnings. You can ignore these warnings or modify dse.yaml so that only the required graph system level and gremlin_server properties are present. (DSP-14308)
  • Spark Jobserver is the DSE custom version 0.8.0.44. Applications must use the compatible Spark Jobserver API in DataStax repository. (DSP-14152)
  • Edge label names and property key names allow only [a-zA-Z0-9], underscore, hyphen, and period. The string formatting for vertices with text custom IDs has changed. (DSP-14710)
    Supporting changes (DSP-15167):
    • schema.describe() displays the entire schema, even if it contains illegal names.
    • In-place upgrades allow existing schemas with invalid edge label names and property key names.
    • Schema elements with illegal names cannot be uploaded or added.
  • Invoking toString on a custom vertex ID containing a text property, or on an edge ID that is incident upon a vertex with a custom vertex ID, now returns a value that encloses the text property value in double quotation marks and escapes the value's internal double-quotes. This change protects older formats from irresolvable parsing ambiguity. For example:
    // old
                {~label=v, x=foo}
                {~label=w, x=a"b}
                // new
                {~label=v, x="foo"}
                {~label=w, x="a""b"}
  • Support for math()-step (math) to enable scientific calculator functionality within Gremlin. (DSP-14786)
  • The GraphQueryThreads JMX attribute has been removed. Thread selection occurs with Thread Per Core (TPC) asynchronous request processing architecture. (DSP-15222)
Resolved issues:
  • spark.sql.hive.metastore.barrierPrefixes is set to org.apache.spark.sql.cassandra to properly use CassandraConnector in DSE HiveMetastore implementation. (DSP-14120)
  • Intermittent KryoException: Buffer underflow error when running order by query in OLTP mode. (DSP-12694)
  • DseGraphFrame does not support infix and() and or(). (DSP-12013)
  • Graph could be left in an inconsistent state if a schema migration fails. (DSP-15532)
  • DseGraphFrames properties().count() step return vertex count instead of multi properties count. (DSP-15049)
  • GraphSON parsing error prevents proper type detection under certain conditions. (DSP-14066)

6.0.0 DSE Search

Experimental features. These features are experimental and are not supported for production:
  • The dsetool index_checks use an Apache Lucene® experimental feature.
Known issues:
  • Search index TTL Expiration thread loops without effect with live indexing (RT indexing). (DSP-16038)
Changes and enhancements:
  • Writes are flushed to disk in segments that use a new Lucene codec that does not exist in earlier versions. Unique key values are no longer stored as both docValues and Lucene stored fields. The unique key values are now stored only as docValues in a new codec to store managed fields like Lucene. Downgrades to versions earlier than DSE 6.0 are not supported. (DSP-8465)
  • Document inserts and updates using HTTP are removed. Before upgrading, ensure you are using CQL for all inserts and updates. (DSP-9725).
  • DSENRTCachingDirectoryFactory is removed. Before upgrading, change your search index config. (DSP-10126)
  • The <dataDir> parameter in the solrconfig.xml file is not supported. Instead, follow the steps in Set the location of search indexes. (DSP-13199)
  • Improved performance by early termination of sorting. Ideal for queries that need only a few results returned, from a large number of total matches. (DSP-13253)
  • Native CQL syntax for search queries. (DSP-13411)
  • Delete by id is removed. Delete by query no longer accepts wildcard queries, including queries that match all documents (for example, <delete><query>*:*</query></delete>). Instead, use CQL to DELETE by Primary Key or the TRUNCATE command. (DSP-13436)
  • Search index config changes. (DSP-14137)
    • mergePolicy, maxMergeDocs, and mergeFactor are no longer supported.
    • RAM buffer size settings are no longer required in search index config. Global RAM buffer usage in Lucene is governed by the memtable size limits in cassandra.yaml. RAM buffers are counted toward the memtable_heap_space_in_mb.
  • dsetool core_indexing_status --progress option is always true. (DSP-13465)
  • The HTTP API for Solr core management is removed. Instead, use CQL commands for search index management or dsetool search index commands. (DSP-15350)
  • The Tika functionality bundled with Apache Solr is removed. Instead, use the stand-alone Apache Tika project. (DSP-13892)
  • Logging configuration improvements. (DSP-14137)
    • The solrvalidation.log is removed. You can safely remove appender SolrValidationErrorAppender and the logger SolrValidationErrorLogger from logback.xml. Indexing errors manifest as:
      • failures at the coordinator if they represent failures that might succeed at some later point in time using the hint replay mechanism
      • as messages in the system.log if the failures are due to non-recoverable indexing validation errors (for data that is written to the database, but not indexed properly)
  • The DSE custom update request processor (URP) implementation is deprecated. Use the field input/output (FIT) transformer API instead. (DSP-14360)
  • The stored flag in search index schemas is deprecated and is no longer added to auto-generated schemas. If the flag exists in custom schemas, it is ignored. (DSP-14425)
  • Tuning improvements for indexing. (DSP-14785)
    • Indexing is no longer asynchronous. Document updates are written to the Lucene RAM buffer synchronously with the mutation backing table.
    • See Configuring and tuning indexing performance.
    • The back_pressure_threshold_per_core in dse.yaml affects only index rebuilding/reindexing.
    • These options in dse.yaml are removed:
      • enable_back_pressure_adaptive_nrt_commit
      • max_solr_concurrency_per_core
      • solr_indexing_error_log_options
      DSE 6.0 will not start with these options present.
  • StallMetrics MBean is removed. Before upgrading to DSE 6.0, change operators that use the MBean. (DSP-14860)
  • Optimize Paging when limit is smaller than the page size. (DSP-15207)
Resolved issues includes all bug fixes up to DSE 5.1.8. Additional 6.0.0 fixes:
  • Isolate Solr Resource Loading at startup to the Local DC. (DSP-10911)

DataStax Studio 6.0.0

  • For use with DSE 6.0.x, DataStax Studio 6.0.0 is installed as a standalone tool. (DSP-13999, DSP-15623)

For details, see DataStax Studio 6.0.0 release notes.

DataStax Bulk Loader 1.0.1

  • DataStax Bulk Loader (dsbulk) version 1.0.1 is automatically installed with DataStax Enterprise 6.0.0, and can also be installed as a standalone tool. (DSP-13999, DSP-15623)
For details, see DataStax Bulk Loader 1.0.1 release notes.

Cassandra enhancements for DSE 6.0

DataStax Enterprise 6.0.0 is compatible with Apache Cassandra™ 3.11 and adds production-certified enhancements.

DataStax Enterprise 6.0.0 is compatible with Apache Cassandra™ 3.11 and adds these production-certified enhancements:

  • Add DEFAULT, UNSET, MBEAN and MBEANS to `ReservedKeywords`. (CASSANDRA-14205)
  • Add Unittest for schema migration fix (CASSANDRA-14140)
  • Print correct snitch info from nodetool describecluster (CASSANDRA-13528)
  • Close socket on error during connect on OutboundTcpConnection (CASSANDRA-9630)
  • Enable CDC unittest (CASSANDRA-14141)
  • Split CommitLogStressTest to avoid timeout (CASSANDRA-14143)
  • Improve commit log chain marker updating (CASSANDRA-14108)
  • Fix updating base table rows with TTL not removing view entries (CASSANDRA-14071)
  • Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
  • More frequent commitlog chained markers (CASSANDRA-13987)
  • RPM package spec: fix permissions for installed jars and config files (CASSANDRA-14181)
  • More PEP8 compiance for cqlsh (CASSANDRA-14021)
  • Fix support for SuperColumn tables (CASSANDRA-12373)
  • Fix missing original update in TriggerExecutor (CASSANDRA-13894)
  • Improve short read protection performance (CASSANDRA-13794)
  • Fix counter application order in short read protection (CASSANDRA-12872)
  • Fix MV timestamp issues (CASSANDRA-11500)
  • Fix AssertionError in short read protection (CASSANDRA-13747)
  • Gossip thread slows down when using batch commit log (CASSANDRA-12966)
  • Allow native function calls in CQLSSTableWriter (CASSANDRA-12606)
  • Copy session properties on cqlsh.py do_login (CASSANDRA-13847)
  • Fix load over calculated issue in IndexSummaryRedistribution (CASSANDRA-13738)
  • Obfuscate password in stress-graphs (CASSANDRA-12233)
  • ReverseIndexedReader may drop rows during 2.1 to 3.0 upgrade (CASSANDRA-13525)
  • Avoid reading static row twice from old format sstables (CASSANDRA-13236)
  • Fix possible NPE on upgrade to 3.0/3.X in case of IO errors (CASSANDRA-13389)
  • Add duration data type (CASSANDRA-11873)
  • Properly report LWT contention (CASSANDRA-12626)
  • Stress daemon help is incorrect(CASSANDRA-12563)
  • Remove ALTER TYPE support (CASSANDRA-12443)
  • Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203)
  • Remove support for non-JavaScript UDFs (CASSANDRA-12883)
  • Better handle invalid system roles table (CASSANDRA-12700)
  • Upgrade netty version to fix memory leak with client encryption (CASSANDRA-13114)
  • Fix trivial log format error (CASSANDRA-14015)
  • Allow SSTabledump to do a JSON object per partition (CASSANDRA-13848)
  • Remove unused and deprecated methods from AbstractCompactionStrategy (CASSANDRA-14081)
  • Fix Distribution.average in cassandra-stress (CASSANDRA-14090)
  • Presize collections (CASSANDRA-13760)
  • Add GroupCommitLogService (CASSANDRA-13530)
  • Parallelize initial materialized view build (CASSANDRA-12245)
  • Fix flaky SecondaryIndexManagerTest.assert[Not]MarkedAsBuilt (CASSANDRA-13965)
  • Make LWTs send resultset metadata on every request (CASSANDRA-13992)
  • Fix flaky indexWithFailedInitializationIsNotQueryableAfterPartialRebuild (CASSANDRA-13963)
  • Introduce leaf-only iterator (CASSANDRA-9988)
  • Allow only one concurrent call to StatusLogger (CASSANDRA-12182)
  • Refactoring to specialised functional interfaces (CASSANDRA-13982)
  • Speculative retry should allow more friendly parameters (CASSANDRA-13876)
  • Throw exception if we send/receive repair messages to incompatible nodes (CASSANDRA-13944)
  • Replace usages of MessageDigest with Guava's Hasher (CASSANDRA-13291)
  • Add nodetool command to print hinted handoff window (CASSANDRA-13728)
  • Fix some alerts raised by static analysis (CASSANDRA-13799)
  • Checksum SSTable metadata (CASSANDRA-13321, CASSANDRA-13593)
  • Add result set metadata to prepared statement MD5 hash calculation (CASSANDRA-10786)
  • Add incremental repair support for --hosts, --force, and subrange repair (CASSANDRA-13818)
  • Refactor GcCompactionTest to avoid boxing (CASSANDRA-13941)
  • Expose recent histograms in JmxHistograms (CASSANDRA-13642)
  • Add SERIAL and LOCAL_SERIAL support for cassandra-stress (CASSANDRA-13925)
  • LCS needlessly checks for L0 STCS candidates multiple times (CASSANDRA-12961)
  • Correctly close netty channels when a stream session ends (CASSANDRA-13905)
  • Update lz4 to 1.4.0 (CASSANDRA-13741)
  • Throttle base partitions during MV repair streaming to prevent OOM (CASSANDRA-13299)
  • Improve short read protection performance (CASSANDRA-13794)
  • Fix AssertionError in short read protection (CASSANDRA-13747)
  • Use compaction threshold for STCS in L0 (CASSANDRA-13861)
  • Fix problem with min_compress_ratio: 1 and disallow ratio < 1 (CASSANDRA-13703)
  • Add extra information to SASI timeout exception (CASSANDRA-13677)
  • Rework CompactionStrategyManager.getScanners synchronization (CASSANDRA-13786)
  • Add additional unit tests for batch behavior, TTLs, Timestamps (CASSANDRA-13846)
  • Add keyspace and table name in schema validation exception (CASSANDRA-13845)
  • Emit metrics whenever we hit tombstone failures and warn thresholds (CASSANDRA-13771)
  • Allow changing log levels via nodetool for related classes (CASSANDRA-12696)
  • Add stress profile yaml with LWT (CASSANDRA-7960)
  • Reduce memory copies and object creations when acting on ByteBufs (CASSANDRA-13789)
  • simplify mx4j configuration (Cassandra-13578)
  • Fix trigger example on 4.0 (CASSANDRA-13796)
  • force minumum timeout value (CASSANDRA-9375)
  • Add bytes repaired/unrepaired to nodetool tablestats (CASSANDRA-13774)
  • Don't delete incremental repair sessions if they still have sstables (CASSANDRA-13758)
  • Fix pending repair manager index out of bounds check (CASSANDRA-13769)
  • Don't use RangeFetchMapCalculator when RF=1 (CASSANDRA-13576)
  • Don't optimise trivial ranges in RangeFetchMapCalculator (CASSANDRA-13664)
  • Use an ExecutorService for repair commands instead of new Thread(..).start() (CASSANDRA-13594)
  • Fix race / ref leak in anticompaction (CASSANDRA-13688)
  • Fix race / ref leak in PendingRepairManager (CASSANDRA-13751)
  • Enable ppc64le runtime as unsupported architecture (CASSANDRA-13615)
  • Improve sstablemetadata output (CASSANDRA-11483)
  • Support for migrating legacy users to roles has been dropped (CASSANDRA-13371)
  • Introduce error metrics for repair (CASSANDRA-13387)
  • Refactoring to primitive functional interfaces in AuthCache (CASSANDRA-13732)
  • Update metrics to 3.1.5 (CASSANDRA-13648)
  • batch_size_warn_threshold_in_kb can now be set at runtime (CASSANDRA-13699)
  • Avoid always rebuilding secondary indexes at startup (CASSANDRA-13725)
  • Upgrade JMH from 1.13 to 1.19 (CASSANDRA-13727)
  • Upgrade SLF4J from 1.7.7 to 1.7.25 (CASSANDRA-12996)
  • Default for start_native_transport now true if not set in config (CASSANDRA-13656)
  • Don't add localhost to the graph when calculating where to stream from (CASSANDRA-13583)
  • Allow skipping equality-restricted clustering columns in ORDER BY clause (CASSANDRA-10271)
  • Use common nowInSec for validation compactions (CASSANDRA-13671)
  • Improve handling of IR prepare failures (CASSANDRA-13672)
  • Send IR coordinator messages synchronously (CASSANDRA-13673)
  • Flush system.repair table before IR finalize promise (CASSANDRA-13660)
  • Fix column filter creation for wildcard queries (CASSANDRA-13650)
  • Add 'nodetool getbatchlogreplaythrottle' and 'nodetool setbatchlogreplaythrottle' (CASSANDRA-13614)
  • fix race condition in PendingRepairManager (CASSANDRA-13659)
  • Allow noop incremental repair state transitions (CASSANDRA-13658)
  • Run repair with down replicas (CASSANDRA-10446)
  • Added started & completed repair metrics (CASSANDRA-13598)
  • Added started & completed repair metrics (CASSANDRA-13598)
  • Improve secondary index (re)build failure and concurrency handling (CASSANDRA-10130)
  • Improve calculation of available disk space for compaction (CASSANDRA-13068)
  • Change the accessibility of RowCacheSerializer for third party row cache plugins (CASSANDRA-13579)
  • Allow sub-range repairs for a preview of repaired data (CASSANDRA-13570)
  • NPE in IR cleanup when columnfamily has no sstables (CASSANDRA-13585)
  • Fix Randomness of stress values (CASSANDRA-12744)
  • Allow selecting Map values and Set elements (CASSANDRA-7396)
  • Fast and garbage-free Streaming Histogram (CASSANDRA-13444)
  • Update repairTime for keyspaces on completion (CASSANDRA-13539)
  • Add configurable upper bound for validation executor threads (CASSANDRA-13521)
  • Bring back maxHintTTL propery (CASSANDRA-12982)
  • Add testing guidelines (CASSANDRA-13497)
  • Add more repair metrics (CASSANDRA-13531)
  • RangeStreamer should be smarter when picking endpoints for streaming (CASSANDRA-4650)
  • Avoid rewrapping an exception thrown for cache load functions (CASSANDRA-13367)
  • Log time elapsed for each incremental repair phase (CASSANDRA-13498)
  • Add multiple table operation support to cassandra-stress (CASSANDRA-8780)
  • Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262)
  • Fix WriteResponseHandlerTest is sensitive to test execution order (CASSANDRA-13421)
  • Improve incremental repair logging (CASSANDRA-13468)
  • Start compaction when incremental repair finishes (CASSANDRA-13454)
  • Add repair streaming preview (CASSANDRA-13257)
  • Cleanup isIncremental/repairedAt usage (CASSANDRA-13430)
  • Change protocol to allow sending key space independent of query string (CASSANDRA-10145)
  • Make gc_log and gc_warn settable at runtime (CASSANDRA-12661)
  • Take number of files in L0 in account when estimating remaining compaction tasks (CASSANDRA-13354)
  • Skip building views during base table streams on range movements (CASSANDRA-13065)
  • Improve error messages for +/- operations on maps and tuples (CASSANDRA-13197)
  • Remove deprecated repair JMX APIs (CASSANDRA-11530)
  • Fix version check to enable streaming keep-alive (CASSANDRA-12929)
  • Make it possible to monitor an ideal consistency level separate from actual consistency level (CASSANDRA-13289)
  • Outbound TCP connections ignore internode authenticator (CASSANDRA-13324)
  • Upgrade junit from 4.6 to 4.12 (CASSANDRA-13360)
  • Cleanup ParentRepairSession after repairs (CASSANDRA-13359)
  • Upgrade snappy-java to 1.1.2.6 (CASSANDRA-13336)
  • Incremental repair not streaming correct sstables (CASSANDRA-13328)
  • Upgrade the JNA version to 4.3.0 (CASSANDRA-13300)
  • Add the currentTimestamp, currentDate, currentTime and currentTimeUUID functions (CASSANDRA-13132)
  • Remove config option index_interval (CASSANDRA-10671)
  • Reduce lock contention for collection types and serializers (CASSANDRA-13271)
  • Make it possible to override MessagingService.Verb ids (CASSANDRA-13283)
  • Avoid synchronized on prepareForRepair in ActiveRepairService (CASSANDRA-9292)
  • Adds the ability to use uncompressed chunks in compressed files (CASSANDRA-10520)
  • Don't flush sstables when streaming for incremental repair (CASSANDRA-13226)
  • Remove unused method (CASSANDRA-13227)
  • Fix minor bugs related to #9143 (CASSANDRA-13217)
  • Output warning if user increases RF (CASSANDRA-13079)
  • Remove pre-3.0 streaming compatibility code for 4.0 (CASSANDRA-13081)
  • Add support for + and - operations on dates (CASSANDRA-11936)
  • Fix consistency of incrementally repaired data (CASSANDRA-9143)
  • Increase commitlog version (CASSANDRA-13161)
  • Make TableMetadata immutable, optimize Schema (CASSANDRA-9425)
  • Refactor ColumnCondition (CASSANDRA-12981)
  • Parallelize streaming of different keyspaces (CASSANDRA-4663)
  • Improved compactions metrics (CASSANDRA-13015)
  • Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031)
  • Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855)
  • Thrift removal (CASSANDRA-11115)
  • Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716)
  • Add column definition kind to dropped columns in schema (CASSANDRA-12705)
  • Add (automate) Nodetool Documentation (CASSANDRA-12672)
  • Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736)
  • Reject invalid replication settings when creating or altering a keyspace (CASSANDRA-12681)
  • Clean up the SSTableReader#getScanner API wrt removal of RateLimiter (CASSANDRA-12422)
  • Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
  • Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
  • Require forceful decommission if number of nodes is less than replication factor (CASSANDRA-12510)
  • Allow IN restrictions on column families with collections (CASSANDRA-12654)
  • Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028)
  • Add timeUnit Days for cassandra-stress (CASSANDRA-13029)
  • Add mutation size and batch metrics (CASSANDRA-12649)
  • Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999)
  • Expose time spent waiting in thread pool queue (CASSANDRA-8398)
  • Conditionally update index built status to avoid unnecessary flushes (CASSANDRA-12969)
  • cqlsh auto completion: refactor definition of compaction strategy options (CASSANDRA-12946)
  • Add support for arithmetic operators (CASSANDRA-11935)
  • Add histogram for delay to deliver hints (CASSANDRA-13234)
  • Fix cqlsh automatic protocol downgrade regression (CASSANDRA-13307)
  • Changing `max_hint_window_in_ms` at runtime (CASSANDRA-11720)
  • Nodetool repair can hang forever if we lose the notification for the repair completing/failing (CASSANDRA-13480)
  • Anticompaction can cause noisy log messages (CASSANDRA-13684)
  • Switch to client init for sstabledump (CASSANDRA-13683)
  • CQLSH: Don't pause when capturing data (CASSANDRA-13743)

General upgrade advice for DSE 6.0.0

General upgrade advice for DataStax Enterprise 6.0.0

General upgrade advice for DataStax Enterprise (DSE) 6.0.0:
GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

DSE 6.0
=======

Operations
----------
   - Added batchlog_endpoint_strategy option and 2 new strategies to cassandra.yaml. The default is
     "random_remote", which works as in previous versions. Two new strategies help to choose better
     endpoints and improve batchlog write latency. See cassandra.yaml for details. (DB-1367)
   - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
     The command must be executed on the node where the rebuild operation is running. Streams
     may continue until they finish or timeout.
   - Added "-Dcassandra.replace_consistency" to support streaming each range from more than one source.
     By default, node replacement only streams one copy of data which may be stale wrt to QUORUM/LOCAL_QUORUM.
     After replacing a down node with "-Dcassandra.replace_consistency=LOCAL_DC_QUORUM", replacement node will be up
     with consistent data assuming data is inserted with LOCAL_QUORUM. Note that streaming multiple copies causes more
     IO and compactions. Alternatively, operators can run repair before bootstrapping the replacement with the
     default replace_consistency=ONE.
     Available options: "ONE"(default), "LOCAL_DC_QUORUM", "GLOBAL_QUORUM".
   - Rebuild functionality now requires to specify the streaming source endpoints and/or DC/racks
     and no longer implicitly prefers the local data center. Specifying no sources or DC/racks is
     now considered an error.
     See `nodetool help rebuild` about which combinations of DCs/racks or endpoints to include or
     exclude are possible. (DB-581)
   - Nodetool repair will now run full repairs by default, use "-inc" option to run incremental repair.
   - Incremental repairs are disallowed on tables with Materialized Views or CDC. Run full repairs on these
     tables.
   - A new HINTS stage was created to handle incoming hints. The maximum size of this thread pool is governed by
     the system property cassandra.hints.max_receive_threads, which has default value max_hints_delivery_threads
     from cassandra.yaml.
   - A new BACKGROUND_IO stage was created to handle blocking background IO operations like cache loading,
     commit log segment allocation and replay. The maximum size of this thread pool is governed by the system
     property cassandra.io.background.max_pool_size.
   - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
     set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.


Upgrading
---------
   - The ITrigger interface has been modified from "augment" to "augmentNonBlocking" for non-blocking
     internal architecture (DB-1046).
   - Removed support for directly setting JMX remote port via the com.sun.management.jmxremote.port
     system property, deprecated since 3.6. See CASSANDRA-11725 and DB-1040 for more details.
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 6.0.
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. See CASSANDRA-12888 for details.
   - Changes to permissions (CassandraAuthorizer, DseAuthorizer), roles (CassandraRoleManager,
     DseRoleManager) and credentials (PasswordAuthenticator) are broadcasted throughout the cluster.
     This eliminates the need to have short-ish validity (permissions_validity_in_ms,
     roles_validity_in_ms, credentials_validity_in_ms) and update-interval
     (permissions_update_interval_in_ms, roles_update_interval_in_ms,
     credentials_update_interval_in_ms) settings in cassandra.yaml. Consider raising the values
     for *_validity_in_ms and *_update_interval_in_ms, if configured.
     Please note, that only "reachable" nodes will receive the invalidation messages and
     unreachable nodes may still hold outdated information after DCL command.
   - REVOKE (and the new UNRESTRICT statement) now error out when the statement would not change
     anything. This means, an error is raised when the statement targets permissions which are
     not present for that resource.
   - The caches for JMX permissions and credentials no longer exist and therefore the JMX beans
     for these caches have been removed, too. Also the configuration options for the credentials
     cache need to be removed from the cassandra.yaml file.

Deprecation
-----------
    - The following properties were renamed and are now deprecated before they are removed in the next major release:
      - rpc_address -> native_transport_address
      - rpc_interface -> native_transport_interface
      - rpc_interface_prefer_ipv6 -> native_transport_interface_prefer_ipv6
      - broadcast_rpc_address -> native_transport_broadcast_address
      - rpc_keepalive -> native_transport_keepalive
    - Removed org.apache.cassandra.metrics.Table.{AntiCompactedBytes|NonAntiCompactedBytes} metrics.
    - Removed following properties from cassandra.yaml: concurrent_reads, concurrent_writes,
      concurrent_counter_writes, concurrent_materialized_view_writes, streaming_socket_timeout_in_ms


4.0
===

New features
------------
   - SSTableDump now supports the -l option to output each partition as its own json object
     See CASSANDRA-13848 for more detail
   - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
     See CASSANDRA-13132
   - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
     See CASSANDRA-11936
   - Support for arithmetic operations on number has been added. See CASSANDRA-11935
   - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
     consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
   - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
   - The initial build of materialized views can be parallelized. The number of concurrent builder
     threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
     This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.

TinkerPop Changes

A list of DataStax Enterprise 6.0.0 production-certified changes in addition to TinkerPop 3.2.8.

DataStax Enterprise (DSE) 6.0.0 includes all changes from previous releases plus these production-certified changes that are in addition to TinkerPop 3.2.8:
  • Made iterate() a first class step. (TINKERPOP-1834)
  • Fixed a bug in NumberHelper that led to wrong min/max results if numbers exceeded the Integer limits. (TINKERPOP-1873)
  • Improved error messaging for failed serialization and deserialization of request/response messages.
  • Fixed bug in handling of Direction.BOTH in Messenger implementations to pass the message to the opposite side of the `StarGraph` in VertexPrograms for OLAP traversals. (TINKERPOP-1862)
  • Fixed a bug in Gremlin Console which prevented handling of gremlin.sh flags that had an equal sign (=) between the flag and its arguments. (TINKERPOP-1879)
  • Fixed a bug where SparkMessenger was not applying the edgeFunction`from MessageScope`in VertexPrograms for OLAP-based traversals. (TINKERPOP-1872)
  • TinkerPop drivers prior to 3.2.4 won't authenticate with Kerberos anymore. A long-deprecated option on the Gremlin Server protocol was removed.