DataStax Enterprise 6.0 release notes

DataStax Enterprise release notes include cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues for DataStax Enterprise 6.0.x.

DataStax Enterprise release notes cover cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues for DataStax Enterprise (DSE) 6.0.x.
Note: Each point release contains a highlights and executive summary section to provide guidance and add visibility to important improvements.
Warning:

DSE 6.0.3 DataStax recommends installing the latest patch release. Due to DB-2477, DataStax does not recommend using DSE 6.0.3 for production.

Requirement for uniform clusters

All nodes in each cluster must be uniformly licensed to use the same subscription. For example, if a cluster contains 5 nodes, all 5 nodes within that cluster must be either DataStax Basic, or all 5 nodes must be DataStax Enterprise. Mixing different subscriptions within a cluster is not permitted. “Cluster” means a collection of nodes running the software which communicate with one another via Gossip, and “Gossip” means the mechanism within the software enabling related nodes to communicate with one another. For more information, see Enterprise Terms.

Before you upgrade

The latest version of DataStax Enterprise is 6.0.4.

Be sure to read all of the relevant upgrade documentation, including Planning your DataStax Enterprise upgrade. Upgrades are supported from DSE 5.1 to 6.0 and from DSE 5.0 to 6.0.

  • Product compatibility
    • DataStax Enterprise 6.0 is compatible with:
      • DSE OpsCenter 6.5
      • DataStax Studio 6.0
    • DataStax Drivers

      Depending on the driver version, you might need to recompile your client application code. See Upgrading DataStax drivers.

    • DataStax Bulk Loader compatibility
      • Can load data into DSE 5.0 or later
      • Can unload data from any Apache CassandraTM 2.1 or later data source

DSE 6.0.4 release notes

8 October 2018

DataStax recommends the latest patch release for most environments.

DSE Database important bug fix

  • Fix wrong offset in size calculation in trie builder. (DB-2477)

6.0.4 Components

All components from DSE 6.0.0 are listed. Components that are updated for DSE 6.0.4 are indicated.
  • Apache Solr™ 6.0.1.1.2338
  • Apache Spark™ 2.2.2.5
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.9
  • Key Management Interoperability Protocol (KMIP) 1.7.1e
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.45 DSE custom version
  • TinkerPop 3.3.3 with additional production-certified changes

DSE 6.0.4 is compatible with Apache Cassandra™ 3.11 and adds no additional production-certified changes.

DSE 6.0.3 release notes

20 September 2018

Warning: DataStax recommends installing the latest patch release. Due to DB-2477, DataStax does not recommend using DSE 6.0.3 for production.

6.0.3 Components

All components from DSE 6.0.0 are listed. Components that are updated for DSE 6.0.3 are indicated.
  • Apache Solr™ 6.0.1.1.2338 (updated)
  • Apache Spark™ 2.2.2.5 (updated)
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.9
  • Key Management Interoperability Protocol (KMIP) 1.7.1e
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.45 DSE custom version
  • TinkerPop 3.3.3 with additional production-certified changes (updated)

DSE 6.0.3 is compatible with Apache Cassandra™ 3.11 and adds no additional production-certified changes.

DSE 6.0.3 Highlights

High-value benefits of upgrading to DSE 6.0.3 include these highlights:

DSE Database (DSE core) highlights

Improvements:
  • Deleting a static column and adding it back as a non-static column introduces corruption. (DB-1630)
  • NodeSync command line tool only connects over JMX to a single node. (DB-1693)
  • Create a log message when DDL statements are executed. (DB-2383)
Known issue:
  • Wrong offset in size calculation in trie builder. (DB-2477)
Important bug fixes:
  • Authentication cache loading can exhaust native threads. (DB-2248)
  • The nodesync tasks fail with assertion error. (DB-2323)
  • Unexpected behavior change when using row-level permissions with modification conditions like IF EXISTS. (DB-2429)
  • Non-internal users are unable to use permissions granted on CREATE. (DSP-16824)

DSE Analytics highlights

Improvements:
  • Improved security isolates Spark applications. (DSP-16093)
  • Upgrade to Spark 2.2.2. (DSP-16761)
  • Jetty 9.4.1 upgrade addresses security vulnerabilities in Spark dependencies packaged with DSE. (DSP-16893)
  • dse spark-submit kill and status commands support optionally explicit Spark Master IP address. (DSP-16910, DSP-16991)
Important bug fixes:
  • Fixed problems with temporary and data directories for Spark applications. (DSP-15476, DSP-15880)
  • Spark Cassandra Connector method saveToCassandra should not require solr_query column when search is enabled. (DSP-16427)
  • Cassandra streaming sink doesn't work with some sources. (DSP-16635)
  • Metastore can't handle table with 100+ columns. (DSP-16742)
  • Fully qualified paths with resource URL are correctly resolved in Spark structured streaming checkpointing. Backport SPARK-20894. (DSP-16972)

DSEFS highlights

Important bug fixes:
  • Only superusers are allowed to remove corrupted non-empty directories when authentication is enabled for DSEFS. Improved error message when performing an operation on a corrupted path. (DSP-16340)
  • cassandra nonsuperuser gets dsefs AccessDeniedException due to Insufficient permissions. (DSP-16713)
  • DSEFS Hadoop layer doesn't properly translate DSEFS exceptions to Hadoop exceptions in some methods. (DSP-16933)
  • Closing DSEFS client before all issued requests are completed causes unexpected message type: DefaultLastHttpContent error. (DSP-16953)
  • Under high loads, DSEFS reports temporary incorrect state for various files/directories. (DSP-17178)

DSE Graph highlights

  • Aligned query behavior using geo.inside() predicate for polygon search with and without search indexes. (DSP-16108)
  • Added convenience methods for reading graph configuration: getEffectiveAllowScan and getEffectiveSchemaMode. (DSP-16650)
  • Fixed bug where deleting a search index that was defined inside a graph fails. (DSP-16765)
  • Changed default write consistency level (CL) for Graph to LOCAL_QUORUM. (DSP-17140)
    Attention: In earlier DSE versions, the default QUORUM write consistency level (CL) was not appropriate for multi-datacenter production environments.

DSE Search highlights

Improvements:
  • Reduce the number of token filters for distributed searches with vnodes. (DSP-14189)
  • Avoid unnecessary exception and error creation in the Solr query parser. (DSP-17147)
Important bug fixes:
  • Avoid accumulating redundant router state updates during schema disagreement. (DSP-15615)
  • A search enabled node could return different exceptions than a non-search enabled node when a keyspace or table did not exist. (DSP-16834)
  • DSE does not start without appropriate Tomcat JAR scanning exclusions. (DSP-16841)
  • CQL single-pass queries have incorrect results when query is run with primary key and search index schema does not contain all columns in selection. (DSP-16895)
  • Node health score of 1 is not obtainable. Search node gets stuck at 0.00 node health score after replacing a node in a cluster. (DSP-17107)

6.0.3 DSE core

Changes and enhancements:
  • Create a log message when DDL statements are executed. (DB-2383)
  • Due to Thread Per Core (TPC) asynchronous request processing architecture, the index_summary_capacity_in_mb and index_summary_resize_interval_in_minutes settings in cassandra.yaml are removed. (DB-2390)
  • Connections on non-serialization errors are not dropped. (DB-2233)
  • NetworkTopologyStrategy warning about unrecognized option at startup. (DB-2235)
  • NodeSync waits to start until all nodes in the cluster are upgraded. (DB-2385)
  • Improved error handling and logging for TDE encryption key management. (DP-15314)
  • DataStax does more extensive testing on OpenJDK 8 due to the end of public updates for Oracle JRE/JDK 8. (DSP-16179)
  • Non-internal users are unable to use permissions granted on CREATE. (DSP-16824)
Resolved issues:
  • NodeSync command line tool only connects over JMX to a single node. (DB-1693)
  • TotalBlockedTasksGauge metric value is computed incorrectly. (DB-2002)
  • Move TWCS message "No compaction necessary for bucket size" to Trace level or NoSpam. (DB-2022)
  • Non-portable syntax (MX4J bash-isms) in cassandra-env.sh broke service scripts. (DB-2123)
  • sstableloader options assume the RPC/native (client) interface is the same as the internode (node-to-node) interface. (DB-2184)
  • The nodesync tasks fail with assertion error. (DB-2323)
  • StackOverflowError around IncrementalTrieWriterPageAware#writeRecursive() during compaction. (DB-2364)
  • NodeSync fails on upgraded nodes while a cluster is in a partially upgraded state. (DB-2385)
  • Compaction strategy instantiation errors don't generate meaningful error messages, instead return only InvocationTargetException. (DB-2404)
  • Unexpected behavior change when using row-level permissions with modification conditions like IF EXISTS. (DB-2429)
  • Authentication cache loading can exhaust native threads. The Spark master node is not able to be elected. (DB-2248)
  • Audit events for CREATE ROLE and ALTER ROLE with incorrect spacing exposes PASSWORD in plain text. (DB-2285)
  • Client warnings are not always propagated via LocalSessionWrapper. (DB-2304)
  • Timestamps inserted with ISO 8601 format are saved with wrong millisecond value. (DB-2312)
  • Compaction fails with IllegalArgumentException: null. (DB-2329)
  • Error out if not all permissions for GRANT/REVOKE/RESTRICT/UNRESTRICT are applicable for a resource. (DB-2373)
  • BulkLoader class exits without printing the stack trace for throwable error. (DB-2377)
  • Unexpected behavior change when using row-level permissions with modification conditions like IF EXISTS. (DB-2429)
  • nodetool describecluster incorrectly shows DseDelegateSnitch instead of the snitch configured in cassandra.yaml. (DSP-16158)
  • Using geo types does not work when memtable allocation type is set to offheap_objects. (DSP-16302)
  • Heap-size calculation is incorrect for RpcCallStatement + SearchIndexStatement. (DSP-16731)
  • The -graph option for the cassandra-stress tool failed on generating the target output html in the JAR file. (DSP-17046)

6.0.3 DSE Analytics

Changes and enhancements:
  • DSE pyspark libraries are added to PYTHONPATH for dse exec command. Add support for Jupyter integration. (DSP-16797)
  • DSE custom strategies allowed in Spark Structured Streaming. (DSP-16856)
  • dse spark-submit kill and status commands support optionally explicit master address. (DSP-16910, DSP-16991)
  • Address security vulnerabilities in Spark dependencies packaged with DSE. Upgrade Netty to 9.4.11. (DSP-16893)
  • Jetty 9.4.1 upgrade addresses security vulnerabilities in Spark dependencies packaged with DSE. (DSP-16893)
Resolved issues:
  • A Spark application can be registered twice in rare instances. (DSP-15247)
  • Problems with temporary and data directories for Spark applications. (DSP-15476, DSP-15880)
    • DSE client applications, like Spark, will not start if user HOME environment variable is not defined, user home directory does not exist, or the current user does not have write permissions.
    • Temporary data directory for AOSS is /var/log/spark/rdd, the same as the server-side temporary data location for Spark. Configurable with SPARK_EXECUTOR_DIRS environment variable in spark-env.sh.
    • If TMPDIR environment variable is missing, /tmp is set for all DSE apps. If /tmp directory does not exist, it is created with 1777 permissions. If directory creation fails, perform a hard stop.
  • Improved security isolates Spark applications; prevents run_as runner for Spark from running a malicious program. (DSP-16093)
  • Spark Cassandra Connector method saveToCassandra should not require solr_query column when search is enabled. (DSP-16427)
  • Cassandra streaming sink doesn't work with some sources. (DSP-16635)
  • cassandra nonsuperuser gets dsefs AccessDeniedException due to Insufficient permissions. (DSP-16713)
  • DSE Spark logging does not match OSS Spark logging levels. (DSP-16726)
  • Metastore can't handle table with 100+ columns with auto Spark SQL table creation. (DSP-16742)
  • DseDirectJoin and reading from Hive Tables does not work in Spark Structured Streaming. (DSP-16856)
  • Fully qualified paths with resource URL are resolved in Spark structured streaming checkpointing. Backport SPARK-20894. (DSP-16972)
  • AlwaysOn SQL (AOSS) dsefs directory creation does not wait for all operations to finish before closing DSEFS client. (DSP-16997)

6.0.3 DSEFS

Changes and enhancements:
  • Improved error message when performing an operation on a corrupted path. (DSP-16340)
  • Only superusers are able to remove corrupted non-empty directories when authentication is enabled for DSEFS. (DSP-16340)
Resolved issues:
  • 8 ms timeout failure when a data directory is removed. (DSP-16645)
  • In DSEFS shell, listing too many local file system directories in a single session causes a file descriptor leak. (DSP-16657)
  • DSEFS fails to start when there is a table with duration type or other type DSEFS can't understand. (DSP-16825)
  • DSEFS Hadoop layer doesn't properly translate DSEFS exceptions to Hadoop exceptions in some methods. (DSP-16933)
  • Closing DSEFS client before all issued requests are completed causes unexpected message type: DefaultLastHttpContent error. (DSP-16953)
  • Under high loads, DSEFS reports temporary incorrect state for various files/directories. (DSP-17178)

6.0.3 DSE Graph

Changes and enhancements:
  • Maximum evaluation timeout limit is 1094 days. (DSP-16709)
  • Default write consistency level (CL) for Graph is LOCAL_QUORUM. (DSP-17140)
    Attention: In earlier DSE versions, the default QUORUM write consistency level (CL) was not appropriate for multi-datacenter production environments.
Known issue:
  • Point-in-polygon queries no longer work without JTS. (DSP-17284)

    Although point-in-polygon queries previously worked without JTS, the queries used a Cartesian coordinate system implementation that did not understand the dateline. For best results, install JTS. See Spatial queries with polygons require JTS.

Resolved issues:
  • Align query behavior using geo.inside() predicate for polygon search with and without search indexes. (DSP-16108)
  • Avoid looping indefinitely when a thread making internode requests is interrupted while trying to acquire a connection. (DSP-16544)
  • Setting graph.traversal_sources.g.evaluation_timeout breaks graph. (DSP-16709)
  • Deleting a search index that was defined inside a graph fails. (DSP-16765)

6.0.3 DSE Search

Changes and enhancements:
  • Reduce the number of unique token selections for distributed searches with vnodes. (DSP-14189)

    Search load balancing strategies are per search index (per core) and are set with dsetool set_core_property.

  • Log fewer messages at INFO level in TTLIndexRebuildTask. (DSP-15600)
  • Avoid unnecessary exception and error creation in the Solr query parser. (DSP-17147)
Resolved issues:
  • Avoid accumulating redundant router state updates during schema disagreement. (DSP-15615)
  • Should not allow search index rebuild during drain. (DSP-16504)
  • NRT codec is not registered at startup for Solr cores that have switched to RT. (DSP-16663)
  • Dropping search index when index build is in progress can interrupt Solr core closure. (DSP-16774)
  • Exceptions thrown when search is enabled and table is not found in existing keyspace. (DSP-16834)
  • DSE should not start without appropriate Tomcat JAR scanning exclusions. (DSP-16841)
  • CQL single-pass queries have incorrect results when query is run with primary key and search index schema does not contain all columns in selection. (DSP-16895)

    Best practice: For optimal single-pass queries, including queries where solr_query is used with a partition restriction, and queries with partition restrictions and a search predicate, ensure that the columns to SELECT are not indexed in the search index schema.

    Workaround: Since auto-generation indexes all columns by default, you can ensure that the field is not indexed but still returned in a single-pass query. For example, this statement indexes everything except for column c3, and informs the search index schema about column c3 for efficient and correct single-pass queries.
    CREATE SEARCH INDEX ON test_search.abc WITH COLUMNS * { indexed : true }, c3 { indexed : false }; 
  • Node health score of 1 is not obtainable. Search node gets stuck at 0.00 node health score after replacing a node in a cluster. (DSP-17107)

Cassandra enhancements for DSE 6.0.3

DataStax Enterprise 6.0.3 is compatible with Apache Cassandra™ 3.11 and adds no production-certified enhancements.

DataStax Enterprise 6.0.3 is compatible with Apache Cassandra™ 3.11 and adds no production-certified enhancements.

General upgrade advice for DSE 6.0.3

General upgrade advice for DataStax Enterprise 6.0.3.

General upgrade advice for DataStax Enterprise (DSE) 6.0.3:
PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

DSE 6.0
=======

Operations
----------
   - Added batchlog_endpoint_strategy option and 2 new strategies to cassandra.yaml. The default is
     "random_remote", which works as in previous versions. Two new strategies help to choose better
     endpoints and improve batchlog write latency. See cassandra.yaml for details. (DB-1367)
   - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
     The command must be executed on the node where the rebuild operation is running. Streams
     may continue until they finish or timeout.
   - Added "-Dcassandra.replace_consistency" to support streaming each range from more than one source.
     By default, node replacement only streams one copy of data which may be stale wrt to QUORUM/LOCAL_QUORUM.
     After replacing a down node with "-Dcassandra.replace_consistency=LOCAL_DC_QUORUM", replacement node will be up
     with consistent data assuming data is inserted with LOCAL_QUORUM. Note that streaming multiple copies causes more
     IO and compactions. Alternatively, operators can run repair before bootstrapping the replacement with the
     default replace_consistency=ONE.
     Available options: "ONE"(default), "LOCAL_DC_QUORUM", "GLOBAL_QUORUM".
   - Rebuild functionality now requires to specify the streaming source endpoints and/or DC/racks
     and no longer implicitly prefers the local data center. Specifying no sources or DC/racks is
     now considered an error.
     See `nodetool help rebuild` about which combinations of DCs/racks or endpoints to include or
     exclude are possible. (DB-581)
   - Nodetool repair will now run full repairs by default, use "-inc" option to run incremental repair.
   - Incremental repairs are disallowed on tables with Materialized Views or CDC. Run full repairs on these
     tables.
   - A new HINTS stage was created to handle incoming hints. The maximum size of this thread pool is governed by
     the system property cassandra.hints.max_receive_threads, which has default value max_hints_delivery_threads
     from cassandra.yaml.
   - A new BACKGROUND_IO stage was created to handle blocking background IO operations like cache loading,
     commit log segment allocation and replay. The maximum size of this thread pool is governed by the system
     property cassandra.io.background.max_pool_size.
   - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
     set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.


Upgrading
---------
   - The ITrigger interface has been modified from "augment" to "augmentNonBlocking" for non-blocking
     internal architecture (DB-1046).
   - Removed support for directly setting JMX remote port via the com.sun.management.jmxremote.port
     system property, deprecated since 3.6. See CASSANDRA-11725 and DB-1040 for more details.
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 6.0.
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. See CASSANDRA-12888 for details.
   - Changes to permissions (CassandraAuthorizer, DseAuthorizer), roles (CassandraRoleManager,
     DseRoleManager) and credentials (PasswordAuthenticator) are broadcasted throughout the cluster.
     This eliminates the need to have short-ish validity (permissions_validity_in_ms,
     roles_validity_in_ms, credentials_validity_in_ms) and update-interval
     (permissions_update_interval_in_ms, roles_update_interval_in_ms,
     credentials_update_interval_in_ms) settings in cassandra.yaml. Consider raising the values
     for *_validity_in_ms and *_update_interval_in_ms, if configured.
     Please note, that only "reachable" nodes will receive the invalidation messages and
     unreachable nodes may still hold outdated information after DCL command.
   - REVOKE (and the new UNRESTRICT statement) now error out when the statement would not change
     anything. This means, an error is raised when the statement targets permissions which are
     not present for that resource.
   - The caches for JMX permissions and credentials no longer exist and therefore the JMX beans
     for these caches have been removed, too. Also the configuration options for the credentials
     cache need to be removed from the cassandra.yaml file.

Deprecation
-----------
    - The following properties were renamed and are now deprecated before they are removed in the next major release:
      - rpc_address -> native_transport_address
      - rpc_interface -> native_transport_interface
      - rpc_interface_prefer_ipv6 -> native_transport_interface_prefer_ipv6
      - broadcast_rpc_address -> native_transport_broadcast_address
      - rpc_keepalive -> native_transport_keepalive
    - Removed org.apache.cassandra.metrics.Table.{AntiCompactedBytes|NonAntiCompactedBytes} metrics.
    - Removed following properties from cassandra.yaml: concurrent_reads, concurrent_writes,
      concurrent_counter_writes, concurrent_materialized_view_writes, streaming_socket_timeout_in_ms


4.0
===

New features
------------
   - SSTableDump now supports the -l option to output each partition as its own json object
     See CASSANDRA-13848 for more detail
   - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
     See CASSANDRA-13132
   - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
     See CASSANDRA-11936
   - Support for arithmetic operations on number has been added. See CASSANDRA-11935
   - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
     consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
   - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
   - The initial build of materialized views can be parallelized. The number of concurrent builder
     threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
     This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.

Upgrading
---------
    - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables
      have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11.
      Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL.
      Syntax for creating compact tables is also deprecated.
    - Support for legacy auth tables in the system_auth keyspace (users,
      permissions, credentials) and the migration code has been removed. Migration
      of these legacy auth tables must have been completed before the upgrade to
      4.0 and the legacy tables must have been removed. See the 'Upgrading' section
      for version 2.2 for migration instructions.
    - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst
      Tother things, this imply the removal of all yaml option related to thrift
      ('start_rpc', rpc_port, ...).
    - Cassandra 4.0 removed support for any pre-3.0 format. This means you
      cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to
      a 3.0.x/3.x version first (and run upgradesstable). In particular, this
      mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
      will need to upgrade those sstable in 3.0.x/3.x first.
    - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous
      versions will causes issues during rolling upgrades (CASSANDRA-13274).
    - Cassandra will no longer allow invalid keyspace replication options, such
      as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
      add new nodes to a datacenter before they can set set ALTER or CREATE
      keyspace replication policies using that datacenter. Existing keyspaces
      will continue to operate, but CREATE and ALTER will validate that all
      datacenters specified exist in the cluster.
    - Cassandra 4.0 fixes a problem with incremental repair which caused repaired
      data to be inconsistent between nodes. The fix changes the behavior of both
      full and incremental repairs. For full repairs, data is no longer marked
      repaired. For incremental repairs, anticompaction is run at the beginning
      of the repair, instead of at the end. If incremental repair was being used
      prior to upgrading, a full repair should be run after upgrading to resolve
      any inconsistencies.
    - Config option index_interval has been removed (it was deprecated since 2.0)
    - Deprecated repair JMX APIs are removed.
    - The version of snappy-java has been upgraded to 1.1.2.6
    - the miniumum value for internode message timeouts is 10ms. Previously, any
      positive value was allowed. See cassandra.yaml entries like
      read_request_timeout_in_ms for more details.
    - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's
      documentation has been incorrect and the setting itself near useless.
      Batch mode remains a valid commit log mode, however.
    - There is a new commit log mode, group, which is similar to batch mode
      but blocks for up to a configurable number of milliseconds between disk flushes.
    - Due to the parallelization of the initial build of materialized views,
      the per token range view building status is stored in the new table
      `system.view_builds_in_progress`. The old table `system.views_builds_in_progress`
      is no longer used and can be removed. See CASSANDRA-12245 for more details.
    - nodetool clearsnapshot now required the --all flag to remove all snapshots.
      Previous behavior would delete all snapshots by default.

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and
      read_repair_chance table options have been deprecated, and will be removed entirely in 4.0.
      See CASSANDRA-13910 for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details


3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).


Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

3.10
====

New features
------------
   - New `DurationType` (cql duration). See CASSANDRA-11873
   - Runtime modification of concurrent_compactors is now available via nodetool
   - Support for the assignment operators +=/-= has been added for update queries.
   - An Index implementation may now provide a task which runs prior to joining
     the ring. See CASSANDRA-12039
   - Filtering on partition key columns is now also supported for queries without
     secondary indexes.
   - A slow query log has been added: slow queries will be logged at DEBUG level.
     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
     in cassandra.yaml.
   - Support for GROUP BY queries has been added.
   - A new compaction-stress tool has been added to test the throughput of compaction
     for any cassandra-stress user schema.  see compaction-stress help for how to use.
   - Compaction can now take into account overlapping tables that don't take part
     in the compaction to look for deleted or overwritten data in the compacted tables.
     Then such data is found, it can be safely discarded, which in turn should enable
     the removal of tombstones over that data.

     The behavior can be engaged in two ways:
       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
         single-table compaction on all sstables to discard deleted data in one step.
       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
         which uses overlapping tables as a source of deletions/overwrites during all
         compactions.
     The argument specifies the granularity at which deleted data is to be found:
       - If ROW is specified, only whole deleted rows (or sets of rows) will be
         discarded.
       - If CELL is specified, any columns whose value is overwritten or deleted
         will also be discarded.
       - NONE (default) specifies the old behavior, overlapping tables are not used to
         decide when to discard data.
     Which option to use depends on your workload, both ROW and CELL increase the
     disk load on compaction (especially with the size-tiered compaction strategy),
     with CELL being more resource-intensive. Both should lead to better read
     performance if deleting rows (resp. overwriting or deleting cells) is common.
   - Prepared statements are now persisted in the table prepared_statements in
     the system keyspace. Upon startup, this table is used to preload all
     previously prepared statements - i.e. in many cases clients do not need to
     re-prepare statements against restarted nodes.
   - cqlsh can now connect to older Cassandra versions by downgrading the native
     protocol version. Please note that this is currently not part of our release
     testing and, as a consequence, it is not guaranteed to work in all cases.
     See CASSANDRA-12150 for more details.
   - Snapshots that are automatically taken before a table is dropped or truncated
     will have a "dropped" or "truncated" prefix on their snapshot tag name.
   - Metrics are exposed for successful and failed authentication attempts.
     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
     See CASSANDRA-11424 for details
   - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0.
   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.

Upgrading
---------
    - Support for alter types of already defined tables and of UDTs fields has been disabled.
      If it is necessary to return a different type, please use casting instead. See
      CASSANDRA-12443 for more details.
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
    - Only Java and JavaScript are now supported UDF languages.
      The sandbox in 3.0 already prevented the use of script languages except Java
      and JavaScript.
    - Compaction now correctly drops sstables out of CompactionTask when there
      isn't enough disk space to perform the full compaction.  This should reduce
      pending compaction tasks on systems with little remaining disk space.
    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the
      "full" request time on the coordinator.  Previously, they only covered the time from
      when the coordinator sent a message to a replica until the time that the replica
      responded.  Additionally, the previous behavior was to reset the timeout when performing
      a read repair, making a second read to fix a short read, and when subranges were read
      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
      is no longer reset for these "subqueries".  The entire request must complete within
      the specified timeout.  As a consequence, your timeouts may need to be adjusted
      to account for this.  See CASSANDRA-12256 for more details.
    - Logs written to stdout are now consistent with logs written to files.
      Time is now local (it was UTC on the console and local in files). Date, thread, file
      and line info where added to stdout. (see CASSANDRA-12004)
    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided.
      The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on
      that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided
      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable
      accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues).
    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like
      commit log replay. This may break existing 3rd party tools and clients. In order to startup
      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or
      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
      snitch, encryption context. Client initialization just applies the configuration but does not
      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are
      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
    - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from
      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms
      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property.
    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format
      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).

3.8
===

New features
------------
   - Shared pool threads are now named according to the stage they are executing
     tasks for. Thread names mentioned in traced queries change accordingly.
   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
     that forces a scheduled rate of operations/sec over time. Using this, stress can
     accurately account for coordinated ommission from the stress process.
   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
     "-log hdrfile=" option. This histogram includes response/service/wait times when used with the
     fixed or throttle rate options.  The histogram file can be plotted on
     http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
     to time series compaction and new tables should use this instead of DTCS. See
     CASSANDRA-9666 for details.
   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and
     a brief explanation of on-disk locations for archived data in CommitLog form. This can
     be enabled via ALTER TABLE ... WITH cdc=true.
     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
     between unflushed CommitLogSegments and cdc_raw.
     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.

Upgrading
---------
   - The ReversedType behaviour has been corrected for clustering columns of
     BYTES type containing empty value. Scrub should be run on the existing
     SSTables containing a descending clustering column of BYTES type to correct
     their ordering. See CASSANDRA-12127 for more details.
   - Ec2MultiRegionSnitch will no longer automatically set broadcast_rpc_address
     to the public instance IP if this property is defined on cassandra.yaml.
   - The name "json" and "distinct" are not valid anymore a user-defined function
     names (they are still valid as column name however). In the unlikely case where
     you had defined functions with such names, you will need to recreate
     those under a different name, change your code to use the new names and
     drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 for more
     details).

Deprecation
-----------
   - DateTieredCompactionStrategy has been deprecated - new tables should use
     TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table to TWCS might
     cause increased compaction load for a while after the migration so make sure you run
     tests before migrating. Read CASSANDRA-9666 for background on this.

3.7
===

Upgrading
---------
   - A maximum size for SSTables values has been introduced, to prevent out of memory
     exceptions when reading corrupt SSTables. This maximum size can be set via
     max_value_size_in_mb in cassandra.yaml. The default is 256MB, which matches the default
     value of native_transport_max_frame_size_in_mb. SSTables will be considered corrupt if
     they contain values whose size exceeds this limit. See CASSANDRA-9530 for more details.


3.6
=====

New features
------------
   - JMX connections can now use the same auth mechanisms as CQL clients. New options
     in cassandra-env.(sh|ps1) enable JMX authentication and authorization to be delegated
     to the IAuthenticator and IAuthorizer configured in cassandra.yaml. The default settings
     still only expose JMX locally, and use the JVM's own security mechanisms when remote
     connections are permitted. For more details on how to enable the new options, see the
     comments in cassandra-env.sh. A new class of IResource, JMXResource, is provided for
     the purposes of GRANT/REVOKE via CQL. See CASSANDRA-10091 for more details.
     Also, directly setting JMX remote port via the com.sun.management.jmxremote.port system
     property at startup is deprecated. See CASSANDRA-11725 for more details.
   - JSON timestamps are now in UTC and contain the timezone information, see CASSANDRA-11137 for more details.
   - Collision checks are performed when joining the token ring, regardless of whether
     the node should bootstrap. Additionally, replace_address can legitimately be used
     without bootstrapping to help with recovery of nodes with partially failed disks.
     See CASSANDRA-10134 for more details.
   - Key cache will only hold indexed entries up to the size configured by
     column_index_cache_size_in_kb in cassandra.yaml in memory. Larger indexed entries
     will never go into memory. See CASSANDRA-11206 for more details.
   - For tables having a default_time_to_live specifying a TTL of 0 will remove the TTL
     from the inserted or updated values.
   - Startup is now aborted if corrupted transaction log files are found. The details
     of the affected log files are now logged, allowing the operator to decide how
     to resolve the situation.
   - Filtering expressions are made more pluggable and can be added programatically via
     a QueryHandler implementation. See CASSANDRA-11295 for more details.



3.4
===

New features
------------
    - Internal authentication now supports caching of encrypted credentials.
      Reference cassandra.yaml:credentials_validity_in_ms
    - Remote configuration of auth caches via JMX can be disabled using the
      the system property cassandra.disable_auth_caches_remote_configuration
    - sstabledump tool is added to be 3.0 version of former sstable2json. The tool only
      supports v3.0+ SSTables. See tool's help for more detail.

Upgrading
---------
    - Nothing specific to 3.4 but please see previous versions upgrading section,
      especially if you are upgrading from 2.2.

Deprecation
-----------
    - The mbean interfaces org.apache.cassandra.auth.PermissionsCacheMBean and
      org.apache.cassandra.auth.RolesCacheMBean are deprecated in favor of
      org.apache.cassandra.auth.AuthCacheMBean. This generalized interface is
      common across all caches in the auth subsystem. The specific mbean interfaces
      for each individual cache will be removed in a subsequent major version.


3.2
===

New features
------------
   - We now make sure that a token does not exist in several data directories. This
     means that we run one compaction strategy per data_file_directory and we use
     one thread per directory to flush. Use nodetool relocatesstables to make sure your
     tokens are in the correct place, or just wait and compaction will handle it. See
     CASSANDRA-6696 for more details.
   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
     tunable via cassandra.commitlog_max_outstanding_replay_bytes
   - Support for type casting has been added to the selection clause.
   - Hinted handoff now supports compression. Reference cassandra.yaml:hints_compression.
     Note: hints compression is currently disabled by default.

Upgrading
---------
   - The compression ratio metrics computation has been modified to be more accurate.
   - Running Cassandra as root is prevented by default.
   - JVM options are moved from cassandra-env.(sh|ps1) to jvm.options file

Deprecation
-----------
   - The Thrift API is deprecated and will be removed in Cassandra 4.0.


3.1
=====

Upgrading
---------
   - The return value of SelectStatement::getLimit as been changed from DataLimits
     to int.
   - Custom index implementation should be aware that the method Indexer::indexes()
     has been removed as its contract was misleading and all custom implementation
     should have almost surely returned true inconditionally for that method.
   - GC logging is now enabled by default (you can disable it in the jvm.options
     file if you prefer).

TinkerPop changes for DSE 6.0.3

A list of DataStax Enterprise 6.0.3 production-certified enhancements to TinkerPop 3.2.9.

DataStax Enterprise (DSE) 6.0.3 includes no production-certified enhancements to TinkerPop 3.2.9.

DSE 6.0.2 release notes

dse.yaml

The location of the dse.yaml file depends on the type of installation:
Package installations /etc/dse/dse.yaml
Tarball installations installation_location/resources/dse/conf/dse.yaml

19 July 2018

6.0.2 Components

All components from DSE 6.0.0 are listed. Components that are updated for DSE 6.0.2 are indicated.
  • Apache Solr™ 6.0.1.1.2321 (updated)
  • Apache Spark™ 2.2.1.2
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.5
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.45 DSE custom version
  • TinkerPop 3.3.3 with additional production-certified changes (updated)

DSE 6.0 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.0.2 Highlights

High-value benefits of upgrading to DSE 6.0.2 include these highlights:

DSE Analytics and DSEFS

  • Fixed issue where CassandraConnectionConf creates excessive database connections and reports too many HashedWheelTimer instances. (DSP-16365)

DSE Graph

  • Fixed several edge cases of using search indexes. (DSP-14802, DSP-16292)

DSE Search

  • Search index permissions can be applied at the keyspace level. (DSP-15385)
  • Schemas with stored=true work because stored=true is ignored. The workaround for 6.0.x upgrades with schema.xml fields with “indexed=false, stored=true, docValues=true” is no longer required. (DSP-16392)
  • Minor bug fixes and error handling improvements. (DSP-16435, DSP-16061, DSP-16078)

6.0.2 DSE core

Changes and enhancements:
  • sstableloader supports custom config file locations. (DSP-16092)
  • -d option to create local encryption keys without configuring the directory in dse.yaml. (DSP-15380)
Resolved issues:
  • Tool sstablepartitions for identifying large partitions. (DB-803)
  • Show delegated snitch in nodetool describecluster. (DB-2057)
  • Use more precise grep patterns to prevent accidental matches in cassandra-env.sh. (DB-2114)
  • Add missing equality sign to SASI schema snapshot. (DB-2129)
  • For tables using DSE Tiered Storage, nodetool cleanup places cleaned SSTables in the wrong tier. (DB-2173)
  • Support creating system keys before the output directory is configured in dse.yaml. (DSP-15380)
  • Client prepared statements are not populated in system.prepared_statements table. (DSP-15900)
  • Improved compatibility with external tables stored in the DSE Metastore in remote systems. (DSP-16561)

6.0.2 DSE Analytics

Changes and enhancements:

  • Apache Hadoop Azure libraries for Hadoop 2.7.1 have been added to the Spark classpath to simplify integration with Microsoft Azure and Microsoft Azure Blob Storage. (DSP-15943)
  • AlwaysOn SQL (AOSS) improvements:
    • AlwaysOn SQL (AOSS) support for enabling Kerberos and SSL at the same time. (DSP-16087)
    • Add 120 seconds wait time so that Spark Master recovery process completes before status check of AlwaysOn SQL (AOSS) app. (DSP-16249)
    • AlwaysOn SQL (AOSS) driver continually runs on a node even when DSE is down. (DSP-16297)
    • AlwaysOn SQL (AOSS) binds to native_transport_address. (DSP-16469)
    • Improved defaults and errors for AlwaysOn SQL (AOSS) workpool. (DSP-16343)
Resolved issues:
  • CassandraConnectionConf creates excessive database connections and reports too many HashedWheelTimer instances. (DSP-16365)
  • Need to disable cluster object JMX metrics report to prevent count exceptions spam in Spark driver log. (DSP-16442)
  • Fixed Spark-Connector dependencies and published SparkBuildExamples. (DSP-16699)

6.0.2 DSEFS

Changes and enhancements:

  • DSEFS operations: chown, chgrp, and chmod support recursive (-R) and verbose (-v) flag. (DSP-14238)
  • Client and internode connection improvements. (DSP-14284, DSP-16065)
  • DSEFS clients close idle connections after 60 seconds, configurable in dse.yaml. (DSP-14284)
  • Improvements to DataSourceInputStream remove possible lockup. (DSP-16409)
    • If the second read is issued after a failed read, it is not blocked forever. The stream is automatically closed on errors, and subsequent reads will fail with IllegalStateException.
    • The timeout message includes information about the underlying DataSource object.
    • No more reads are issued to the underlying DataSource after it reports hasMoreData = false.
    • The read loop has been simplified to properly move to the next buffer if the requested number of bytes hasn't been delivered yet.
    • Empty buffer returned from the DataSource when hasMoreData = true is not treated as an EOF. The read method validates offset and length arguments.
  • Security improvement: DSEFS uses an isolated native memory pool for file data and metadata sent between nodes. This isolation makes it harder to exploit potential memory management bugs. (DSP-16492)

Resolved issues:

  • DSEFS silently fails when TCP port 5599 is not open between nodes. (DSP-16101)

6.0.2 DSE Graph

Changes and enhancements:
  • Vertices and vertex properties created or modified with graphframes respect TTL as defined in the schema. In earlier versions, vertices and vertex properties had no TTL. Edges created or modified with graphframes continue to have no TTL. (DSP-15555)
  • Improved Gremlin console authentication configuration. (DSP-9905)
Resolved issues:
  • 0 (zero) is not treated as unlimited abort of max num errors. (DGL-307)
  • Search indexes are broken for multi cardinality properties. (DSP-14802)
  • DGF interceptor does not take into account GraphStep parameters with g.V(id) queries. (DSP-16172)
  • The clause LIMIT does not work in a graph traversal with search predicate TOKEN, returning only a subset of expected results. (DSP-16292)

6.0.2 DSE Search

Changes and enhancements:

  • The node health option uptime_ramp_up_period_seconds default value in dse.yaml is reduced to 3 hours (10800 seconds). (DSP-15752)
  • CQL solr_query supports Solr facet heatmaps. (DSP-16404)
  • Improved handling of asynchronous I/O timeouts during search read-before-write. (DSP-16061)
  • Schemas with stored=true work because stored=true is ignored. (DSP-16392)
  • Use monotonically increasing time source for search query execution latency calculation. (DSP-16435)

Resolved issues:

  • Search index permissions can be applied at keyspace level. (DSP-15835)
  • The encryptors thread cache in ThreadLocalIndexEncryptionConfiguration leaves entries in the cache. (DSP-16078)
  • Classpath conflict between Lucene and SASI versions of Snowball. (DSP-16116)
  • Indexing fails if fields have 'indexed=false', 'stored=true', and `docValues=true'. (DSP-16392)

DataStax Bulk Loader 1.1.0

Changes and enhancements:

Cassandra enhancements for DSE 6.0.2

DataStax Enterprise 6.0.2 is compatible with Apache Cassandra™ 3.11 and adds no production-certified enhancements.

DataStax Enterprise 6.0.2 is compatible with Apache Cassandra™ 3.11 and adds no production-certified enhancements.

General upgrade advice for DSE 6.0.2

General upgrade advice for DataStax Enterprise 6.0.2.

General upgrade advice for DataStax Enterprise (DSE) 6.0.2:
PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

DSE 6.0
=======

Operations
----------
   - Added batchlog_endpoint_strategy option and 2 new strategies to cassandra.yaml. The default is
     "random_remote", which works as in previous versions. Two new strategies help to choose better
     endpoints and improve batchlog write latency. See cassandra.yaml for details. (DB-1367)
   - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
     The command must be executed on the node where the rebuild operation is running. Streams
     may continue until they finish or timeout.
   - Added "-Dcassandra.replace_consistency" to support streaming each range from more than one source.
     By default, node replacement only streams one copy of data which may be stale wrt to QUORUM/LOCAL_QUORUM.
     After replacing a down node with "-Dcassandra.replace_consistency=LOCAL_DC_QUORUM", replacement node will be up
     with consistent data assuming data is inserted with LOCAL_QUORUM. Note that streaming multiple copies causes more
     IO and compactions. Alternatively, operators can run repair before bootstrapping the replacement with the
     default replace_consistency=ONE.
     Available options: "ONE"(default), "LOCAL_DC_QUORUM", "GLOBAL_QUORUM".
   - Rebuild functionality now requires to specify the streaming source endpoints and/or DC/racks
     and no longer implicitly prefers the local data center. Specifying no sources or DC/racks is
     now considered an error.
     See `nodetool help rebuild` about which combinations of DCs/racks or endpoints to include or
     exclude are possible. (DB-581)
   - Nodetool repair will now run full repairs by default, use "-inc" option to run incremental repair.
   - Incremental repairs are disallowed on tables with Materialized Views or CDC. Run full repairs on these
     tables.
   - A new HINTS stage was created to handle incoming hints. The maximum size of this thread pool is governed by
     the system property cassandra.hints.max_receive_threads, which has default value max_hints_delivery_threads
     from cassandra.yaml.
   - A new BACKGROUND_IO stage was created to handle blocking background IO operations like cache loading,
     commit log segment allocation and replay. The maximum size of this thread pool is governed by the system
     property cassandra.io.background.max_pool_size.
   - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
     set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.


Upgrading
---------
   - The ITrigger interface has been modified from "augment" to "augmentNonBlocking" for non-blocking
     internal architecture (DB-1046).
   - Removed support for directly setting JMX remote port via the com.sun.management.jmxremote.port
     system property, deprecated since 3.6. See CASSANDRA-11725 and DB-1040 for more details.
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 6.0.
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. See CASSANDRA-12888 for details.
   - Changes to permissions (CassandraAuthorizer, DseAuthorizer), roles (CassandraRoleManager,
     DseRoleManager) and credentials (PasswordAuthenticator) are broadcasted throughout the cluster.
     This eliminates the need to have short-ish validity (permissions_validity_in_ms,
     roles_validity_in_ms, credentials_validity_in_ms) and update-interval
     (permissions_update_interval_in_ms, roles_update_interval_in_ms,
     credentials_update_interval_in_ms) settings in cassandra.yaml. Consider raising the values
     for *_validity_in_ms and *_update_interval_in_ms, if configured.
     Please note, that only "reachable" nodes will receive the invalidation messages and
     unreachable nodes may still hold outdated information after DCL command.
   - REVOKE (and the new UNRESTRICT statement) now error out when the statement would not change
     anything. This means, an error is raised when the statement targets permissions which are
     not present for that resource.
   - The caches for JMX permissions and credentials no longer exist and therefore the JMX beans
     for these caches have been removed, too. Also the configuration options for the credentials
     cache need to be removed from the cassandra.yaml file.

Deprecation
-----------
    - The following properties were renamed and are now deprecated before they are removed in the next major release:
      - rpc_address -> native_transport_address
      - rpc_interface -> native_transport_interface
      - rpc_interface_prefer_ipv6 -> native_transport_interface_prefer_ipv6
      - broadcast_rpc_address -> native_transport_broadcast_address
      - rpc_keepalive -> native_transport_keepalive
    - Removed org.apache.cassandra.metrics.Table.{AntiCompactedBytes|NonAntiCompactedBytes} metrics.
    - Removed following properties from cassandra.yaml: concurrent_reads, concurrent_writes,
      concurrent_counter_writes, concurrent_materialized_view_writes, streaming_socket_timeout_in_ms


4.0
===

New features
------------
   - SSTableDump now supports the -l option to output each partition as its own json object
     See CASSANDRA-13848 for more detail
   - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
     See CASSANDRA-13132
   - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
     See CASSANDRA-11936
   - Support for arithmetic operations on number has been added. See CASSANDRA-11935
   - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
     consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
   - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
   - The initial build of materialized views can be parallelized. The number of concurrent builder
     threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
     This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.

Upgrading
---------
    - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables
      have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11.
      Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL.
      Syntax for creating compact tables is also deprecated.
    - Support for legacy auth tables in the system_auth keyspace (users,
      permissions, credentials) and the migration code has been removed. Migration
      of these legacy auth tables must have been completed before the upgrade to
      4.0 and the legacy tables must have been removed. See the 'Upgrading' section
      for version 2.2 for migration instructions.
    - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst
      Tother things, this imply the removal of all yaml option related to thrift
      ('start_rpc', rpc_port, ...).
    - Cassandra 4.0 removed support for any pre-3.0 format. This means you
      cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to
      a 3.0.x/3.x version first (and run upgradesstable). In particular, this
      mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
      will need to upgrade those sstable in 3.0.x/3.x first.
    - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous
      versions will causes issues during rolling upgrades (CASSANDRA-13274).
    - Cassandra will no longer allow invalid keyspace replication options, such
      as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
      add new nodes to a datacenter before they can set set ALTER or CREATE
      keyspace replication policies using that datacenter. Existing keyspaces
      will continue to operate, but CREATE and ALTER will validate that all
      datacenters specified exist in the cluster.
    - Cassandra 4.0 fixes a problem with incremental repair which caused repaired
      data to be inconsistent between nodes. The fix changes the behavior of both
      full and incremental repairs. For full repairs, data is no longer marked
      repaired. For incremental repairs, anticompaction is run at the beginning
      of the repair, instead of at the end. If incremental repair was being used
      prior to upgrading, a full repair should be run after upgrading to resolve
      any inconsistencies.
    - Config option index_interval has been removed (it was deprecated since 2.0)
    - Deprecated repair JMX APIs are removed.
    - The version of snappy-java has been upgraded to 1.1.2.6
    - the miniumum value for internode message timeouts is 10ms. Previously, any
      positive value was allowed. See cassandra.yaml entries like
      read_request_timeout_in_ms for more details.
    - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's
      documentation has been incorrect and the setting itself near useless.
      Batch mode remains a valid commit log mode, however.
    - There is a new commit log mode, group, which is similar to batch mode
      but blocks for up to a configurable number of milliseconds between disk flushes.
    - Due to the parallelization of the initial build of materialized views,
      the per token range view building status is stored in the new table
      `system.view_builds_in_progress`. The old table `system.views_builds_in_progress`
      is no longer used and can be removed. See CASSANDRA-12245 for more details.
    - nodetool clearsnapshot now required the --all flag to remove all snapshots.
      Previous behavior would delete all snapshots by default.

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and
      read_repair_chance table options have been deprecated, and will be removed entirely in 4.0.
      See CASSANDRA-13910 for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details


3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).


Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

3.10
====

New features
------------
   - New `DurationType` (cql duration). See CASSANDRA-11873
   - Runtime modification of concurrent_compactors is now available via nodetool
   - Support for the assignment operators +=/-= has been added for update queries.
   - An Index implementation may now provide a task which runs prior to joining
     the ring. See CASSANDRA-12039
   - Filtering on partition key columns is now also supported for queries without
     secondary indexes.
   - A slow query log has been added: slow queries will be logged at DEBUG level.
     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
     in cassandra.yaml.
   - Support for GROUP BY queries has been added.
   - A new compaction-stress tool has been added to test the throughput of compaction
     for any cassandra-stress user schema.  see compaction-stress help for how to use.
   - Compaction can now take into account overlapping tables that don't take part
     in the compaction to look for deleted or overwritten data in the compacted tables.
     Then such data is found, it can be safely discarded, which in turn should enable
     the removal of tombstones over that data.

     The behavior can be engaged in two ways:
       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
         single-table compaction on all sstables to discard deleted data in one step.
       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
         which uses overlapping tables as a source of deletions/overwrites during all
         compactions.
     The argument specifies the granularity at which deleted data is to be found:
       - If ROW is specified, only whole deleted rows (or sets of rows) will be
         discarded.
       - If CELL is specified, any columns whose value is overwritten or deleted
         will also be discarded.
       - NONE (default) specifies the old behavior, overlapping tables are not used to
         decide when to discard data.
     Which option to use depends on your workload, both ROW and CELL increase the
     disk load on compaction (especially with the size-tiered compaction strategy),
     with CELL being more resource-intensive. Both should lead to better read
     performance if deleting rows (resp. overwriting or deleting cells) is common.
   - Prepared statements are now persisted in the table prepared_statements in
     the system keyspace. Upon startup, this table is used to preload all
     previously prepared statements - i.e. in many cases clients do not need to
     re-prepare statements against restarted nodes.
   - cqlsh can now connect to older Cassandra versions by downgrading the native
     protocol version. Please note that this is currently not part of our release
     testing and, as a consequence, it is not guaranteed to work in all cases.
     See CASSANDRA-12150 for more details.
   - Snapshots that are automatically taken before a table is dropped or truncated
     will have a "dropped" or "truncated" prefix on their snapshot tag name.
   - Metrics are exposed for successful and failed authentication attempts.
     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
     See CASSANDRA-11424 for details
   - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0.
   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.

Upgrading
---------
    - Support for alter types of already defined tables and of UDTs fields has been disabled.
      If it is necessary to return a different type, please use casting instead. See
      CASSANDRA-12443 for more details.
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
    - Only Java and JavaScript are now supported UDF languages.
      The sandbox in 3.0 already prevented the use of script languages except Java
      and JavaScript.
    - Compaction now correctly drops sstables out of CompactionTask when there
      isn't enough disk space to perform the full compaction.  This should reduce
      pending compaction tasks on systems with little remaining disk space.
    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the
      "full" request time on the coordinator.  Previously, they only covered the time from
      when the coordinator sent a message to a replica until the time that the replica
      responded.  Additionally, the previous behavior was to reset the timeout when performing
      a read repair, making a second read to fix a short read, and when subranges were read
      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
      is no longer reset for these "subqueries".  The entire request must complete within
      the specified timeout.  As a consequence, your timeouts may need to be adjusted
      to account for this.  See CASSANDRA-12256 for more details.
    - Logs written to stdout are now consistent with logs written to files.
      Time is now local (it was UTC on the console and local in files). Date, thread, file
      and line info where added to stdout. (see CASSANDRA-12004)
    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided.
      The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on
      that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided
      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable
      accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues).
    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like
      commit log replay. This may break existing 3rd party tools and clients. In order to startup
      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or
      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
      snitch, encryption context. Client initialization just applies the configuration but does not
      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are
      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
    - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from
      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms
      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property.
    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format
      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).

3.8
===

New features
------------
   - Shared pool threads are now named according to the stage they are executing
     tasks for. Thread names mentioned in traced queries change accordingly.
   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
     that forces a scheduled rate of operations/sec over time. Using this, stress can
     accurately account for coordinated ommission from the stress process.
   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
     "-log hdrfile=" option. This histogram includes response/service/wait times when used with the
     fixed or throttle rate options.  The histogram file can be plotted on
     http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
     to time series compaction and new tables should use this instead of DTCS. See
     CASSANDRA-9666 for details.
   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and
     a brief explanation of on-disk locations for archived data in CommitLog form. This can
     be enabled via ALTER TABLE ... WITH cdc=true.
     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
     between unflushed CommitLogSegments and cdc_raw.
     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.

Upgrading
---------
   - The ReversedType behaviour has been corrected for clustering columns of
     BYTES type containing empty value. Scrub should be run on the existing
     SSTables containing a descending clustering column of BYTES type to correct
     their ordering. See CASSANDRA-12127 for more details.
   - Ec2MultiRegionSnitch will no longer automatically set broadcast_rpc_address
     to the public instance IP if this property is defined on cassandra.yaml.
   - The name "json" and "distinct" are not valid anymore a user-defined function
     names (they are still valid as column name however). In the unlikely case where
     you had defined functions with such names, you will need to recreate
     those under a different name, change your code to use the new names and
     drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 for more
     details).

Deprecation
-----------
   - DateTieredCompactionStrategy has been deprecated - new tables should use
     TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table to TWCS might
     cause increased compaction load for a while after the migration so make sure you run
     tests before migrating. Read CASSANDRA-9666 for background on this.

3.7
===

Upgrading
---------
   - A maximum size for SSTables values has been introduced, to prevent out of memory
     exceptions when reading corrupt SSTables. This maximum size can be set via
     max_value_size_in_mb in cassandra.yaml. The default is 256MB, which matches the default
     value of native_transport_max_frame_size_in_mb. SSTables will be considered corrupt if
     they contain values whose size exceeds this limit. See CASSANDRA-9530 for more details.


3.6
=====

New features
------------
   - JMX connections can now use the same auth mechanisms as CQL clients. New options
     in cassandra-env.(sh|ps1) enable JMX authentication and authorization to be delegated
     to the IAuthenticator and IAuthorizer configured in cassandra.yaml. The default settings
     still only expose JMX locally, and use the JVM's own security mechanisms when remote
     connections are permitted. For more details on how to enable the new options, see the
     comments in cassandra-env.sh. A new class of IResource, JMXResource, is provided for
     the purposes of GRANT/REVOKE via CQL. See CASSANDRA-10091 for more details.
     Also, directly setting JMX remote port via the com.sun.management.jmxremote.port system
     property at startup is deprecated. See CASSANDRA-11725 for more details.
   - JSON timestamps are now in UTC and contain the timezone information, see CASSANDRA-11137 for more details.
   - Collision checks are performed when joining the token ring, regardless of whether
     the node should bootstrap. Additionally, replace_address can legitimately be used
     without bootstrapping to help with recovery of nodes with partially failed disks.
     See CASSANDRA-10134 for more details.
   - Key cache will only hold indexed entries up to the size configured by
     column_index_cache_size_in_kb in cassandra.yaml in memory. Larger indexed entries
     will never go into memory. See CASSANDRA-11206 for more details.
   - For tables having a default_time_to_live specifying a TTL of 0 will remove the TTL
     from the inserted or updated values.
   - Startup is now aborted if corrupted transaction log files are found. The details
     of the affected log files are now logged, allowing the operator to decide how
     to resolve the situation.
   - Filtering expressions are made more pluggable and can be added programatically via
     a QueryHandler implementation. See CASSANDRA-11295 for more details.



3.4
===

New features
------------
    - Internal authentication now supports caching of encrypted credentials.
      Reference cassandra.yaml:credentials_validity_in_ms
    - Remote configuration of auth caches via JMX can be disabled using the
      the system property cassandra.disable_auth_caches_remote_configuration
    - sstabledump tool is added to be 3.0 version of former sstable2json. The tool only
      supports v3.0+ SSTables. See tool's help for more detail.

Upgrading
---------
    - Nothing specific to 3.4 but please see previous versions upgrading section,
      especially if you are upgrading from 2.2.

Deprecation
-----------
    - The mbean interfaces org.apache.cassandra.auth.PermissionsCacheMBean and
      org.apache.cassandra.auth.RolesCacheMBean are deprecated in favor of
      org.apache.cassandra.auth.AuthCacheMBean. This generalized interface is
      common across all caches in the auth subsystem. The specific mbean interfaces
      for each individual cache will be removed in a subsequent major version.


3.2
===

New features
------------
   - We now make sure that a token does not exist in several data directories. This
     means that we run one compaction strategy per data_file_directory and we use
     one thread per directory to flush. Use nodetool relocatesstables to make sure your
     tokens are in the correct place, or just wait and compaction will handle it. See
     CASSANDRA-6696 for more details.
   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
     tunable via cassandra.commitlog_max_outstanding_replay_bytes
   - Support for type casting has been added to the selection clause.
   - Hinted handoff now supports compression. Reference cassandra.yaml:hints_compression.
     Note: hints compression is currently disabled by default.

Upgrading
---------
   - The compression ratio metrics computation has been modified to be more accurate.
   - Running Cassandra as root is prevented by default.
   - JVM options are moved from cassandra-env.(sh|ps1) to jvm.options file

Deprecation
-----------
   - The Thrift API is deprecated and will be removed in Cassandra 4.0.


3.1
=====

Upgrading
---------
   - The return value of SelectStatement::getLimit as been changed from DataLimits
     to int.
   - Custom index implementation should be aware that the method Indexer::indexes()
     has been removed as its contract was misleading and all custom implementation
     should have almost surely returned true inconditionally for that method.
   - GC logging is now enabled by default (you can disable it in the jvm.options
     file if you prefer).

TinkerPop changes for DSE 6.0.2

A list of DataStax Enterprise 6.0.2 production-certified enhancements to TinkerPop 3.2.9.

DataStax Enterprise (DSE) 6.0.2 includes these production-certified enhancements to TinkerPop 3.2.9:

  • Implemented TraversalSelectStep which allows to select() runtime-generated keys.
  • Coerced BulkSet to g:List in GraphSON 3.0.
  • Deprecated CredentialsGraph DSL in favor of CredentialsTraversalDsl which uses the recommended method for Gremlin DSL development.
  • Allowed iterate() to be called after profile().
  • Fixed regression issue where the HTTPChannelizer doesn’t instantiate the specified AuthenticationHandler.
  • Defaulted GLV tests for gremlin-python to run for GraphSON 3.0.
  • Fixed a bug with Tree serialization in GraphSON 3.0.
  • In gremlin-python, the GraphSON 3.0 g:Set type is now deserialized to List.

DSE 6.0.1 release notes

5 June 2018

6.0.1 Components

All components from DSE 6.0.0 are listed. Components that are updated for DSE 6.0.1 are indicated.
  • Apache Solr™ 6.0.1.1.2295 (updated)
  • Apache Spark™ 2.2.1.2 (updated)
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.5
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.45 DSE custom version (updated)
  • TinkerPop 3.3.2 with additional production-certified changes (updated)

DSE 6.0.1 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.0.1 Highlights

High-value benefits of upgrading to DSE 6.0.1 include these highlights:

DataStax Enterprise core

  • Fix binding JMX to any address. (DB-2081)
  • DataStax Bulk Loader 1.0.2 is bundled with DSE 6.0.1. (DSP-16206)

DSE Analytics and DSEFS

  • Upgrade to Spark 2.2.1 for bug fixes.
  • Fixed issue where multiple Spark Masters can be started on the same machine. (DSP-15636)
  • Improved Spark Master discovery and reliability. (DSP-15801, DSP-14405)
  • Improved AlwaysOn SQL (AOSS) startup reliability. (DSP-15871, DSP-15468, DSP-15695, DSP-15839)
  • Resolved the missing /tmp directory in DSEFS after fresh cluster installation. (DSP-16058)
  • Fixed handling of Parquet files with partitions. (DSP-16067)
  • Fixed the HashedWheelTimer leak in Spark Connector that affected BYOS. (DSP-15569)

DSE Search

  • Fix for the known issue that prevented using TTL (time-to-live) with DSE Search live indexing (RT indexing). (DSP-16038, DSP-14216)
  • Addresses security vulnerabilities in libraries packaged with DSE. (DSP-15978)
  • Fix for using faceting with non-zero offsets. (DSP-15946)
  • Fix for ORDER BY clauses in native CQL syntax. (DSP-16064)

6.0.1 DSE core

Changes and enhancements:
  • Improved NodeSync usability with secure environments. (DB-2034)
  • sstableloader supports custom config file locations. (DSP-16092)
  • LDAP tuning parameters allow all LDAP connection pool options to be set. (DSP-15948)
Resolved issues:
  • Use the indexed item type as backing table key validator of 2i on collections. (DB-1121)
  • Add getConcurrentCompactors to JMX in order to avoid loading DatabaseDescriptor to check its value in nodetool. (DB-1730)
  • Send a final error message when a continuous paging session is cancelled. (DB-1798)
  • Ignore empty counter cells on digest calculation. (DB-1881)
  • Apply view batchlog mutation parallel with local view mutations. (DB-1900)
  • Use same IO queue depth as Linux scheduler and advise against overriding it. (DB-1909)
  • Fix startup error message rejecting COMPACT STORAGE after upgrade. (DB-1916)
  • Improve user warnings on startup when libaio package is not installed. (DB-1917)
  • Set MX4J_ADDRESS to 127.0.0.1 if not explicitly set. (DB-1950)
  • Prevent OOM due to OutboundTcpConnection backlog by dropping request messages after the queue becomes too large. (DB-2001)
  • Fix exception in trace log messages of non-frozen user types. (DB-2005)
  • Limit max cached direct buffer on NIO to 1 MB. (DB-2028)
  • Reusing table ID with CREATE TABLE causes failure on restart. (DB-2032)
  • BulkLoader class exits without printing the stack trace. (DB-2033)
  • Fix binding JMX to any address. (DB-2081)
  • sstableloader does not decrypt passwords using config encryption in DSE. (DSP-13492)
  • dse client-tool help doesn't work if ~/.dserc file exists. (DSP-15869)

6.0.1 DSE Analytics

Known issues:

  • The Spark Jobserver demo has an incorrect version for the Spark Jobserver API. (DSP-15832)

    Workaround: In the demo's gradle.properties file, change the version from 0.6.2 to 0.6.2.238.

Changes and enhancements:
  • Decreased the number of exceptions logged during master move from node to node. (DSP-14405)
  • When querying remote cluster from Spark job, connector does not route requests to data replicas. (DSP-15202)
  • Long CassandraRDD.where clauses throw StackOverflow exceptions. (DSP-15438)
  • AlwaysOn SQL dependency on JPS is removed. The jps_directory entry in dse.yaml is removed. (DSP-15468)
  • Improved AlwaysOn SQL configuration. (DSP-15734)
  • Improved security for Spark JobServer. All uploaded JARs, temporary files, and logs are created under the current user's home directory: ~/.spark-jobserver. (DSP-15832)
  • Improved process scanning for AlwaysOn SQL driver. (DSP-15839)
  • In Portfolio demo, pricer is no longer required to be run with sudo. (DSP-15970)
  • Scala 2.10 in BYOS is no longer supported. (DSP-15999)
  • Improved validation for authentication configuration for AlwaysOn SQL. (DSP-16018)
  • Optimize memoizing converters for UDTs. (DSP-16121)
  • During misconfigured cluster bootstrap, the AlwaysOn SqlServer does not start due to missing /tmp/hive directory in DSEFS. (DSP-16058)
Resolved issues:
  • A shard request timeout caused an assertion error from Lucene getNumericDocValues in the log. (DSP-14216)
  • Multiple Spark Masters can be started on the same machine. (DSP-15636)
  • Do not start AlwaysOn SQL until Spark Master is ready. (DSP-15695)
  • DSE client tool returns wrong Spark Master address. (DSP-15801)
  • In some situations, AlwaysOn SQL cannot start unless DSE node is restarted. (DSP-15871)
  • Portfolio demo does not work on package installs. (DSP-15970)
  • Java driver in Spark Connector uses daemon threads to prevent shutdown hooks from being blocked by driver thread pools. (DSP-16051)
  • dse client-tool spark sql-schema --all exports definitions for solr_admin keyspace. (DSP-16073).
  • HashedWheelTimer leak in Spark Connector, affecting BYOS. (DSP-15569)

6.0.1 DSEFS

Resolved issues:
  • Can't quote file patterns in DSEFS shell. (DSP-15550)

6.0.1 DSE Graph

Changes and enhancements:
  • DseGraphFrame performance improvement reduces number of joins for count() and other id only queries. (DSP-15554)
  • Performance improvements for traversal execution with Fluent API and script-based executions. (DSP-15686)
Resolved issues:
  • edge_threads and vertex_threads can end up being 0. (DGL-305)
  • When using graph frames, cannot upload edges when ids for vertices are complex non-text ids. (DSP-15614)
  • CassandraHiveMetastore is prevented from adding multiple partitions for file-based data sources. Fixes MSCK REPAIR TABLE command. (DSP-16067)

6.0.1 DSE Search

Changes and enhancements:
Resolved issues:
  • Offline sstable tools fail is DSE Search index is present on a table. (DSP-15628)
  • HTTP read on solr_stress doesn't inject random data into placeholders. (DSP-15727)
  • Servlet container shutdown (Tomcat) prematurely stops logback context. (DSP-15807)
  • ERROR 500 on distributed http json.facet with non-zero offset. (DSP-15946)
  • Search index TTL Expiration thread loops without effect with live indexing (RT indexing). (DSP-16038)
  • Search incorrectly assumes only single-row ORDER BY clauses on first clustering key. (DSP-16064)

DataStax Bulk Loader 1.0.2

  • DataStax Bulk Loader 1.0.2 is bundled with DSE 6.0.1. (DSP-16206)
DataStax recommends using the latest DataStax Bulk Loader 1.2.0 For details, see DataStax Bulk Loader.

Cassandra enhancements for DSE 6.0.1

DataStax Enterprise 6.0.1 is compatible with Apache Cassandra™ 3.11 and adds production-certified enhancements.

DataStax Enterprise 6.0.1 is compatible with Apache Cassandra™ 3.11 and adds these production-certified changes:

  • cassandra-stress throws NPE if insert section isn't specified in user profile (CASSSANDRA-14426)
  • nodetool listsnapshots is missing local system keyspace snapshots (CASSANDRA-14381)
  • Remove string formatting lines from BufferPool hot path (CASSANDRA-14416)
  • Detect OpenJDK jvm type and architecture (CASSANDRA-12793)
  • Don't use guava collections in the non-system keyspace jmx attributes (CASSANDRA-12271)
  • Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
  • Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
  • Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
  • CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  • Serialize empty buffer as empty string for json output format (CASSANDRA-14245)
  • Cassandra not starting when using enhanced startup scripts in windows (CASSANDRA-14418)
  • Fix progress stats and units in compactionstats (CASSANDRA-12244)
  • Better handle missing partition columns in system_schema.columns (CASSANDRA-14379)
  • Deprecate background repair and probablistic read_repair_chance table options (CASSANDRA-13910)
  • Delay hints store excise by write timeout to avoid race with decommission (CASSANDRA-13740)
  • Add missed CQL keywords to documentation (CASSANDRA-14359)
  • Avoid deadlock when running nodetool refresh before node is fully up (CASSANDRA-14310)
  • Handle all exceptions when opening sstables (CASSANDRA-14202)
  • Handle incompletely written hint descriptors during startup (CASSANDRA-14080)
  • Handle repeat open bound from SRP in read repair (CASSANDRA-14330)
  • CqlRecordReader no longer quotes the keyspace when connecting, as the java driver will (CASSANDRA-10751)
  • Fix compaction failure caused by reading un-flushed data (CASSANDRA-12743)
  • Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
  • CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
  • Check checksum before decompressing data (CASSANDRA-14284)

General upgrade advice DSE 6.0.1

General upgrade advice for DataStax Enterprise 6.0.1.

General upgrade advice for DataStax Enterprise (DSE) 6.0.1:
PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

DSE 6.0
=======

Operations
----------
   - Added batchlog_endpoint_strategy option and 2 new strategies to cassandra.yaml. The default is
     "random_remote", which works as in previous versions. Two new strategies help to choose better
     endpoints and improve batchlog write latency. See cassandra.yaml for details. (DB-1367)
   - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
     The command must be executed on the node where the rebuild operation is running. Streams
     may continue until they finish or timeout.
   - Added "-Dcassandra.replace_consistency" to support streaming each range from more than one source.
     By default, node replacement only streams one copy of data which may be stale wrt to QUORUM/LOCAL_QUORUM.
     After replacing a down node with "-Dcassandra.replace_consistency=LOCAL_DC_QUORUM", replacement node will be up
     with consistent data assuming data is inserted with LOCAL_QUORUM. Note that streaming multiple copies causes more
     IO and compactions. Alternatively, operators can run repair before bootstrapping the replacement with the
     default replace_consistency=ONE.
     Available options: "ONE"(default), "LOCAL_DC_QUORUM", "GLOBAL_QUORUM".
   - Rebuild functionality now requires to specify the streaming source endpoints and/or DC/racks
     and no longer implicitly prefers the local data center. Specifying no sources or DC/racks is
     now considered an error.
     See `nodetool help rebuild` about which combinations of DCs/racks or endpoints to include or
     exclude are possible. (DB-581)
   - Nodetool repair will now run full repairs by default, use "-inc" option to run incremental repair.
   - Incremental repairs are disallowed on tables with Materialized Views or CDC. Run full repairs on these
     tables.
   - A new HINTS stage was created to handle incoming hints. The maximum size of this thread pool is governed by
     the system property cassandra.hints.max_receive_threads, which has default value max_hints_delivery_threads
     from cassandra.yaml.
   - A new BACKGROUND_IO stage was created to handle blocking background IO operations like cache loading,
     commit log segment allocation and replay. The maximum size of this thread pool is governed by the system
     property cassandra.io.background.max_pool_size.
   - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
     set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.


Upgrading
---------
   - The ITrigger interface has been modified from "augment" to "augmentNonBlocking" for non-blocking
     internal architecture (DB-1046).
   - Removed support for directly setting JMX remote port via the com.sun.management.jmxremote.port
     system property, deprecated since 3.6. See CASSANDRA-11725 and DB-1040 for more details.
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 6.0.
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. See CASSANDRA-12888 for details.
   - Changes to permissions (CassandraAuthorizer, DseAuthorizer), roles (CassandraRoleManager,
     DseRoleManager) and credentials (PasswordAuthenticator) are broadcasted throughout the cluster.
     This eliminates the need to have short-ish validity (permissions_validity_in_ms,
     roles_validity_in_ms, credentials_validity_in_ms) and update-interval
     (permissions_update_interval_in_ms, roles_update_interval_in_ms,
     credentials_update_interval_in_ms) settings in cassandra.yaml. Consider raising the values
     for *_validity_in_ms and *_update_interval_in_ms, if configured.
     Please note, that only "reachable" nodes will receive the invalidation messages and
     unreachable nodes may still hold outdated information after DCL command.
   - REVOKE (and the new UNRESTRICT statement) now error out when the statement would not change
     anything. This means, an error is raised when the statement targets permissions which are
     not present for that resource.
   - The caches for JMX permissions and credentials no longer exist and therefore the JMX beans
     for these caches have been removed, too. Also the configuration options for the credentials
     cache need to be removed from the cassandra.yaml file.

Deprecation
-----------
    - The following properties were renamed and are now deprecated before they are removed in the next major release:
      - rpc_address -> native_transport_address
      - rpc_interface -> native_transport_interface
      - rpc_interface_prefer_ipv6 -> native_transport_interface_prefer_ipv6
      - broadcast_rpc_address -> native_transport_broadcast_address
      - rpc_keepalive -> native_transport_keepalive
    - Removed org.apache.cassandra.metrics.Table.{AntiCompactedBytes|NonAntiCompactedBytes} metrics.
    - Removed following properties from cassandra.yaml: concurrent_reads, concurrent_writes,
      concurrent_counter_writes, concurrent_materialized_view_writes, streaming_socket_timeout_in_ms


4.0
===

New features
------------
   - SSTableDump now supports the -l option to output each partition as its own json object
     See CASSANDRA-13848 for more detail
   - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
     See CASSANDRA-13132
   - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
     See CASSANDRA-11936
   - Support for arithmetic operations on number has been added. See CASSANDRA-11935
   - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
     consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
   - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
   - The initial build of materialized views can be parallelized. The number of concurrent builder
     threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
     This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.

Upgrading
---------
    - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables
      have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11.
      Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL.
      Syntax for creating compact tables is also deprecated.
    - Support for legacy auth tables in the system_auth keyspace (users,
      permissions, credentials) and the migration code has been removed. Migration
      of these legacy auth tables must have been completed before the upgrade to
      4.0 and the legacy tables must have been removed. See the 'Upgrading' section
      for version 2.2 for migration instructions.
    - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst
      Tother things, this imply the removal of all yaml option related to thrift
      ('start_rpc', rpc_port, ...).
    - Cassandra 4.0 removed support for any pre-3.0 format. This means you
      cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to
      a 3.0.x/3.x version first (and run upgradesstable). In particular, this
      mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
      will need to upgrade those sstable in 3.0.x/3.x first.
    - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous
      versions will causes issues during rolling upgrades (CASSANDRA-13274).
    - Cassandra will no longer allow invalid keyspace replication options, such
      as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
      add new nodes to a datacenter before they can set set ALTER or CREATE
      keyspace replication policies using that datacenter. Existing keyspaces
      will continue to operate, but CREATE and ALTER will validate that all
      datacenters specified exist in the cluster.
    - Cassandra 4.0 fixes a problem with incremental repair which caused repaired
      data to be inconsistent between nodes. The fix changes the behavior of both
      full and incremental repairs. For full repairs, data is no longer marked
      repaired. For incremental repairs, anticompaction is run at the beginning
      of the repair, instead of at the end. If incremental repair was being used
      prior to upgrading, a full repair should be run after upgrading to resolve
      any inconsistencies.
    - Config option index_interval has been removed (it was deprecated since 2.0)
    - Deprecated repair JMX APIs are removed.
    - The version of snappy-java has been upgraded to 1.1.2.6
    - the miniumum value for internode message timeouts is 10ms. Previously, any
      positive value was allowed. See cassandra.yaml entries like
      read_request_timeout_in_ms for more details.
    - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's
      documentation has been incorrect and the setting itself near useless.
      Batch mode remains a valid commit log mode, however.
    - There is a new commit log mode, group, which is similar to batch mode
      but blocks for up to a configurable number of milliseconds between disk flushes.
    - Due to the parallelization of the initial build of materialized views,
      the per token range view building status is stored in the new table
      `system.view_builds_in_progress`. The old table `system.views_builds_in_progress`
      is no longer used and can be removed. See CASSANDRA-12245 for more details.
    - nodetool clearsnapshot now required the --all flag to remove all snapshots.
      Previous behavior would delete all snapshots by default.

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and
      read_repair_chance table options have been deprecated, and will be removed entirely in 4.0.
      See CASSANDRA-13910 for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details


3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).


Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

3.10
====

New features
------------
   - New `DurationType` (cql duration). See CASSANDRA-11873
   - Runtime modification of concurrent_compactors is now available via nodetool
   - Support for the assignment operators +=/-= has been added for update queries.
   - An Index implementation may now provide a task which runs prior to joining
     the ring. See CASSANDRA-12039
   - Filtering on partition key columns is now also supported for queries without
     secondary indexes.
   - A slow query log has been added: slow queries will be logged at DEBUG level.
     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
     in cassandra.yaml.
   - Support for GROUP BY queries has been added.
   - A new compaction-stress tool has been added to test the throughput of compaction
     for any cassandra-stress user schema.  see compaction-stress help for how to use.
   - Compaction can now take into account overlapping tables that don't take part
     in the compaction to look for deleted or overwritten data in the compacted tables.
     Then such data is found, it can be safely discarded, which in turn should enable
     the removal of tombstones over that data.

     The behavior can be engaged in two ways:
       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
         single-table compaction on all sstables to discard deleted data in one step.
       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
         which uses overlapping tables as a source of deletions/overwrites during all
         compactions.
     The argument specifies the granularity at which deleted data is to be found:
       - If ROW is specified, only whole deleted rows (or sets of rows) will be
         discarded.
       - If CELL is specified, any columns whose value is overwritten or deleted
         will also be discarded.
       - NONE (default) specifies the old behavior, overlapping tables are not used to
         decide when to discard data.
     Which option to use depends on your workload, both ROW and CELL increase the
     disk load on compaction (especially with the size-tiered compaction strategy),
     with CELL being more resource-intensive. Both should lead to better read
     performance if deleting rows (resp. overwriting or deleting cells) is common.
   - Prepared statements are now persisted in the table prepared_statements in
     the system keyspace. Upon startup, this table is used to preload all
     previously prepared statements - i.e. in many cases clients do not need to
     re-prepare statements against restarted nodes.
   - cqlsh can now connect to older Cassandra versions by downgrading the native
     protocol version. Please note that this is currently not part of our release
     testing and, as a consequence, it is not guaranteed to work in all cases.
     See CASSANDRA-12150 for more details.
   - Snapshots that are automatically taken before a table is dropped or truncated
     will have a "dropped" or "truncated" prefix on their snapshot tag name.
   - Metrics are exposed for successful and failed authentication attempts.
     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
     See CASSANDRA-11424 for details
   - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0.
   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.

Upgrading
---------
    - Support for alter types of already defined tables and of UDTs fields has been disabled.
      If it is necessary to return a different type, please use casting instead. See
      CASSANDRA-12443 for more details.
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
    - Only Java and JavaScript are now supported UDF languages.
      The sandbox in 3.0 already prevented the use of script languages except Java
      and JavaScript.
    - Compaction now correctly drops sstables out of CompactionTask when there
      isn't enough disk space to perform the full compaction.  This should reduce
      pending compaction tasks on systems with little remaining disk space.
    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the
      "full" request time on the coordinator.  Previously, they only covered the time from
      when the coordinator sent a message to a replica until the time that the replica
      responded.  Additionally, the previous behavior was to reset the timeout when performing
      a read repair, making a second read to fix a short read, and when subranges were read
      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
      is no longer reset for these "subqueries".  The entire request must complete within
      the specified timeout.  As a consequence, your timeouts may need to be adjusted
      to account for this.  See CASSANDRA-12256 for more details.
    - Logs written to stdout are now consistent with logs written to files.
      Time is now local (it was UTC on the console and local in files). Date, thread, file
      and line info where added to stdout. (see CASSANDRA-12004)
    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided.
      The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on
      that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided
      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable
      accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues).
    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like
      commit log replay. This may break existing 3rd party tools and clients. In order to startup
      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or
      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
      snitch, encryption context. Client initialization just applies the configuration but does not
      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are
      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
    - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from
      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms
      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property.
    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format
      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).

3.8
===

New features
------------
   - Shared pool threads are now named according to the stage they are executing
     tasks for. Thread names mentioned in traced queries change accordingly.
   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
     that forces a scheduled rate of operations/sec over time. Using this, stress can
     accurately account for coordinated ommission from the stress process.
   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
     "-log hdrfile=" option. This histogram includes response/service/wait times when used with the
     fixed or throttle rate options.  The histogram file can be plotted on
     http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
     to time series compaction and new tables should use this instead of DTCS. See
     CASSANDRA-9666 for details.
   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and
     a brief explanation of on-disk locations for archived data in CommitLog form. This can
     be enabled via ALTER TABLE ... WITH cdc=true.
     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
     between unflushed CommitLogSegments and cdc_raw.
     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.

Upgrading
---------
   - The ReversedType behaviour has been corrected for clustering columns of
     BYTES type containing empty value. Scrub should be run on the existing
     SSTables containing a descending clustering column of BYTES type to correct
     their ordering. See CASSANDRA-12127 for more details.
   - Ec2MultiRegionSnitch will no longer automatically set broadcast_rpc_address
     to the public instance IP if this property is defined on cassandra.yaml.
   - The name "json" and "distinct" are not valid anymore a user-defined function
     names (they are still valid as column name however). In the unlikely case where
     you had defined functions with such names, you will need to recreate
     those under a different name, change your code to use the new names and
     drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 for more
     details).

Deprecation
-----------
   - DateTieredCompactionStrategy has been deprecated - new tables should use
     TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table to TWCS might
     cause increased compaction load for a while after the migration so make sure you run
     tests before migrating. Read CASSANDRA-9666 for background on this.

3.7
===

Upgrading
---------
   - A maximum size for SSTables values has been introduced, to prevent out of memory
     exceptions when reading corrupt SSTables. This maximum size can be set via
     max_value_size_in_mb in cassandra.yaml. The default is 256MB, which matches the default
     value of native_transport_max_frame_size_in_mb. SSTables will be considered corrupt if
     they contain values whose size exceeds this limit. See CASSANDRA-9530 for more details.


3.6
=====

New features
------------
   - JMX connections can now use the same auth mechanisms as CQL clients. New options
     in cassandra-env.(sh|ps1) enable JMX authentication and authorization to be delegated
     to the IAuthenticator and IAuthorizer configured in cassandra.yaml. The default settings
     still only expose JMX locally, and use the JVM's own security mechanisms when remote
     connections are permitted. For more details on how to enable the new options, see the
     comments in cassandra-env.sh. A new class of IResource, JMXResource, is provided for
     the purposes of GRANT/REVOKE via CQL. See CASSANDRA-10091 for more details.
     Also, directly setting JMX remote port via the com.sun.management.jmxremote.port system
     property at startup is deprecated. See CASSANDRA-11725 for more details.
   - JSON timestamps are now in UTC and contain the timezone information, see CASSANDRA-11137 for more details.
   - Collision checks are performed when joining the token ring, regardless of whether
     the node should bootstrap. Additionally, replace_address can legitimately be used
     without bootstrapping to help with recovery of nodes with partially failed disks.
     See CASSANDRA-10134 for more details.
   - Key cache will only hold indexed entries up to the size configured by
     column_index_cache_size_in_kb in cassandra.yaml in memory. Larger indexed entries
     will never go into memory. See CASSANDRA-11206 for more details.
   - For tables having a default_time_to_live specifying a TTL of 0 will remove the TTL
     from the inserted or updated values.
   - Startup is now aborted if corrupted transaction log files are found. The details
     of the affected log files are now logged, allowing the operator to decide how
     to resolve the situation.
   - Filtering expressions are made more pluggable and can be added programatically via
     a QueryHandler implementation. See CASSANDRA-11295 for more details.



3.4
===

New features
------------
    - Internal authentication now supports caching of encrypted credentials.
      Reference cassandra.yaml:credentials_validity_in_ms
    - Remote configuration of auth caches via JMX can be disabled using the
      the system property cassandra.disable_auth_caches_remote_configuration
    - sstabledump tool is added to be 3.0 version of former sstable2json. The tool only
      supports v3.0+ SSTables. See tool's help for more detail.

Upgrading
---------
    - Nothing specific to 3.4 but please see previous versions upgrading section,
      especially if you are upgrading from 2.2.

Deprecation
-----------
    - The mbean interfaces org.apache.cassandra.auth.PermissionsCacheMBean and
      org.apache.cassandra.auth.RolesCacheMBean are deprecated in favor of
      org.apache.cassandra.auth.AuthCacheMBean. This generalized interface is
      common across all caches in the auth subsystem. The specific mbean interfaces
      for each individual cache will be removed in a subsequent major version.


3.2
===

New features
------------
   - We now make sure that a token does not exist in several data directories. This
     means that we run one compaction strategy per data_file_directory and we use
     one thread per directory to flush. Use nodetool relocatesstables to make sure your
     tokens are in the correct place, or just wait and compaction will handle it. See
     CASSANDRA-6696 for more details.
   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
     tunable via cassandra.commitlog_max_outstanding_replay_bytes
   - Support for type casting has been added to the selection clause.
   - Hinted handoff now supports compression. Reference cassandra.yaml:hints_compression.
     Note: hints compression is currently disabled by default.

Upgrading
---------
   - The compression ratio metrics computation has been modified to be more accurate.
   - Running Cassandra as root is prevented by default.
   - JVM options are moved from cassandra-env.(sh|ps1) to jvm.options file

Deprecation
-----------
   - The Thrift API is deprecated and will be removed in Cassandra 4.0.


3.1
=====

Upgrading
---------
   - The return value of SelectStatement::getLimit as been changed from DataLimits
     to int.
   - Custom index implementation should be aware that the method Indexer::indexes()
     has been removed as its contract was misleading and all custom implementation
     should have almost surely returned true inconditionally for that method.
   - GC logging is now enabled by default (you can disable it in the jvm.options
     file if you prefer).

TinkerPop changes for 6.0.1

A list of DataStax Enterprise 6.0.1 production-certified enhancements to TinkerPop 3.2.8.

DataStax Enterprise (DSE) 6.0.1 includes these production-certified enhancements to TinkerPop 3.2.9:
  • Performance enhancement to Bytecode deserialization. (TINKERPOP-1936)
  • Path history isn't preserved for keys in mutations. (TINKERPOP-1947)
  • Traversal construction performance enhancements (TINKERPOP-1950)
  • Bump to Groovy 2.4.15 - resolves a Groovy bug preventing Lambda creation in GLVs in some cases. (TINKERPOP-1953)

DSE 6.0.0 release notes

dse.yaml

The location of the dse.yaml file depends on the type of installation:
Package installations /etc/dse/dse.yaml
Tarball installations installation_location/resources/dse/conf/dse.yaml

logback.xml

The location of the logback.xml file depends on the type of installation:
Package installations /etc/dse/cassandra/logback.xml
Tarball installations installation_location/resources/cassandra/conf/logback.xml

cassandra.yaml

The location of the cassandra.yaml file depends on the type of installation:
Package installations /etc/dse/cassandra/cassandra.yaml
Tarball installations installation_location/resources/cassandra/conf/cassandra.yaml

17 April 2018

Warning: DSE 6.0.0 Do not use TTL (time-to-live) with DSE Search live indexing (RT indexing). To use these features together, upgrade to DSE 6.0.1. (DSP-16038)

6.0.0 Components

  • Apache Solr™ 6.0.1.1.2234
  • Apache Spark™ 2.2.0.14
  • Apache Tomcat® 8.0.47
  • DSE Java Driver 1.6.5
  • Netty 4.1.13.11.dse
  • Spark Jobserver 0.8.0.44 (DSE custom version)
  • TinkerPop 3.3.2 with additional production-certified changes

DSE 6.0 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

6.0 New features

See DataStax Enterprise 6.0 new features.

6.0.0 DSE core

Experimental features. These features are experimental and are not supported for production:
Known issues:

Changes and enhancements:

Important: Support for Thrift-compatible tables (COMPACT STORAGE) is dropped. Before upgrading to DSE 6.0, you must migrate all tables that have COMPACT STORAGE to CQL table format.

Upgrades from DSE 5.0.x or DSE 5.1.x with Thrift-compatible tables require DSE 5.1.6 or later or DSE 5.0.12 or later.

Resolved issues:
  • Warn when the cassandra superuser logs in. (DB-104)
  • Prevent multiple serializations of mutation. (DB-370)
  • Internal implementation of paging by bytes. (DB-414)
  • Connection refused should be logged less frequently. (DB-455)
  • Refactor messaging service code. (DB-497)
  • Change protocol to allow sending keyspace independent of query string. (DB-600)
  • Add result set metadata to prepared statement MD5 hash calculation. (DB-608)
  • Fix LWT asserts for immutable TableMetadat.a (DB-728)
  • MigrationManager should use toDebugString() when logging TableMetadata. (DB-739)
  • Create administrator roles who can carry out everyday administrative tasks without having unnecessary access to data. (DB-757)
  • When repairing Paxos commits, only block on nodes are being repaired. (DB-761)
  • Allow to grant/revoke multiple permissions in one statement (DB-792)
  • SystemKeyspace.snapshotOnVersionChange() never called in production code. (DB-797)
  • Error in counting iterated SSTables when choosing whether to defrag in timestamp ordered path. (DB-1018)
  • Check for mismatched versions when answering schema pulls. (DB-1026)
  • Expose ports (storage, native protocol, JMX) in system local and peers tables. (DB-1040)
  • Rename ITrigger interface method from augment to augmentNonBlocking. (DB-1046)
  • Load mapped buffer into physical memory after mlocking it for MemoryOnlyStrategy. (DB-1052)
  • New STARTUP message parameters identify clients. (DB-1054)
  • Emit client warning when a GRANT/REVOKE/RESTRICT/UNRESTRICT command has no effect. (DB-1083)
  • Update bundled Python driver to 2.2.0.post0-d075d57. (DB-1152)
  • Forbid advancing KeyScanningIterator before exhausting or closing the current iterator. (DB-1199)
  • Ensure that empty clusterings with kind==CLUSTERING are Clustering.EMPTY. (DB-1248)
  • New nodetool abortrebuild command stops a currently running rebuild operation. (DB-1234)
  • Batchlog replays do not leverage remote coordinators. (DB-1337)
  • Avoid copying EMPTY_STATIC_ROW to heap again with offheap memtable. (DB-1375)
  • Allow DiskBoundaryManager to cache different directories. (DB-1454)
  • Abort repair when there is only one node. (DB-1511)
  • OutOfMemory during view update. (DB-1493)
  • Drop response on view lock acquisition timeout and add ViewLockAcquisitionTimeouts metric. (DB-1522)
  • Handle race condition on dropping keyspace and opening keyspace. (DB-1570)
  • dsetool ring prints ERROR when data_file_directories is removed from cassandra.yaml. (DSP-13547)
  • Driver: Jackson-databind is vulnerable to remote code execution (RCE) attacks. (DSP-13498)
  • LDAP library issue. (DSP-15927)

6.0.0 DSE Advanced Replication

Changes and enhancements:
  • Support for DSE Advanced Replication V1 is removed. For V1 installations, you must first upgrade to DSE 5.1.x and migrate your DSE Advanced Replication to V2, and then upgrade to DSE 6.0. (DSP-13376)
  • Enhanced CLI security prevents injection attacks and sanitizes and validates the command line inputs. (DSP-13682)
Resolved issues:
  • Improve logging on unsupported operation failure and remove the failed mutation from replog. (DSP-15043)
  • Channel creation fails with NPE when using mixed case destination name. (DSP-15538)

6.0.0 DSE Analytics

Experimental features. These features are experimental and are not supported for production:
Known issues:
  • DSE Analytics: Additional configuration is required when enabling context-per-jvm in the Spark Jobserver. (DSP-15163)
Changes and enhancements:
  • Previously deprecated environment variables, including SPARK_CLASSPATH, are removed in Spark 2.2.0. (DSP-8379)
  • AlwaysOn SQL service, a HA (highly available) Spark SQL Thrift server. (DSP-10996)
    • JPS is an option required for nodes with AlwaysON SQL enabled.
    • The spark_config_settings and hive_config_settings are removed from dse.yaml. The configuration is provided in the spark-alwayson-sql.conf file in DSEHOME/resources/spark/conf with the same default contents as DSEHOME/resources/spark/conf/spark-defaults.conf. (DSP-15837)
  • Cassandra File System (CFS) is removed. Use DSEFS instead. Before upgrading to DSE 6.0, remove CFS keyspaces. See the From CFS to DSEFS dev blog post. (DSP-12470)
  • Optimization for SearchAnalytics with SELECT COUNT(*) and no predicates. (DSP-12669)
  • Authenticate JDBC users to Spark SQL Thrift Server. Queries that are executed during JDBC session are run as the user who authenticated through JDBC. (DSP-13395)
  • Solr optimization is automatic; spark.sql.dse.solr.enabled is deprecated, use spark.sql.dse.search.enableOptimization instead. (DSP-13398)
  • Optimization for SearchAnalytics with SELECT COUNT(*) and no predicates. (DSP-13398)
  • dse spark-beeline command is removed, use dse beeline instead. (DSP-13468)
  • cfs-stress tool is replaced by fs-stress tool. (DSP-13549)
  • Encryption for data stored on the server and encryption of Spark spill files is supported. (DSP-13841)
  • Improved security with Spark. (DSP-13991)
  • Spark local applications no longer use /var/lib/spark/rdd, instead configure and use .sparkdirectory for processes started by the user. (DSP-14380)
  • Input metrics are not thread-safe and are not used properly in CassandraJoinRDD and CassandraLeftJoinRDD. (DSP-14569)
  • AlwaysOn SQL workpool option adds high availability (HA) for the JDBC or ODBC connections for analytics node. (DSP-14719)
  • CFS is removed. Before upgrade, move HiveMetaStore from CFS to DSEFS and update URL references. (DSP-14831)
  • Include SPARK-21494 to use correct app id when authenticating to external service. (DSP-14140)
  • Upgrade to DSE 6.0 must be complete on all nodes in the cluster before Spark Worker and Spark Master will start. (DSP-14735)
  • Spark Cassandra Connector in DSE 6.0.0, has the following changes:
    • Changes to default values: spark.output.concurrent.writes: 100, spark.task.maxFailures: 10 (DSP-15164)
    • spark.cassandra.connection.connections_per_executor_max is removed; use new properties spark.cassandra.connection.local_connections_per_executor, spark.cassandra.connection.remote_connections_per_executor_min, and spark.cassandra.connection.remote_connections_per_executor_max. (DSP-15193
    • All Spark-related parameters are now camelCase. Parameters are case-sensitive. The snake_versions are automatically translated to the camelCaseVersions except when the parameters are used as table options. In SparkSQL and with spark.read.options(...), the parameters are case-insensitive because of internal SQL implementation.
    • The DSE artifact is com.datastax.dse : spark-connector: 6.0.0.
    • The DseSparkDependencies JAR is still required. (DSP-15694)
  • Use NodeSync (continuous repair) and LOCAL_QUORUM for reading from Spark recovery storage. (DSP-15219)
    Supporting changes:
    • Spark Master will not start until LOCAL_QUORUM is achieved for dse_analytics keyspace.
    • Spark Master recovery data is first attempted to be updated with LOCAL_QUORUM, and if that fails, then attempt with LOCAL_ONE. Recovery data are always queried with LOCAL_QUORUM (unlike previous versions of DSE where we used LOCAL_ONE)
    • DSE Analytics internal data moved from spark_system to dse_analytics keyspace.
    Note: DataStax strongly recommends enabling NodeSync for continuous repair on all tables in the dse_analytics keyspace. NodeSync is required on the rm_shared_data keyspace that stores Spark recovery information.
Resolved issues:
  • DSE does not work with Spark Crypto based encryption. (DSP-14140)

6.0.0 DSEFS

Changes and enhancements:
  • Wildcard characters are supported in DSEFS shell commands. (DSP-10583)
  • DSEFS should support all DSE authentication schemes. (DSP-12956)
  • Improved authorization security sets the default permission to 755 for directories and 644 for files. New DSEFS clusters create the root directory / with 755 permission to prevent non-super users from modifying root content; for example, by using mkdir or put commands. (DSP-13609)
  • Enable SSL for DSEFS encryption. (DSP-13771)
  • HTTP communication logging level changed from DEBUG to TRACE. (DSP-14400)
  • DSEFS shell history has been moved to ~/.dse/dsefs_history. (DSP-15070)
  • New tool to move hive metastore from CFS to DSEFS and update references.
  • Add echo command to DSEFS shell. (DSP-15446)
  • Changes in dse.yaml for advanced DSEFS settings.
  • Alternatives wildcards are Hadoop compatible. (DSP-15249)

6.0.0 DSE Graph

Known issues:
  • Dropping a property of vertex label with materialized view (MV) indices breaks graph. To drop a property key for a vertex label that has a materialized view index, additional steps are required to prevent data loss or cluster errors. See Dropping graph schema. (DSP-15532)
  • Secondary indexes used for DSE Graph queries have higher latency in DSE 6.0 than in the previous version. (DB-1928)
Changes and enhancements:
  • Improved and simplified data batch loading of pre-formatted data. (DGL-235)

    Supporting changes:

    • Schema discovery and schema generation are deprecated. (DGL-246)
    • Standard vertex IDs are deprecated. Use custom vertex IDs instead. (DSP-13485)
    • Standard IDs are deprecated. (DGL-247)
    • Transformations are deprecated. (DGL-248)
  • Schema API changes: all .remove() methods are renamed to .drop() and schema.clear() is renamed to schema.drop(). Schema API supports removing vertex/edge labels and property keys. Unify use of drop | remove | clear in the Schema API and use .drop() everywhere. (DSP-8385, DSP-14150)
  • Include materialized view (MV) indexes in query optimizer only if the MV was fully built. (DSP-10219)
  • DSE profiling of graph statements from the gremlin shell. (DSP-13484)
  • Improve Graph OLAP performance by smart routing query to DseGraphFrame engine with DseGraphFrameInterceptorStrategy. (DSP-13489)
  • OSS TinkerPop 3.3 supports Spark 2.2. (DSP-13632)
  • Partitioned vertex tables (PVT) are removed. (DSP-13676)
  • Graph online analytical processing (OLAP) supports drop() with DseGraphFrame interceptor. Simple queries can be used in drop operations. (DSP-13998)
  • DSE Graphs vertices and edges tables are accessible from SparkSQL and automated to dse_graph SparkSQL database. (DSP-12046)
  • More Gremlin APIs are supported in DSEGraphFrames: dedup, sort, limit, filter, as()/select(), or(). (DSP-13649)
  • Some graph and gremlin_server properties in earlier versions of DSE are no longer required for DSE 6.0. The default settings from the earlier versions of dse.yaml are preserved. These settings were removed from dse.yaml.
    • adjacency_cache_clean_rate
    • adjacency_cache_max_entry_size_in_mb
    • adjacency_cache_size_in_mb
    • gremlin_server_enabled
    • index_cache_clean_rate
    • index_cache_max_entry_size_in_mb
    • schema_mode - default schema_mode is production
    • window_size
    • ids (all vertex ID assignment and partitioning strategy options)
    • various gremlin_server settings
    If these properties exist in the dse.yaml file after upgrading to DSE 6.0, logs display warnings. You can ignore these warnings or modify dse.yaml so that only the required graph system level and gremlin_server properties are present. (DSP-14308)
  • Spark Jobserver is the DSE custom version 0.8.0.44. Applications must use the compatible Spark Jobserver API in DataStax repository. (DSP-14152)
  • Edge label names and property key names allow only [a-zA-Z0-9], underscore, hyphen, and period. The string formatting for vertices with text custom IDs has changed. (DSP-14710)
    Supporting changes (DSP-15167):
    • schema.describe() displays the entire schema, even if it contains illegal names.
    • In-place upgrades allow existing schemas with invalid edge label names and property key names.
    • Schema elements with illegal names cannot be uploaded or added.
  • Invoking toString on a custom vertex ID containing a text property, or on an edge ID that is incident upon a vertex with a custom vertex ID, now returns a value that encloses the text property value in double quotation marks and escapes the value's internal double-quotes. This change protects older formats from irresolvable parsing ambiguity. For example:
    // old
                {~label=v, x=foo}
                {~label=w, x=a"b}
                // new
                {~label=v, x="foo"}
                {~label=w, x="a""b"}
  • Support for math()-step (math) to enable scientific calculator functionality within Gremlin. (DSP-14786)
  • The GraphQueryThreads JMX attribute has been removed. Thread selection occurs with Thread Per Core (TPC) asynchronous request processing architecture. (DSP-15222)
Resolved issues:
  • spark.sql.hive.metastore.barrierPrefixes is set to org.apache.spark.sql.cassandra to properly use CassandraConnector in DSE HiveMetastore implementation. (DSP-14120)
  • Intermittent KryoException: Buffer underflow error when running order by query in OLTP mode. (DSP-12694)
  • DseGraphFrame does not support infix and() and or(). (DSP-12013)
  • Graph could be left in an inconsistent state if a schema migration fails. (DSP-15532)
  • DseGraphFrames properties().count() step return vertex count instead of multi properties count. (DSP-15049)
  • GraphSON parsing error prevents proper type detection under certain conditions. (DSP-14066)

6.0.0 DSE Search

Experimental features. These features are experimental and are not supported for production:
  • The dsetool index_checks use an Apache Lucene® experimental feature.
Known issues:
  • Search index TTL Expiration thread loops without effect with live indexing (RT indexing). (DSP-16038)
Changes and enhancements:
  • Writes are flushed to disk in segments that use a new Lucene codec that does not exist in earlier versions. Unique key values are no longer stored as both docValues and Lucene stored fields. The unique key values are now stored only as docValues in a new codec to store managed fields like Lucene. Downgrades to versions earlier than DSE 6.0 are not supported. (DSP-8465)
  • Document inserts and updates using HTTP are removed. Before upgrading, ensure you are using CQL for all inserts and updates. (DSP-9725).
  • DSENRTCachingDirectoryFactory is removed. Before upgrading, change your search index config. (DSP-10126)
  • The <dataDir> parameter in the solrconfig.xml file is not supported. Instead, follow the steps in Set the location of search indexes. (DSP-13199)
  • Improved performance by early termination of sorting. Ideal for queries that need only a few results returned, from a large number of total matches. (DSP-13253)
  • Native CQL syntax for search queries. (DSP-13411)
  • Delete by id is removed. Delete by query no longer accepts wildcard queries, including queries that match all documents (for example, <delete><query>*:*</query></delete>). Instead, use CQL to DELETE by Primary Key or the TRUNCATE command. (DSP-13436)
  • Search index config changes. (DSP-14137)
    • mergePolicy, maxMergeDocs, and mergeFactor are no longer supported.
    • RAM buffer size settings are no longer required in search index config. Global RAM buffer usage in Lucene is governed by the memtable size limits in cassandra.yaml. RAM buffers are counted toward the memtable_heap_space_in_mb.
  • dsetool core_indexing_status --progress option is always true. (DSP-13465)
  • The HTTP API for Solr core management is removed. Instead, use CQL commands for search index management or dsetool search index commands. (DSP-15350)
  • The Tika functionality bundled with Apache Solr is removed. Instead, use the stand-alone Apache Tika project. (DSP-13892)
  • Logging configuration improvements. (DSP-14137)
    • The solrvalidation.log is removed. You can safely remove appender SolrValidationErrorAppender and the logger SolrValidationErrorLogger from logback.xml. Indexing errors manifest as:
      • failures at the coordinator if they represent failures that might succeed at some later point in time using the hint replay mechanism
      • as messages in the system.log if the failures are due to non-recoverable indexing validation errors (for data that is written to the database, but not indexed properly)
  • The DSE custom update request processor (URP) implementation is deprecated. Use the field input/output (FIT) transformer API instead. (DSP-14360)
  • The stored flag in search index schemas is deprecated and is no longer added to auto-generated schemas. If the flag exists in custom schemas, it is ignored. (DSP-14425)
  • Tuning improvements for indexing. (DSP-14785)
    • Indexing is no longer asynchronous. Document updates are written to the Lucene RAM buffer synchronously with the mutation backing table.
    • See Configuring and tuning indexing performance.
    • The back_pressure_threshold_per_core in dse.yaml affects only index rebuilding/reindexing.
    • These options in dse.yaml are removed:
      • enable_back_pressure_adaptive_nrt_commit
      • max_solr_concurrency_per_core
      • solr_indexing_error_log_options
      DSE 6.0 will not start with these options present.
  • StallMetrics MBean is removed. Before upgrading to DSE 6.0, change operators that use the MBean. (DSP-14860)
  • Optimize Paging when limit is smaller than the page size. (DSP-15207)
Resolved issues includes all bug fixes up to DSE 5.1.8. Additional 6.0.0 fixes:
  • Isolate Solr Resource Loading at startup to the Local DC. (DSP-10911)

DataStax Studio 6.0.0

  • For use with DSE 6.0.x, DataStax Studio 6.0.0 is installed as a standalone tool. (DSP-13999, DSP-15623)

For details, see DataStax Studio 6.0.0 release notes.

DataStax Bulk Loader 1.0.1

  • DataStax Bulk Loader (dsbulk) version 1.0.1 is automatically installed with DataStax Enterprise 6.0.0, and can also be installed as a standalone tool. (DSP-13999, DSP-15623)
For details, see DataStax Bulk Loader 1.0.1 release notes.

Cassandra enhancements for DSE 6.0

DataStax Enterprise 6.0.0 is compatible with Apache Cassandra™ 3.11 and adds production-certified enhancements.

DataStax Enterprise 6.0.0 is compatible with Apache Cassandra™ 3.11 and adds these production-certified enhancements:

  • Add DEFAULT, UNSET, MBEAN and MBEANS to `ReservedKeywords`. (CASSANDRA-14205)
  • Add Unittest for schema migration fix (CASSANDRA-14140)
  • Print correct snitch info from nodetool describecluster (CASSANDRA-13528)
  • Close socket on error during connect on OutboundTcpConnection (CASSANDRA-9630)
  • Enable CDC unittest (CASSANDRA-14141)
  • Split CommitLogStressTest to avoid timeout (CASSANDRA-14143)
  • Improve commit log chain marker updating (CASSANDRA-14108)
  • Fix updating base table rows with TTL not removing view entries (CASSANDRA-14071)
  • Reduce garbage created by DynamicSnitch (CASSANDRA-14091)
  • More frequent commitlog chained markers (CASSANDRA-13987)
  • RPM package spec: fix permissions for installed jars and config files (CASSANDRA-14181)
  • More PEP8 compiance for cqlsh (CASSANDRA-14021)
  • Fix support for SuperColumn tables (CASSANDRA-12373)
  • Fix missing original update in TriggerExecutor (CASSANDRA-13894)
  • Improve short read protection performance (CASSANDRA-13794)
  • Fix counter application order in short read protection (CASSANDRA-12872)
  • Fix MV timestamp issues (CASSANDRA-11500)
  • Fix AssertionError in short read protection (CASSANDRA-13747)
  • Gossip thread slows down when using batch commit log (CASSANDRA-12966)
  • Allow native function calls in CQLSSTableWriter (CASSANDRA-12606)
  • Copy session properties on cqlsh.py do_login (CASSANDRA-13847)
  • Fix load over calculated issue in IndexSummaryRedistribution (CASSANDRA-13738)
  • Obfuscate password in stress-graphs (CASSANDRA-12233)
  • ReverseIndexedReader may drop rows during 2.1 to 3.0 upgrade (CASSANDRA-13525)
  • Avoid reading static row twice from old format sstables (CASSANDRA-13236)
  • Fix possible NPE on upgrade to 3.0/3.X in case of IO errors (CASSANDRA-13389)
  • Add duration data type (CASSANDRA-11873)
  • Properly report LWT contention (CASSANDRA-12626)
  • Stress daemon help is incorrect(CASSANDRA-12563)
  • Remove ALTER TYPE support (CASSANDRA-12443)
  • Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203)
  • Remove support for non-JavaScript UDFs (CASSANDRA-12883)
  • Better handle invalid system roles table (CASSANDRA-12700)
  • Upgrade netty version to fix memory leak with client encryption (CASSANDRA-13114)
  • Fix trivial log format error (CASSANDRA-14015)
  • Allow SSTabledump to do a JSON object per partition (CASSANDRA-13848)
  • Remove unused and deprecated methods from AbstractCompactionStrategy (CASSANDRA-14081)
  • Fix Distribution.average in cassandra-stress (CASSANDRA-14090)
  • Presize collections (CASSANDRA-13760)
  • Add GroupCommitLogService (CASSANDRA-13530)
  • Parallelize initial materialized view build (CASSANDRA-12245)
  • Fix flaky SecondaryIndexManagerTest.assert[Not]MarkedAsBuilt (CASSANDRA-13965)
  • Make LWTs send resultset metadata on every request (CASSANDRA-13992)
  • Fix flaky indexWithFailedInitializationIsNotQueryableAfterPartialRebuild (CASSANDRA-13963)
  • Introduce leaf-only iterator (CASSANDRA-9988)
  • Allow only one concurrent call to StatusLogger (CASSANDRA-12182)
  • Refactoring to specialised functional interfaces (CASSANDRA-13982)
  • Speculative retry should allow more friendly parameters (CASSANDRA-13876)
  • Throw exception if we send/receive repair messages to incompatible nodes (CASSANDRA-13944)
  • Replace usages of MessageDigest with Guava's Hasher (CASSANDRA-13291)
  • Add nodetool command to print hinted handoff window (CASSANDRA-13728)
  • Fix some alerts raised by static analysis (CASSANDRA-13799)
  • Checksum SSTable metadata (CASSANDRA-13321, CASSANDRA-13593)
  • Add result set metadata to prepared statement MD5 hash calculation (CASSANDRA-10786)
  • Add incremental repair support for --hosts, --force, and subrange repair (CASSANDRA-13818)
  • Refactor GcCompactionTest to avoid boxing (CASSANDRA-13941)
  • Expose recent histograms in JmxHistograms (CASSANDRA-13642)
  • Add SERIAL and LOCAL_SERIAL support for cassandra-stress (CASSANDRA-13925)
  • LCS needlessly checks for L0 STCS candidates multiple times (CASSANDRA-12961)
  • Correctly close netty channels when a stream session ends (CASSANDRA-13905)
  • Update lz4 to 1.4.0 (CASSANDRA-13741)
  • Throttle base partitions during MV repair streaming to prevent OOM (CASSANDRA-13299)
  • Improve short read protection performance (CASSANDRA-13794)
  • Fix AssertionError in short read protection (CASSANDRA-13747)
  • Use compaction threshold for STCS in L0 (CASSANDRA-13861)
  • Fix problem with min_compress_ratio: 1 and disallow ratio < 1 (CASSANDRA-13703)
  • Add extra information to SASI timeout exception (CASSANDRA-13677)
  • Rework CompactionStrategyManager.getScanners synchronization (CASSANDRA-13786)
  • Add additional unit tests for batch behavior, TTLs, Timestamps (CASSANDRA-13846)
  • Add keyspace and table name in schema validation exception (CASSANDRA-13845)
  • Emit metrics whenever we hit tombstone failures and warn thresholds (CASSANDRA-13771)
  • Allow changing log levels via nodetool for related classes (CASSANDRA-12696)
  • Add stress profile yaml with LWT (CASSANDRA-7960)
  • Reduce memory copies and object creations when acting on ByteBufs (CASSANDRA-13789)
  • simplify mx4j configuration (Cassandra-13578)
  • Fix trigger example on 4.0 (CASSANDRA-13796)
  • force minumum timeout value (CASSANDRA-9375)
  • Add bytes repaired/unrepaired to nodetool tablestats (CASSANDRA-13774)
  • Don't delete incremental repair sessions if they still have sstables (CASSANDRA-13758)
  • Fix pending repair manager index out of bounds check (CASSANDRA-13769)
  • Don't use RangeFetchMapCalculator when RF=1 (CASSANDRA-13576)
  • Don't optimise trivial ranges in RangeFetchMapCalculator (CASSANDRA-13664)
  • Use an ExecutorService for repair commands instead of new Thread(..).start() (CASSANDRA-13594)
  • Fix race / ref leak in anticompaction (CASSANDRA-13688)
  • Fix race / ref leak in PendingRepairManager (CASSANDRA-13751)
  • Enable ppc64le runtime as unsupported architecture (CASSANDRA-13615)
  • Improve sstablemetadata output (CASSANDRA-11483)
  • Support for migrating legacy users to roles has been dropped (CASSANDRA-13371)
  • Introduce error metrics for repair (CASSANDRA-13387)
  • Refactoring to primitive functional interfaces in AuthCache (CASSANDRA-13732)
  • Update metrics to 3.1.5 (CASSANDRA-13648)
  • batch_size_warn_threshold_in_kb can now be set at runtime (CASSANDRA-13699)
  • Avoid always rebuilding secondary indexes at startup (CASSANDRA-13725)
  • Upgrade JMH from 1.13 to 1.19 (CASSANDRA-13727)
  • Upgrade SLF4J from 1.7.7 to 1.7.25 (CASSANDRA-12996)
  • Default for start_native_transport now true if not set in config (CASSANDRA-13656)
  • Don't add localhost to the graph when calculating where to stream from (CASSANDRA-13583)
  • Allow skipping equality-restricted clustering columns in ORDER BY clause (CASSANDRA-10271)
  • Use common nowInSec for validation compactions (CASSANDRA-13671)
  • Improve handling of IR prepare failures (CASSANDRA-13672)
  • Send IR coordinator messages synchronously (CASSANDRA-13673)
  • Flush system.repair table before IR finalize promise (CASSANDRA-13660)
  • Fix column filter creation for wildcard queries (CASSANDRA-13650)
  • Add 'nodetool getbatchlogreplaythrottle' and 'nodetool setbatchlogreplaythrottle' (CASSANDRA-13614)
  • fix race condition in PendingRepairManager (CASSANDRA-13659)
  • Allow noop incremental repair state transitions (CASSANDRA-13658)
  • Run repair with down replicas (CASSANDRA-10446)
  • Added started & completed repair metrics (CASSANDRA-13598)
  • Added started & completed repair metrics (CASSANDRA-13598)
  • Improve secondary index (re)build failure and concurrency handling (CASSANDRA-10130)
  • Improve calculation of available disk space for compaction (CASSANDRA-13068)
  • Change the accessibility of RowCacheSerializer for third party row cache plugins (CASSANDRA-13579)
  • Allow sub-range repairs for a preview of repaired data (CASSANDRA-13570)
  • NPE in IR cleanup when columnfamily has no sstables (CASSANDRA-13585)
  • Fix Randomness of stress values (CASSANDRA-12744)
  • Allow selecting Map values and Set elements (CASSANDRA-7396)
  • Fast and garbage-free Streaming Histogram (CASSANDRA-13444)
  • Update repairTime for keyspaces on completion (CASSANDRA-13539)
  • Add configurable upper bound for validation executor threads (CASSANDRA-13521)
  • Bring back maxHintTTL propery (CASSANDRA-12982)
  • Add testing guidelines (CASSANDRA-13497)
  • Add more repair metrics (CASSANDRA-13531)
  • RangeStreamer should be smarter when picking endpoints for streaming (CASSANDRA-4650)
  • Avoid rewrapping an exception thrown for cache load functions (CASSANDRA-13367)
  • Log time elapsed for each incremental repair phase (CASSANDRA-13498)
  • Add multiple table operation support to cassandra-stress (CASSANDRA-8780)
  • Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262)
  • Fix WriteResponseHandlerTest is sensitive to test execution order (CASSANDRA-13421)
  • Improve incremental repair logging (CASSANDRA-13468)
  • Start compaction when incremental repair finishes (CASSANDRA-13454)
  • Add repair streaming preview (CASSANDRA-13257)
  • Cleanup isIncremental/repairedAt usage (CASSANDRA-13430)
  • Change protocol to allow sending key space independent of query string (CASSANDRA-10145)
  • Make gc_log and gc_warn settable at runtime (CASSANDRA-12661)
  • Take number of files in L0 in account when estimating remaining compaction tasks (CASSANDRA-13354)
  • Skip building views during base table streams on range movements (CASSANDRA-13065)
  • Improve error messages for +/- operations on maps and tuples (CASSANDRA-13197)
  • Remove deprecated repair JMX APIs (CASSANDRA-11530)
  • Fix version check to enable streaming keep-alive (CASSANDRA-12929)
  • Make it possible to monitor an ideal consistency level separate from actual consistency level (CASSANDRA-13289)
  • Outbound TCP connections ignore internode authenticator (CASSANDRA-13324)
  • Upgrade junit from 4.6 to 4.12 (CASSANDRA-13360)
  • Cleanup ParentRepairSession after repairs (CASSANDRA-13359)
  • Upgrade snappy-java to 1.1.2.6 (CASSANDRA-13336)
  • Incremental repair not streaming correct sstables (CASSANDRA-13328)
  • Upgrade the JNA version to 4.3.0 (CASSANDRA-13300)
  • Add the currentTimestamp, currentDate, currentTime and currentTimeUUID functions (CASSANDRA-13132)
  • Remove config option index_interval (CASSANDRA-10671)
  • Reduce lock contention for collection types and serializers (CASSANDRA-13271)
  • Make it possible to override MessagingService.Verb ids (CASSANDRA-13283)
  • Avoid synchronized on prepareForRepair in ActiveRepairService (CASSANDRA-9292)
  • Adds the ability to use uncompressed chunks in compressed files (CASSANDRA-10520)
  • Don't flush sstables when streaming for incremental repair (CASSANDRA-13226)
  • Remove unused method (CASSANDRA-13227)
  • Fix minor bugs related to #9143 (CASSANDRA-13217)
  • Output warning if user increases RF (CASSANDRA-13079)
  • Remove pre-3.0 streaming compatibility code for 4.0 (CASSANDRA-13081)
  • Add support for + and - operations on dates (CASSANDRA-11936)
  • Fix consistency of incrementally repaired data (CASSANDRA-9143)
  • Increase commitlog version (CASSANDRA-13161)
  • Make TableMetadata immutable, optimize Schema (CASSANDRA-9425)
  • Refactor ColumnCondition (CASSANDRA-12981)
  • Parallelize streaming of different keyspaces (CASSANDRA-4663)
  • Improved compactions metrics (CASSANDRA-13015)
  • Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031)
  • Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855)
  • Thrift removal (CASSANDRA-11115)
  • Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716)
  • Add column definition kind to dropped columns in schema (CASSANDRA-12705)
  • Add (automate) Nodetool Documentation (CASSANDRA-12672)
  • Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736)
  • Reject invalid replication settings when creating or altering a keyspace (CASSANDRA-12681)
  • Clean up the SSTableReader#getScanner API wrt removal of RateLimiter (CASSANDRA-12422)
  • Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
  • Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
  • Require forceful decommission if number of nodes is less than replication factor (CASSANDRA-12510)
  • Allow IN restrictions on column families with collections (CASSANDRA-12654)
  • Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028)
  • Add timeUnit Days for cassandra-stress (CASSANDRA-13029)
  • Add mutation size and batch metrics (CASSANDRA-12649)
  • Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999)
  • Expose time spent waiting in thread pool queue (CASSANDRA-8398)
  • Conditionally update index built status to avoid unnecessary flushes (CASSANDRA-12969)
  • cqlsh auto completion: refactor definition of compaction strategy options (CASSANDRA-12946)
  • Add support for arithmetic operators (CASSANDRA-11935)
  • Add histogram for delay to deliver hints (CASSANDRA-13234)
  • Fix cqlsh automatic protocol downgrade regression (CASSANDRA-13307)
  • Changing `max_hint_window_in_ms` at runtime (CASSANDRA-11720)
  • Nodetool repair can hang forever if we lose the notification for the repair completing/failing (CASSANDRA-13480)
  • Anticompaction can cause noisy log messages (CASSANDRA-13684)
  • Switch to client init for sstabledump (CASSANDRA-13683)
  • CQLSH: Don't pause when capturing data (CASSANDRA-13743)

General upgrade advice for DSE 6.0.0

General upgrade advice for DataStax Enterprise 6.0.0

General upgrade advice for DataStax Enterprise (DSE) 6.0.0:
PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

DSE 6.0
=======

Operations
----------
   - Added batchlog_endpoint_strategy option and 2 new strategies to cassandra.yaml. The default is
     "random_remote", which works as in previous versions. Two new strategies help to choose better
     endpoints and improve batchlog write latency. See cassandra.yaml for details. (DB-1367)
   - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
     The command must be executed on the node where the rebuild operation is running. Streams
     may continue until they finish or timeout.
   - Added "-Dcassandra.replace_consistency" to support streaming each range from more than one source.
     By default, node replacement only streams one copy of data which may be stale wrt to QUORUM/LOCAL_QUORUM.
     After replacing a down node with "-Dcassandra.replace_consistency=LOCAL_DC_QUORUM", replacement node will be up
     with consistent data assuming data is inserted with LOCAL_QUORUM. Note that streaming multiple copies causes more
     IO and compactions. Alternatively, operators can run repair before bootstrapping the replacement with the
     default replace_consistency=ONE.
     Available options: "ONE"(default), "LOCAL_DC_QUORUM", "GLOBAL_QUORUM".
   - Rebuild functionality now requires to specify the streaming source endpoints and/or DC/racks
     and no longer implicitly prefers the local data center. Specifying no sources or DC/racks is
     now considered an error.
     See `nodetool help rebuild` about which combinations of DCs/racks or endpoints to include or
     exclude are possible. (DB-581)
   - Nodetool repair will now run full repairs by default, use "-inc" option to run incremental repair.
   - Incremental repairs are disallowed on tables with Materialized Views or CDC. Run full repairs on these
     tables.
   - A new HINTS stage was created to handle incoming hints. The maximum size of this thread pool is governed by
     the system property cassandra.hints.max_receive_threads, which has default value max_hints_delivery_threads
     from cassandra.yaml.
   - A new BACKGROUND_IO stage was created to handle blocking background IO operations like cache loading,
     commit log segment allocation and replay. The maximum size of this thread pool is governed by the system
     property cassandra.io.background.max_pool_size.
   - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
     set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.


Upgrading
---------
   - The ITrigger interface has been modified from "augment" to "augmentNonBlocking" for non-blocking
     internal architecture (DB-1046).
   - Removed support for directly setting JMX remote port via the com.sun.management.jmxremote.port
     system property, deprecated since 3.6. See CASSANDRA-11725 and DB-1040 for more details.
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 6.0.
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. See CASSANDRA-12888 for details.
   - Changes to permissions (CassandraAuthorizer, DseAuthorizer), roles (CassandraRoleManager,
     DseRoleManager) and credentials (PasswordAuthenticator) are broadcasted throughout the cluster.
     This eliminates the need to have short-ish validity (permissions_validity_in_ms,
     roles_validity_in_ms, credentials_validity_in_ms) and update-interval
     (permissions_update_interval_in_ms, roles_update_interval_in_ms,
     credentials_update_interval_in_ms) settings in cassandra.yaml. Consider raising the values
     for *_validity_in_ms and *_update_interval_in_ms, if configured.
     Please note, that only "reachable" nodes will receive the invalidation messages and
     unreachable nodes may still hold outdated information after DCL command.
   - REVOKE (and the new UNRESTRICT statement) now error out when the statement would not change
     anything. This means, an error is raised when the statement targets permissions which are
     not present for that resource.
   - The caches for JMX permissions and credentials no longer exist and therefore the JMX beans
     for these caches have been removed, too. Also the configuration options for the credentials
     cache need to be removed from the cassandra.yaml file.

Deprecation
-----------
    - The following properties were renamed and are now deprecated before they are removed in the next major release:
      - rpc_address -> native_transport_address
      - rpc_interface -> native_transport_interface
      - rpc_interface_prefer_ipv6 -> native_transport_interface_prefer_ipv6
      - broadcast_rpc_address -> native_transport_broadcast_address
      - rpc_keepalive -> native_transport_keepalive
    - Removed org.apache.cassandra.metrics.Table.{AntiCompactedBytes|NonAntiCompactedBytes} metrics.
    - Removed following properties from cassandra.yaml: concurrent_reads, concurrent_writes,
      concurrent_counter_writes, concurrent_materialized_view_writes, streaming_socket_timeout_in_ms


4.0
===

New features
------------
   - SSTableDump now supports the -l option to output each partition as its own json object
     See CASSANDRA-13848 for more detail
   - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added.
     See CASSANDRA-13132
   - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added.
     See CASSANDRA-11936
   - Support for arithmetic operations on number has been added. See CASSANDRA-11935
   - Preview expected streaming required for a repair (nodetool repair --preview), and validate the
     consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257
   - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396
   - The initial build of materialized views can be parallelized. The number of concurrent builder
     threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`.
     This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders`
     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details.

Upgrading
---------
    - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables
      have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11.
      Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL.
      Syntax for creating compact tables is also deprecated.
    - Support for legacy auth tables in the system_auth keyspace (users,
      permissions, credentials) and the migration code has been removed. Migration
      of these legacy auth tables must have been completed before the upgrade to
      4.0 and the legacy tables must have been removed. See the 'Upgrading' section
      for version 2.2 for migration instructions.
    - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst
      Tother things, this imply the removal of all yaml option related to thrift
      ('start_rpc', rpc_port, ...).
    - Cassandra 4.0 removed support for any pre-3.0 format. This means you
      cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to
      a 3.0.x/3.x version first (and run upgradesstable). In particular, this
      mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
      will need to upgrade those sstable in 3.0.x/3.x first.
    - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous
      versions will causes issues during rolling upgrades (CASSANDRA-13274).
    - Cassandra will no longer allow invalid keyspace replication options, such
      as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
      add new nodes to a datacenter before they can set set ALTER or CREATE
      keyspace replication policies using that datacenter. Existing keyspaces
      will continue to operate, but CREATE and ALTER will validate that all
      datacenters specified exist in the cluster.
    - Cassandra 4.0 fixes a problem with incremental repair which caused repaired
      data to be inconsistent between nodes. The fix changes the behavior of both
      full and incremental repairs. For full repairs, data is no longer marked
      repaired. For incremental repairs, anticompaction is run at the beginning
      of the repair, instead of at the end. If incremental repair was being used
      prior to upgrading, a full repair should be run after upgrading to resolve
      any inconsistencies.
    - Config option index_interval has been removed (it was deprecated since 2.0)
    - Deprecated repair JMX APIs are removed.
    - The version of snappy-java has been upgraded to 1.1.2.6
    - the miniumum value for internode message timeouts is 10ms. Previously, any
      positive value was allowed. See cassandra.yaml entries like
      read_request_timeout_in_ms for more details.
    - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's
      documentation has been incorrect and the setting itself near useless.
      Batch mode remains a valid commit log mode, however.
    - There is a new commit log mode, group, which is similar to batch mode
      but blocks for up to a configurable number of milliseconds between disk flushes.
    - Due to the parallelization of the initial build of materialized views,
      the per token range view building status is stored in the new table
      `system.view_builds_in_progress`. The old table `system.views_builds_in_progress`
      is no longer used and can be removed. See CASSANDRA-12245 for more details.
    - nodetool clearsnapshot now required the --all flag to remove all snapshots.
      Previous behavior would delete all snapshots by default.

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and
      read_repair_chance table options have been deprecated, and will be removed entirely in 4.0.
      See CASSANDRA-13910 for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details


3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).


Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

3.10
====

New features
------------
   - New `DurationType` (cql duration). See CASSANDRA-11873
   - Runtime modification of concurrent_compactors is now available via nodetool
   - Support for the assignment operators +=/-= has been added for update queries.
   - An Index implementation may now provide a task which runs prior to joining
     the ring. See CASSANDRA-12039
   - Filtering on partition key columns is now also supported for queries without
     secondary indexes.
   - A slow query log has been added: slow queries will be logged at DEBUG level.
     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
     in cassandra.yaml.
   - Support for GROUP BY queries has been added.
   - A new compaction-stress tool has been added to test the throughput of compaction
     for any cassandra-stress user schema.  see compaction-stress help for how to use.
   - Compaction can now take into account overlapping tables that don't take part
     in the compaction to look for deleted or overwritten data in the compacted tables.
     Then such data is found, it can be safely discarded, which in turn should enable
     the removal of tombstones over that data.

     The behavior can be engaged in two ways:
       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
         single-table compaction on all sstables to discard deleted data in one step.
       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
         which uses overlapping tables as a source of deletions/overwrites during all
         compactions.
     The argument specifies the granularity at which deleted data is to be found:
       - If ROW is specified, only whole deleted rows (or sets of rows) will be
         discarded.
       - If CELL is specified, any columns whose value is overwritten or deleted
         will also be discarded.
       - NONE (default) specifies the old behavior, overlapping tables are not used to
         decide when to discard data.
     Which option to use depends on your workload, both ROW and CELL increase the
     disk load on compaction (especially with the size-tiered compaction strategy),
     with CELL being more resource-intensive. Both should lead to better read
     performance if deleting rows (resp. overwriting or deleting cells) is common.
   - Prepared statements are now persisted in the table prepared_statements in
     the system keyspace. Upon startup, this table is used to preload all
     previously prepared statements - i.e. in many cases clients do not need to
     re-prepare statements against restarted nodes.
   - cqlsh can now connect to older Cassandra versions by downgrading the native
     protocol version. Please note that this is currently not part of our release
     testing and, as a consequence, it is not guaranteed to work in all cases.
     See CASSANDRA-12150 for more details.
   - Snapshots that are automatically taken before a table is dropped or truncated
     will have a "dropped" or "truncated" prefix on their snapshot tag name.
   - Metrics are exposed for successful and failed authentication attempts.
     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
     See CASSANDRA-11424 for details
   - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0.
   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.

Upgrading
---------
    - Support for alter types of already defined tables and of UDTs fields has been disabled.
      If it is necessary to return a different type, please use casting instead. See
      CASSANDRA-12443 for more details.
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
    - Only Java and JavaScript are now supported UDF languages.
      The sandbox in 3.0 already prevented the use of script languages except Java
      and JavaScript.
    - Compaction now correctly drops sstables out of CompactionTask when there
      isn't enough disk space to perform the full compaction.  This should reduce
      pending compaction tasks on systems with little remaining disk space.
    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the
      "full" request time on the coordinator.  Previously, they only covered the time from
      when the coordinator sent a message to a replica until the time that the replica
      responded.  Additionally, the previous behavior was to reset the timeout when performing
      a read repair, making a second read to fix a short read, and when subranges were read
      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
      is no longer reset for these "subqueries".  The entire request must complete within
      the specified timeout.  As a consequence, your timeouts may need to be adjusted
      to account for this.  See CASSANDRA-12256 for more details.
    - Logs written to stdout are now consistent with logs written to files.
      Time is now local (it was UTC on the console and local in files). Date, thread, file
      and line info where added to stdout. (see CASSANDRA-12004)
    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided.
      The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on
      that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided
      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable
      accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues).
    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like
      commit log replay. This may break existing 3rd party tools and clients. In order to startup
      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or
      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
      snitch, encryption context. Client initialization just applies the configuration but does not
      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are
      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
    - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from
      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms
      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property.
    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format
      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).

3.8
===

New features
------------
   - Shared pool threads are now named according to the stage they are executing
     tasks for. Thread names mentioned in traced queries change accordingly.
   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
     that forces a scheduled rate of operations/sec over time. Using this, stress can
     accurately account for coordinated ommission from the stress process.
   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
     "-log hdrfile=" option. This histogram includes response/service/wait times when used with the
     fixed or throttle rate options.  The histogram file can be plotted on
     http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
     to time series compaction and new tables should use this instead of DTCS. See
     CASSANDRA-9666 for details.
   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and
     a brief explanation of on-disk locations for archived data in CommitLog form. This can
     be enabled via ALTER TABLE ... WITH cdc=true.
     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
     between unflushed CommitLogSegments and cdc_raw.
     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.

Upgrading
---------
   - The ReversedType behaviour has been corrected for clustering columns of
     BYTES type containing empty value. Scrub should be run on the existing
     SSTables containing a descending clustering column of BYTES type to correct
     their ordering. See CASSANDRA-12127 for more details.
   - Ec2MultiRegionSnitch will no longer automatically set broadcast_rpc_address
     to the public instance IP if this property is defined on cassandra.yaml.
   - The name "json" and "distinct" are not valid anymore a user-defined function
     names (they are still valid as column name however). In the unlikely case where
     you had defined functions with such names, you will need to recreate
     those under a different name, change your code to use the new names and
     drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 for more
     details).

Deprecation
-----------
   - DateTieredCompactionStrategy has been deprecated - new tables should use
     TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table to TWCS might
     cause increased compaction load for a while after the migration so make sure you run
     tests before migrating. Read CASSANDRA-9666 for background on this.

3.7
===

Upgrading
---------
   - A maximum size for SSTables values has been introduced, to prevent out of memory
     exceptions when reading corrupt SSTables. This maximum size can be set via
     max_value_size_in_mb in cassandra.yaml. The default is 256MB, which matches the default
     value of native_transport_max_frame_size_in_mb. SSTables will be considered corrupt if
     they contain values whose size exceeds this limit. See CASSANDRA-9530 for more details.


3.6
=====

New features
------------
   - JMX connections can now use the same auth mechanisms as CQL clients. New options
     in cassandra-env.(sh|ps1) enable JMX authentication and authorization to be delegated
     to the IAuthenticator and IAuthorizer configured in cassandra.yaml. The default settings
     still only expose JMX locally, and use the JVM's own security mechanisms when remote
     connections are permitted. For more details on how to enable the new options, see the
     comments in cassandra-env.sh. A new class of IResource, JMXResource, is provided for
     the purposes of GRANT/REVOKE via CQL. See CASSANDRA-10091 for more details.
     Also, directly setting JMX remote port via the com.sun.management.jmxremote.port system
     property at startup is deprecated. See CASSANDRA-11725 for more details.
   - JSON timestamps are now in UTC and contain the timezone information, see CASSANDRA-11137 for more details.
   - Collision checks are performed when joining the token ring, regardless of whether
     the node should bootstrap. Additionally, replace_address can legitimately be used
     without bootstrapping to help with recovery of nodes with partially failed disks.
     See CASSANDRA-10134 for more details.
   - Key cache will only hold indexed entries up to the size configured by
     column_index_cache_size_in_kb in cassandra.yaml in memory. Larger indexed entries
     will never go into memory. See CASSANDRA-11206 for more details.
   - For tables having a default_time_to_live specifying a TTL of 0 will remove the TTL
     from the inserted or updated values.
   - Startup is now aborted if corrupted transaction log files are found. The details
     of the affected log files are now logged, allowing the operator to decide how
     to resolve the situation.
   - Filtering expressions are made more pluggable and can be added programatically via
     a QueryHandler implementation. See CASSANDRA-11295 for more details.



3.4
===

New features
------------
    - Internal authentication now supports caching of encrypted credentials.
      Reference cassandra.yaml:credentials_validity_in_ms
    - Remote configuration of auth caches via JMX can be disabled using the
      the system property cassandra.disable_auth_caches_remote_configuration
    - sstabledump tool is added to be 3.0 version of former sstable2json. The tool only
      supports v3.0+ SSTables. See tool's help for more detail.

Upgrading
---------
    - Nothing specific to 3.4 but please see previous versions upgrading section,
      especially if you are upgrading from 2.2.

Deprecation
-----------
    - The mbean interfaces org.apache.cassandra.auth.PermissionsCacheMBean and
      org.apache.cassandra.auth.RolesCacheMBean are deprecated in favor of
      org.apache.cassandra.auth.AuthCacheMBean. This generalized interface is
      common across all caches in the auth subsystem. The specific mbean interfaces
      for each individual cache will be removed in a subsequent major version.


3.2
===

New features
------------
   - We now make sure that a token does not exist in several data directories. This
     means that we run one compaction strategy per data_file_directory and we use
     one thread per directory to flush. Use nodetool relocatesstables to make sure your
     tokens are in the correct place, or just wait and compaction will handle it. See
     CASSANDRA-6696 for more details.
   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
     tunable via cassandra.commitlog_max_outstanding_replay_bytes
   - Support for type casting has been added to the selection clause.
   - Hinted handoff now supports compression. Reference cassandra.yaml:hints_compression.
     Note: hints compression is currently disabled by default.

Upgrading
---------
   - The compression ratio metrics computation has been modified to be more accurate.
   - Running Cassandra as root is prevented by default.
   - JVM options are moved from cassandra-env.(sh|ps1) to jvm.options file

Deprecation
-----------
   - The Thrift API is deprecated and will be removed in Cassandra 4.0.


3.1
=====

Upgrading
---------
   - The return value of SelectStatement::getLimit as been changed from DataLimits
     to int.
   - Custom index implementation should be aware that the method Indexer::indexes()
     has been removed as its contract was misleading and all custom implementation
     should have almost surely returned true inconditionally for that method.
   - GC logging is now enabled by default (you can disable it in the jvm.options
     file if you prefer).

TinkerPop Changes

A list of DataStax Enterprise 6.0.0 production-certified changes in addition to TinkerPop 3.2.8.

DataStax Enterprise (DSE) 6.0.0 includes all changes from previous releases plus these production-certified changes that are in addition to TinkerPop 3.2.8:
  • Made iterate() a first class step. (TINKERPOP-1834)
  • Fixed a bug in NumberHelper that led to wrong min/max results if numbers exceeded the Integer limits. (TINKERPOP-1873)
  • Improved error messaging for failed serialization and deserialization of request/response messages.
  • Fixed bug in handling of Direction.BOTH in Messenger implementations to pass the message to the opposite side of the `StarGraph` in VertexPrograms for OLAP traversals. (TINKERPOP-1862)
  • Fixed a bug in Gremlin Console which prevented handling of gremlin.sh flags that had an equal sign (=) between the flag and its arguments. (TINKERPOP-1879)
  • Fixed a bug where SparkMessenger was not applying the edgeFunction`from MessageScope`in VertexPrograms for OLAP-based traversals. (TINKERPOP-1872)
  • TinkerPop drivers prior to 3.2.4 won't authenticate with Kerberos anymore. A long-deprecated option on the Gremlin Server protocol was removed.